Getting Started With Python Package Managers

What is a Package Manager in Python?

You are probably familiar with the concept of package managers if you are coming from other languages. JavaScript uses npm for package management, Ruby uses gem, PHP uses composer and so on.

Package Manager is a tool that allows you to manage the dependencies for your project that are not part of the Python standard library. A dependency is code that is required for your program to function properly. These often come in the form of packages.

Packages can also have their own dependencies. Managing all these dependencies can be hard because packages may require specific versions of their dependencies. It’s easy to break something by modifying dependencies manually - and to avoid these issues, a good Package Manager is required.

Standard package manager

A beginner’s guide to Pip

Package management is so important, that starting from version 3.4 for Python 3 and 2.7.9 for Python 2, Python has its own standard package manager - pip which is part of Python distribution.

How Does Pip Work

When installing packages, pip will first resolve the dependencies, check if they are already installed on the system, and, if not, install them. Once all dependencies have been satisfied, it proceeds to install the requested package(s). This all happens globally, by default, installing everything onto the machine.

One confusing aspect of installing packages with pip is that it tries to install them into you global Python environment by default. Yes it does and therefore all the new installed packages become available globally on your system, which is great for convenience. But it also causes issues if you're working with multiple projects that require different version of the same package. For example, you will definitely face version conflicts if one of your projects needs 1.0 version of package, and another requires 1.1. And it gets worse in cases when you might also have different programs that need different Python versions.

Sidebar: Package Management and Virtual Environments

The solution of these problems is to create multiple Python environments each catered to specific needs. Each of these Python environments is totally independent of other environments. with virtual environment. In python world, each of these environments is called virtual environment. A virtual environment is an isolated Python environment - a folder containing packages and other dependencies that Python project needs.

To demonstrate how it works, let’s create a virtual environment and install some package.

How To Install Python Virtual Environment

Note - For this example, we’re using Python 3.7, Windows 10, and Windows command prompt (cmd) but we have also provided you with Linux commands.

1. First, let's create a folder that'll contain our files

Windows: mkdir <folder_name> & cd <folder_name>

Linux: mkdir <folder_name> && cd <folder_name>

2. Make sure you’ve got Python & pip.

On Windows:

python -V & pip -V

On Linux:

python3 -V && pip -V

3. Now let’s create and activate virtual environment

Windows: 

python -m venv <environment_name>
source <environment_name>\Scripts\activate

Linux: 

python3 -m venv <environment_name>
source <environment_name>/Scripts/activate

Running the activate command configures your current shell session to use the Python and pip commands from my_venv  environment. Have you noticed how it changed command prompt with (my_venv) ?

4. Next up, let’s install some popular package for sending requests

Windows, Linux:

pip install requests

You should see an output similar to the one above. You can also see that the current environment is using pip version 10.0.1, but version 20.2.4 is available. It also shows the command you should use to update pip, so let’s do that:

On Windows, Linux:

python -m pip install --upgrade pip

Now that we have installed requests and upgraded pip, we can use the list command to see the packages installed in our environment:

On Windows, Linux:

pip list

5. Now let's include Python code in command prompt with  -c option and requests package:

On Windows, Linux:

python -c "import requests; r = requests.get('https://www.usessionbuddy.com/'); print(r.status_code)"

Here we are printing response status code, and as you can see getting 200. Everything works as expected.

6. Deactivate virtual environment

But how do we deactivate or "leave" virtual environment? There's deactivate command that takes us back to global environment.

On Windows, Linux:

deactivate

PyPi

Pip can install packages from many sources, but Package Python Index (PyPi) is the primary package source which it uses.

There are two options to install packages from PyPi:

1. Using command line
2. Directly from website

The way to search PyPi via command line is using pip search <package_name>. Take a look at the example below to search package "requests".

The search term yields quite an extensive collection of packages. Unfortunately, there isn’t much information other than a brief description.

Most of the time, you want to search for packages directly on the PyPI website. PyPI provides search capabilities for its index and a way to filter results by the metadata of the package, name of framework, topic, development status etc.

Another option to find a package is to Google it. The widely used Python libraries will show up at the top of google searches, and you should be able to find a link to the package in PyPI or its source code repository.

Requirements.txt

Understanding and using virtual environments also puts you on the right track to use more advanced dependency management methods like specifying project dependencies with requirements.txt files.

Python requirement files allow you to specify exactly the packages and versions that should be installed on your system.

You can easily dump all the packages and their versions installed in your virtual environment with pip freeze command.

Windows:

pip freeze > requirements.txt
type requirements.txt

Linux:

pip freeze > requirements.txt
cat requirements.txt

In case if you need to replicate the environment in another system, you can run...

pip install -r requirements.txt

Third-party package managers

Pip is the de facto package manager in the Python world, but there's a lot of greater tools and libraries to help simplify and improve package management. In this section, we are going to take a look at pip alternatives, Python community providing to other Python developers.

Pipenv

Pipenv package management tool that “aims to bring the best of all packaging worlds” to Python. It’s gaining a lot of traction among the Python community because it merges virtual environment and package management in a single tool.

It also solves some of the most common hiccups you will run into when manually managing dependencies through pip, like versions of packages, separating development and production dependencies, and locking versions for production.

Poetry

Poetry is another pip alternative that is gaining a lot of traction. Like Pipenv, it simplifies package version management and separates development vs production dependencies, and it works by isolating those dependencies into a virtual environment.

If you are coming from JavaScript and npm, then Poetry will look very familiar. It goes beyond package management, helping you build distributions for your applications and libraries and deploying them to PyPI.

Conda

Conda is a package, dependency, and environment manager for many languages including Python. In fact, its origin comes from Anaconda, which started as a data science package for Python.

Conda is widely used for data science and machine learning applications, and uses its own index to host compatible packages.

Key takeaways

• Pip is a package manager for Python, used by many projects to manage dependencies. It's included with the Python installer.
• Pip allows developers to install a large set of tools and libraries from PyPi.
• As a best practice, all of your Python projects should use virtual environments to store their dependencies.