Pipenv: Python's Official Package Management tool
Tags: python, pipenv, packagemanage, cli
Published: 2022-03-14 13:37:38

## Introduction

Pipenv is Python's officially recommended Package management tool, as the name suggests it combines the power of `pip` and `virtualenv`. It is really powerful and gives control of package management for a python project. It also has support for adding dev packages, python versions, smartly recognizing main and derived package installation/uninstallation, and so on. It is the [official package management tool for Python](https://packaging.python.org/en/latest/tutorials/managing-dependencies/#managing-dependencies). 

It is quite similar to npm for Nodejs in Javascript, or bundle for Ruby, cargo for Rust, and so on. It really simple and easy to use as it manages the overhead for package management for us and hence it is also a high-level package management tool as opposed to pip, which is not as powerful as Pipenv. So, in this article, we'll explore [Pipenv](https://pypi.org/project/pipenv/) package manager for Python and how you can use it in your next python project. Let's get started. 

## Install Pipenv

[Pipenv](https://pypi.org/project/pipenv/) is just like any other package in python, you can install it with pip as normally you install any other package with the command:

```
pip install pipenv
```

![Pipenv Install](https://res.cloudinary.com/techstructive-blog/image/upload/v1647193069/blog-media/hnhoeigfhx2hsypexgm5.png)

You can refer to the documentation of Pipenv from [here](https://pipenv.pypa.io/en/latest/).

## Set up a Python Environment

This step is not mandatory but it avoids any mistakes that you can make in the future while installing the package, so to simply create a new python environment for your project, you simply have to write the following command:

```
pipenv shell
```

![Pipenv initialize](https://res.cloudinary.com/techstructive-blog/image/upload/v1647192853/blog-media/dahwaqnblvvvqyyw62uq.png)
This will create the virtual environment for Python in the local folder. It is quite similar to installing `virtualenv` and then activating the env/venv folder script. Though pipenv is quite powerful as it automatically detects the environment. 

If you look at it carefully, there will be a file generated after the command has been executed successfully. The file called `Pipfile` without any extension will have been created in your current folder where you executed the command from. The file contains the version of python used in this project along with the list of dependencies(currently empty). Also the source from where it will download and manage the dependencies. 

The Pipfile after simply creating the virtualenv via the command `pipenv shell` looks something like follows:

```
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]

[dev-packages]

[requires]
python_version = "3.8"
```

If you want to set up a specific version of python for the virtual environment, you can do this using pipenv. The version should be installed in your system though for the pipenv to pick it up, if you have the specific version of python set up with all path variables configured, you can use the below command to set up the version of python in pipenv.

```
pipenv --python version-number
```

For example : `pipenv --python 3.9` will set up the virtual environment with python version `3.9`. 

## What is Pipfile

Pipfile is basically a TOML file that has all the details of the different dependencies/packages and the version of Python used in the project/directory. A TOML is a simple configuration file that is reader-friendly, it is a map of keys and values as configuration data structures. In Pipenv, we can have keys like `package-names` and the value as the `version-number`, certain groups of dependencies like `dev-packages` and `packages`, etc. Pipenv is the file that Pipenv implements its package management environment. The file is really important and powerful as it can install all dependencies even if provided with `requirements.txt` file. Yes, it can automatically detect that if you provide the path to that file. 

Pipenv also has additional features like adding dev dependencies/packages in the project as a separate dependency. So if you want to test a feature with a specific package you can add it as a dev package and it will be stored separately. The pipfile will segregate the dependencies so that Pipenv can install/uninstall from the virtual environment. In short, Pipfile lets you have great control over your project's packages management. 

## Installing Python Packages

Once your Pipenv is initialized as a virtual environment for Python, we can install dependencies with either `pip` or `pipenv`. This is the mistake that might get you trapped, if you already have not run the command `pipenv shell` and installed any dependencies with `pip`, you will install that dependency globally and make some mess of the project. So, it is advised to use `pipenv shell` in order to activate your virtual environment. If you do not wanna do that, you can use the command to install any dependency without activating the virtual environment.

```
pipenv install <package-name>
```

![Pipenv Install Flask](https://res.cloudinary.com/techstructive-blog/image/upload/v1647192980/blog-media/dfsokm6e1easwvxtgehh.png)

This will detect any virtual environment in the project, if it's not created already it will create it and install that package in that environment.

Installing any package using `pipenv` will update the Pipenv file and the package to its packages list.
```toml
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
flask = "*"

[dev-packages]

[requires]
python_version = "3.8"
```

OR 

If you wish to install with pip, as usual, you need to be in the virtual subshell. If you already are, then Pipenv will add that dependency to the virtual environment. Note though, if you install any package with `pip` and not with `pipenv`, the package won't be added to Pipfile but would be installed in the virtual environment. 

![Pipenv pip install](https://res.cloudinary.com/techstructive-blog/image/upload/v1647193149/blog-media/suaykqsyrgn1w0mou7f0.png)

Here, if we type the command `pipenv graph` it will show us a nice detailed list of all the installed dependencies. 

```
pipenv graph
```
```shell
$ pipenv graph
Flask==2.0.3
  - click [required: >=7.1.2, installed: 8.0.4]
    - colorama [required: Any, installed: 0.4.4]
  - itsdangerous [required: >=2.0, installed: 2.1.1]
  - Jinja2 [required: >=3.0, installed: 3.0.3]
    - MarkupSafe [required: >=2.0, installed: 2.1.0]
  - Werkzeug [required: >=2.0, installed: 2.0.3]
requests==2.27.1
  - certifi [required: >=2017.4.17, installed: 2021.10.8]
  - charset-normalizer [required: ~=2.0.0, installed: 2.0.12]
  - idna [required: >=2.5,<4, installed: 3.3]
  - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.8]
```

As you might have guessed, the above command is equivalent to the `pip freeze` command, but just compare the details both tools have. Pipenv really shines here.


If you compare the output of Pipfile and `pipenv graph` you get a bit confused as to why is there are so fewer packages in the file. So, **Pipfile doesn't store the sub-packages/dependencies of a certain base package**. Let's take, for example, Flask here, we have Flask as the main package, and `click`, `Jinja2`, `Werkzeug` are its sub dependencies, further `colorama` and `MarkupSafe` are in turn dependencies of `click` and `Jinja2`. So, Pipfile just includes the top-level packages, pipenv is smart enough to install the rest. It also doesn't include `requests` but it is indeed included in the `graph`. This is because Pipenv will only put the dependencies in the Pipfile if it has been installed via the `pipenv` command.

![Pipenv Graph vs pip freeze](https://res.cloudinary.com/techstructive-blog/image/upload/v1647193333/blog-media/gzo95pbquaioujtqvntu.png)

You can use `pip freeze` here as well as we are technically in a Python virtual environment. And you can clearly see the difference. Pipenv is a high-level tool compared to pip, it technically uses `pip` and `virtualenv` and leverages both of them to become a smart package management tool. 

## What is the Pipfile.lock

- If you are coming from `Javascript` world, it is similar to `package-lock.json` file.
- If you are coming from `Ruby` world, it is similar to the `Gemfile.lock` file.
- If you are coming from `Rust` world, it is similar to the `cargo.lock` file.

Ok, you get the idea, it is a file that more robustly specifies the version of the packages without conflicting with the other version or the Python version itself. If you look at the Pipfile.lock also has hashes that store the sub-packages as well. The file format here is JSON as opposed to TOML for the Pipfile. 

## Configuring the Pipenv environment

Now, a question you might have is where is the virtual environment? Is it there? Of course, it will be there, it is configured to a different location though. By default, it will be stored in the `~\.virtualenvs\` folder. 

You can get the location of the current virtualenv with the following command:

```
pipenv --venv
```

```
$ pipenv --venv
C:\Users\acer\.virtualenvs\pipenv-blog-gHY6vF9t
```
   For Windows, it is in the Admin user(in my case it is named `acer`) followed by the hidden folder `virtualenvs`, this folder will contain all the virtualenvs for different projects using `pipenv`. 

If you wished to change this location and keep the virtual environment folder in the same directory as your project, you can set up the environment variable for it as follows:

For Linux/macOS:
Add the following to your `~/.bashrc` or other equivalent shell configuration file.
```
export PIPENV_VENV_IN_PROJECT="enabled"
```

For Windows:

Add it to the PATH Environment variable.
```
set PIPENV_VENV_IN_PROJECT="enabled"   
```

This will make sure the virtualenvs for the project in `pipenv` are created inside the current folder itself and not in a single `~\.virtualenvs\` folder. 

## Creating the requirements.txt file

Let's say you also want to create a requirements.txt file for distributing your project to someone else, as not everyone will use Pipenv to manage their dependencies. It is really straightforward and quick to create a requirements.txt file from the Pipenv environment. 

The below command will make the `requirements.txt` file from the existing Pipenv project.

```
pipenv lock -r >requirements.txt
```

![Pipenv to requirements.txt file](https://res.cloudinary.com/techstructive-blog/image/upload/v1647193388/blog-media/e6t68b7ckrsrvadvmeqa.png)   
This will create the requirements.txt file, based on the Pipfile. Pipenv is smart again to provide all of the required dependencies to the requirements.txt in order that pip will be able to install all the required dependencies. 

## Using requirements.txt in Pipenv

We can install all the dependencies from the requirements.txt file while we are migrating from bare-bones virtualenv and pip to Pipenv. Pipenv will install all the mentioned dependencies and it will also add its checks for the appropriate checks for dependencies. 

```
pipenv install -r requirements.txt
```
This will install the dependencies mentioned in the requirements.txt file into the Pipenv virtual environment. 

## Managing Dev Packages 

Let's say we want to install a package but we are not sure to use it in production or the actual script, just a test for how it will work. Thus we have dev packages to install optional or testing packages. 

To install a dev-dependency or package, you can install via the following command:

```
pipenv install <package-name> --dev
```

![Pipenv dev package install](https://res.cloudinary.com/techstructive-blog/image/upload/v1647194653/blog-media/x5dimgfd2ikm2ercbzhv.png)

If we see the Pipfile, the `django` package that we installed tagged with `--dev` will be in the `dev-packages`

```
$ cat Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
flask = "*"

[dev-packages]
django = "*"

[requires]
python_version = "3.8"
```

If we wanted to uninstall all the dev-packages, we can simply enter the command:

```
pipenv uninstall --all-dev
```

[Pipenv uninstall devpackages](https://res.cloudinary.com/techstructive-blog/image/upload/v1647261079/blog-media/bg9n7aj6rfxsvdwflnah.gif)

This will simply uninstall all the dev dependencies/packages from the pipenv environment and also remove them from the packages list in Pipfile. If you wished to uninstall a specific package in pipenv, you can uninstall it by the simple command:

```
pipenv uninstall <package-name>
````


## Installing/Uninstalling all packages

To install only the default packages and not dev-packages. 

```
pipenv install 
```

To install or configure a project, if you want to test the project with all the dev dependencies, you can install them with:

```
pipenv install --dev
```

This will install all the packages both `packages` and `dev-packages`. 

If you want to uninstall all the packages in pipenv, you can use the command :

```
pipenv uninstall --all
```

This will uninstall all the default and dev packages from pipenv. This is like starting a fresh virtual environment. 

### References:

- [Pipenv documentation](https://pipenv.pypa.io/en/latest/)
- [Real Python - Pipenv Guide](https://realpython.com/pipenv-guide/)

## Conclusion

So, this was a brief introduction to `pipenv` which is Python's recommended package management tool. We saw that we have control over the dependencies and packages in a virtual environment which is taken care of automatically for us. We don't have to activate the environment to install or manage any package for a project.