Setup your Python Environment with Pyenv and Poetry
Tags: python, tutorial, beginners, productivity
Published: 2023-11-01 14:30:00

Dependency management in Python is very painful. Everyone that uses Python in multiple projects someday had a problem with package versions across the machine and even between team members. To solve these problems a lot of solutions appeared: venv, conda environments, Pipenv and many other tools that aim to create an isolated environment for you projects.

The objective of this quick tutorial is to help you easily setup a reproducible and isolated virtual environment. using **Pyenv** to manage your python versions and **Poetry** to manage your packages.

## Managing your Python Versions: PyEnv

The objective of [PyEnv](https://github.com/pyenv/pyenv) is to help you install and switch between multiple Python versions in your system without worrying about breaking things.

### Installing PyEnv

To install PyEnv you can follow their [Installation](https://github.com/pyenv/pyenv#installation) tutorial. Personally I like to use their official automatic installer found [here](https://github.com/pyenv/pyenv-installer). For any problem I also recommend reading the [Common build problems](https://github.com/pyenv/pyenv/wiki/Common-build-problems) specially the prerequisites section.

### Installing the Python Version

With PyEnv installed you can run the command below to obtain the list of possible Python versions that you can install:

```shell
pyenv install --list
```

The command above shows a huge list of python versions, you can choose whichever you need. For example, you can install Python 3.11.3 by running:

```shell
pyenv install 3.11.3
```

To know if everything went well you can check all the installed versions running:

```shell
pyenv versions
```

You should see an output similar to this:

```shell
system
* 3.11.3 (set by /usr/local/pyenv/version)
```

This means that are 2 python versions installed into your machine. The **system** is the one that comes with your OS. The other is the versions that we installed.

### Choosing the versions

The * symbol means that this is the default version used whenever you call python (i.e. the global version). You can change the global version by running the command `pyenv global`. For example, to set 3.11.3 as the global version you can run:

```shell
pyenv global 3.11.3
```

You can also set the Python version in a folder, this is useful when you need a different versions of python for many projects. To do this use the `pyenv local` command. Following our example:

```shell
pyenv local 3.11.3
```

## Managing your packages: Poetry

With Python installed in the correct version now you need a way to easily manage your packages in an isolated environment. [Poetry](https://python-poetry.org/) is a tool to help manage your packages and its dependencies. It provides packaging and dependency resolution features (we are going to focus on this one) for you not to worry more about conflicts in your dependencies.

### Installing Poetry

First you need to install the tool by following the [installation](https://python-poetry.org/docs/#installation) guide.

### Setup your project

Now you need to **go to your project folder** and run the `poetry init` command. This command will prompt you for some information including name of the project, authors, packages that you want to install from the beginning and other information. Note that this command will use the Python version setup by PyEnv giving preference to the local version.


```shell
poetry init
```

You will notice that the command will create 2 new files. The `pyproject.toml` is a human-friendly file introduced on [PEP518](https://peps.python.org/pep-0518/) and expanded in some other PEPs ([**PEP 517**](https://peps.python.org/pep-0517/), [**PEP 621**](https://peps.python.org/pep-0621/) and [**PEP 660**](https://peps.python.org/pep-0660/)). This file store many information about your project that can be used by many tools of the Python ecosystem.

The `poetry.lock` files contains the installed packages (including their dependencies) and whatever is necessary for them to work. You should not edit this file manually. 

> If your project already have a `pyproject.toml` inside with `[tool.poetry]` sections it means that someone already setup the tool. You can skip the `poetry init` and go directly to run `poetry install` to install all the required packages in your machine.

### Adding Packages

To add packages to the environment you just need to run the `poetry add` command. The command accepts a flag `--group` that is a way to organize the dependencies into groups. A common example is if you want to differ between production packages and development ones (e.g. test packages, code style and so on). In this case you can create a group called `dev` for the non-production environment. See an example below:

```shell
poetry add pandas==2.1.1             # Add pandas 2.1.1 on prod environment
poetry add pandas==2.1.1 --group dev # Add pandas 2.1.1 on dev environment
```

### Using your environment

With your environment configured, you need to activate it to use every time you call python. This is as simple as running:

```shell
poetry shell
```

If you don't want to activate the shell you can also run any python command using the environment with `poetry run` inside the project folder. For example, to run an example python script using the environment you can do:

```shell
poetry run python some-script.py
```

> For more Poetry commands and usage, see [Poetry Docs](https://python-poetry.org/docs/)

## Final Thoughts

Since I started my python journey I had used many tools to manage packages: venv, Pipenv, requirements, conda environments. This setup was the one that I had less headaches with the many projects and versions that I worked on. This tutorial should help you achieve that. 

Comment below if you use any other setup, if this tutorial helped you or any other comment that you want. Thanks for reading!

## Sources

* [pip: pyproject.toml](https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/)
* [Poetry Docs](https://python-poetry.org/docs/)
* [Pyenv](https://github.com/pyenv/pyenv)