Package Python Projects the Proper Way with Poetry
Tags: python, softwaredevelopment, devops
Published: 2020-01-27 12:46:37

---
title: Package Python Projects the Proper Way with Poetry
published: true
date: 2020-01-26 23:12:00 UTC
tags: Python,Software Development,DevOps
canonical_url: https://hackersandslackers.com/https://hackersandslackers.com/python-poetry-package-manager/
---

 ![Package Python Projects the Proper Way with Poetry](https://res-4.cloudinary.com/hackers/image/upload/q_auto/v1/2020/01/poetry-5@2x.jpg)

Managing software dependencies is typically a short and straightforward topic to cover; unless we're talking about Python, that is. Python has a myriad of options when it comes to virtual environments, which is arguably not a good thing. **Virtualenv** , **Pipenv** , **Conda** , and **Poetry** all aim to keep Python libraries project-specific, and they each do an adequate job of accomplishing this... but _why_ do we need a vast catalog of solutions to solve the same problem? If modern programming languages such as NodeJS manage to sidestep environments entirely (via the **/node\_modules** folder), why couldn't the Python community settle on a standard?

Python isn't exactly an "old" programming language (relatively speaking), but a lot has changed in programming since 1991. This was an era before language-specific package management systems had been conceived. Both **Pip** and **Virtualenv** were introduced to Python in 2011- that's _20 years_ where installing dependencies meant manually downloading packaged source code into a system folder. Using Virtualenv in 2020 feels archaic, but the concept of isolating packages on a per-project level was a somewhat novel concept at the time. Once this minimal implementation of package management became the norm, it didn't take long for developers to realize ways to improve this workflow. Early revelations resulted in the creation of virtual environment solutions, which seemed monumentally better than the last. In retrospect, these improvements were incremental to reaching the state of virtual environments today. This is why we find Python bloated with environment managers accumulated over time as the concept matured.

[Poetry](https://python-poetry.org/) is arguably Python's most sophisticated dependency management option available today. Poetry goes far beyond dependencies, with features like generating **.lock** files, generating project scaffolding, and a ton of configuration options, all of which are handled via a simple CLI. If you're unsure how to cleanly and effectively structure and manage your Python projects, do yourself a favor and use Poetry.

### More Than Just Dependencies

Proper dependency management is only one dimension of preparing a production-ready application. For a Python project to be considered a valid package to be listed on PyPi, the Python Packaging Authority has a [list of guidelines](https://packaging.python.org/tutorials/packaging-projects/) outlining various other requirements involving licensing, setup configuration files, manifests, and so forth. The allure of Poetry is that it is the first tool that handles _all_ of these things cleanly. The [Poetry Github repository](https://github.com/python-poetry/poetry) states its purpose as follows:

> Packaging systems and dependency management in Python are rather convoluted and hard to understand for newcomers. Even for seasoned developers, it might be cumbersome at times to create all files needed in a Python project: `setup.py`, `requirements.txt`, `setup.cfg`, `MANIFEST.in` and the newly added `Pipfile`. So I wanted a tool that would limit everything to a single configuration file to do: dependency management, packaging, and publishing.

It sounds crazy, but Poetry makes covering _all_ bases of a managing a Python project easy via its CLI and a single configuration file.

## Setting up Poetry On OSX/Unix

I don't typically reiterate installation instructions from official docs, I've run into issues following the existing install instructions word-for-word.

I'm assuming we're all using Python3 here since we aren't absolute savages. It's **_critically important_** that we install Poetry for python3 as opposed to Python, contrary to what the official docs would have you copy-and-paste. Mistakenly installing Poetry for the wrong version of Python is a massive headache, so please use this:

```
$ curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3
```
<figcaption>Poetry install script.</figcaption>

This script should install Poetry and automatically modify your system PATH to include the installation, as the success message states. Go ahead and confirm:

```
$ poetry --version
```
<figcaption>Confirm Poetry was installed correctly.</figcaption>

If the above command returns an error, make sure your **.bash\_profile** (or **.zshrc** ) was updated with the following, and restart your terminal:

```
export PATH="$HOME/.poetry/bin:${PATH}"
```

## Create a Python Project with Poetry

Enough with the foreplay, let's see how Poetry streamlines not only dependency management but nearly everything that goes into structuring a Python project. Poetry has a robust CLI, which allows us to create and configure Python projects easily. Here's what getting started fresh looks like:

```
$ poetry new poetry-tutorial-project
```
<figcaption>Create Python project via Poetry CLI.</figcaption>

This is an absurdly convenient way to generate a standard Python folder structure for our new project named **poetry-tutorial-project** :

```
/poetry-tutorial-project
├── README.md
├── poetry_tutorial_project
│ └── __init__.py
├── pyproject.toml
└── tests
    ├── __init__.py
    └── test_poetry_tutorial_project.py
```
<figcaption>Contents of our new project: /poetry-tutorial-project</figcaption>

This saves us the trouble of manually creating this standard folder structure ourselves. Most of the file contents are empty, with one exception: **pyproject.toml**.

## One Configuration to Rule Them All

The secret sauce of every Poetry project is contained in a file called _pyproject.toml_. This is where we define everything from our project's metadata, dependencies, scripts, and more. If you're familiar with Node, think of _pyproject.toml_ as the Python equivalent of package.json.

Starting a new Poetry project automatically creates a minimal version of this file. Here's what mine looks like:

```toml
[tool.poetry]
name = "poetry-tutorial-project"
version = "0.1.0"
description = ""
authors = ["Todd Birchard <todd@example.com>"]

[tool.poetry.dependencies]
python = "^3.7"

[tool.poetry.dev-dependencies]
pytest = "^4.6"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

```
<figcaption>pyproject.toml</figcaption>

The above config has some basic information, but it isn't enough to be useful just yet. A complete _pyproject.toml_ file would look something like this:

```toml
[tool.poetry]
name = "poetry_tutorial_project"
version = "0.1.0"
description = "Simple Python project built with Poetry."
authors = ["Todd Birchard <todd@example.com>"]
maintainers = ["Todd Birchard <todd@example.com>"]
license = "MIT"
readme = "README.md"
homepage = ""
repository = "https://github.com/hackersandslackers/python-poetry-tutorial/"
documentation = "https://hackersandslackers.com/python-poetry/"
keywords = ["Poetry",
            "Virtual Environments",
            "Tutorial",
            "Packages",
            "Packaging"]

[tool.poetry.dependencies]
python = "^3.7"
loguru = "*"
psutil = "*"

[tool.poetry.dev-dependencies]
pytest = "*"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

[tool.poetry.scripts]
run = "wsgi:main"

[tool.poetry.urls]
issues = "https://github.com/hackersandslackers/python-poetry-tutorial/issues"
```
<figcaption>pyproject.toml</figcaption>

Now we're cookin' with gas! Our .toml file is now comprised of 6 "sections," where each section contains config values for our project:

- **[tool.poetry]**: The first section of _pyproject.toml_ is simply informational metadata about our package, such as the package name, description, author details, etc. Most of the config values here are optional unless you're planning on publishing this project as an official PyPi package. Providing values for **repository** and **keywords** isn't going to matter if you're not distributing this package. Still, this sort of metadata would be critical if you ever hope to distribute your package.
- **[tool.poetry.dependencies]**: This is where we define dependencies our application absolutely _must_ download to run. You may specify specific version numbers for required packages (such as `Flask = "1.0.0"`), or if you simply want to grab the latest version, setting the version to "\*" will do just that. You'll also notice that the version of Python we're targeting for our project is provided here as well: this is specifying the _minimum_ version required to run our app. In the example above, a user running Python 3.6 will _not_ be able to run this app, as we specify that Python 3.7 is the absolute lowest version required.
- **[tool.poetry.dev-dependencies]**: Dev dependencies are packages that contributing developers should download to iterate on this project. Dev dependencies are not required to run the app, and won't be downloaded when the app is built by default.
- **[build-system]**: This is rarely a section you'll need to touch unless you upgrade your version of Poetry.
- **[tool.poetry.scripts]**: This is where we specify where our app entry point(s) is by assigning function within modules to the name of a script to be run. The example `run = "wsgi:main"` is specifying that we want to create a command called "run," which will look in **wsgi.py** for a function called **main()**. With this set, we can then launch our app via the Poetry CLI by typing poetry init (more on this in a bit).
- **[tool.poetry.urls]**: This is a completely optional section where you can add any number of helpful links or resources that somebody downloading this package might find useful.

A config like the one above is more than sufficient to have a clean, functioning, packaged app. Poetry supports other types of config values as well, although chances are you'll rarely need most of them. In case you're curious, find the full list here:

https://python-poetry.org/docs/pyproject/

## Poetry CLI

Poetry's command-line interface is impressively simplistic for the scope of what it achieves. The equivalent functionality of both **Pipenv** and **setup.py** are covered by Poetry, as well as numerous other features related to configuration management and package publishing. We'll start with installing and managing the dependencies we just set in _pyproject.toml_.

### Installing & Managing Dependencies

- `poetry shell`: The first time this command is run in your project directory, Poetry creates a Python virtual environment which will forever be associated with this project. Instead of creating a folder containing your dependency libraries (as virtualenv does), Poetry creates an environment on a global system path, therefore separating dependency source code from your project. Once this virtual environment is created, it can be activated again at any time by simply running poetry shell in your project directory in the future. Try comparing the output which python before and after activating your project shell to see how Poetry handles virtual environments.
- `poetry install`: Installs the dependencies specified in _pyproject.toml_. The first time a project's dependencies are installed, a _.lock_ file is created, which contains the actual version numbers of each package that was installed (i.e.: if Flask = "\*" resulted in downloading Flask version 1.0.0, the actual version number would be stored in _.lock_). If a _.lock_ file is present, the version numbers in _.lock_ will always be prioritized over what is in _pyproject.toml_.
- `poetry update`: Mimics the functionality of _install,_ with the exception that version numbers in _.lock_ will NOT be respected. If newer versions exist for packages in _pyproject.toml_, newer versions will be installed, and _.lock_ will be updated accordingly.
- `poetry add [package-name]`: A shortcut for adding a dependency to _pyproject.toml_. The package is installed immediately upon being added.
- `poetry remove [package-name]`: The opposite of the above.
- `poetry export -f requirements.txt > requirements.txt`: Exports the contents of your project's _.lock_ file to requirements.txt. It comes in handy when handing work off to developers who still use requirements.txt for some reason.

### Configuration 

- `poetry config`: "Config" refers to [environment-level configuration](https://python-poetry.org/docs/configuration/), such as the paths of the current virtual environment, or environment variables. Passing the _--list_ option will return your environment's current config values.
- `poetry check`: Checks _pyproject.toml_ for errors.
- `poetry show`: Returns a breakdown of all packages currently installed to the project, including dependencies of dependencies.

### Executing Applications

- `poetry run [script-name]`: Executes a script defined in  the **[tool.poetry.scripts]** section of _pyproject.toml_.

### Building and Publishing

- `poetry build`: Builds the source and wheels archives.
- `poetry publish`: Publishes the output of the previous build to the project's external repository (likely PyPi).

## Become a Poet

I'm well aware that tooling/architectural decisions in software are subjective preferences. Developers are not defined by their tools, blah blah, but look... at the risk of coming off as an ignorant novice, Poetry is objectively the best way to create, manage, and package Python projects. I don't have a horse in this race, but I'd go so far as to say that _not_ embracing Poetry is a waste of your time.

It's not much, but I've posted the code from this tutorial on Github:

https://github.com/hackersandslackers/python-poetry-tutorial