Management dependencies in python
Tags: managementdependenci, python, poetry, hatch
Published: 2022-12-13 04:54:05

---
title: Management dependencies in python
published: true
date: 2022-10-17 07:19:33 UTC
tags: managementdependenci,python,poetry,hatch
canonical_url: 
---

> Putting ideas into words is certainly no guarantee that they'll be right. Far from it. But though it's not a sufficient condition, it is a necessary one.  -[Paul Graham](http://www.paulgraham.com/words.html)

 ![Management dependencies in python](https://jairoandres.com/content/images/2022/10/fernando-hernandez-JdoofvUDUwc-unsplash.jpg)

If you are a developer in python, what would be the answer to the next question: **_How to manage dependencies in python?._** Probably the answer include _pip_, _virtualenv_ and _requirements.txt_. These tools are the stack that python developers have been used for a long of time in different projects. However, due to the concept _'modern python',_ new interesting features can be used as asyn/await, f string, pattern matching, type hints, walrus operator etc. Having said that, modern python come with a new [specification](https://peps.python.org/pep-0621/) to management dependencies called **pyproject.toml**

### What is pyproject.toml?

The **pyproject.toml** is a file written in human-readable syntax, it is located in the root directory of the project and contains all the metadata, dependencies libraries and configuration of additional tools like: _black_, _mypy_ and _pytest_. The next configuration is a basic example.

```
[project]
name = "spam"
version = "2020.0.0"
description = "Lovely Spam! Wonderful Spam!"
readme = "README.rst"
requires-python = ">=3.8"
license = {file = "LICENSE.txt"}
keywords = ["egg", "bacon", "sausage", "tomatoes", "Lobster Thermidor"]
authors = [
  {email = "hi@pradyunsg.me"},
  {name = "Tzu-Ping Chung"}
]
maintainers = [
  {name = "Brett Cannon", email = "brett@python.org"}
]
classifiers = [
  "Development Status :: 4 - Beta",
  "Programming Language :: Python"
]

dependencies = [
  "httpx",
  "gidgethub[httpx]>4.0.0",
  "django>2.1; os_name != 'nt'",
  "django>2.0; os_name == 'nt'"
]

[project.optional-dependencies]
test = [
  "pytest < 5.0.0",
  "pytest-cov[all]"
]

[project.urls]
homepage = "https://example.com"
documentation = "https://readthedocs.org"
repository = "https://github.com"
changelog = "https://github.com/me/spam/blob/master/CHANGELOG.md"

[project.scripts]
spam-cli = "spam:main_cli"

[project.gui-scripts]
spam-gui = "spam:main_gui"

[project.entry-points."spam.magical"]
tomatoes = "spam:main_tomatoes"
```

_Example took from PEP 621_

### Tools 

Different tools use the **pyproject.toml** specification to automate the management dependencies process. There are a lot of tools s **_hown in Figure 1_** , but the most popular are:

![Management dependencies in python](https://jairoandres.com/content/images/2022/10/star-history-20221016.png)

### [Poetry](https://python-poetry.org/docs/) 

Poetry is the most popular library for packing and managing dependencies. In additional, it has a CLI which developers can use easily interact with the **pyproject.toml** file, the next code is an example of _pyproject.toml_

```
[tool.poetry]
name = "poetry-intro"
version = "0.1.0"
description = ""
authors = ["jairo <me@jairoandres.com>"]

[tool.poetry.dependencies]
python = "^3.7"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
```

Poetry includes an incredible CLI to deal with versioning problems which is common in projects that have different environments (DEV, PROD and QA). Developers who use poetry can install dependencies by environment and create group of dependencies, for example define dependencies only for documentation.

![Management dependencies in python](https://jairoandres.com/content/images/2022/10/Screenshot---2022-10-08T103213.199-1.png)
_Figure 2: CLI poetry. Took from poetry docs._

poetry has been growing in popularity, that is the reason why famous libraries like [pendulum](https://github.com/sdispater/pendulum) uses poetry

### [Hatch](https://github.com/pypa/hatch)

Hatch was recently added to [PYPA](https://www.pypa.io/en/latest/) repositories, that is because hatch is probably the tool that implement **PEP** standards in the most meticulous way, the next code is an example from **pyproject.toml**

```
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "test"
description = ''
readme = "README.md"
requires-python = ">=3.7"
license = "MIT"
keywords = []
authors = [
  { name = "Jairo Castaneda", email = "me@jairoandres.com" },
]
classifiers = [
  "Development Status :: 4 - Beta",
  "Programming Language :: Python",
  "Programming Language :: Python :: 3.7",
  "Programming Language :: Python :: 3.8",
  "Programming Language :: Python :: 3.9",
  "Programming Language :: Python :: 3.10",
  "Programming Language :: Python :: 3.11",
  "Programming Language :: Python :: Implementation :: CPython",
  "Programming Language :: Python :: Implementation :: PyPy",
]
dependencies = [
]
dynamic = ["version"]

[project.urls]
Documentation = "https://github.com/unknown/test#readme"
Issues = "https://github.com/unknown/test/issues"
Source = "https://github.com/unknown/test"

[tool.hatch.version]
path = "test/ __about__.py"

[tool.hatch.envs.default]
dependencies = [
  "pytest",
  "pytest-cov",
]
[tool.hatch.envs.default.scripts]
cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=test --cov=tests"
no-cov = "cov --no-cov"

[[tool.hatch.envs.test.matrix]]
python = ["37", "38", "39", "310", "311"]

[tool.coverage.run]
branch = true
parallel = true
omit = [
  "test/ __about__.py",
]

[tool.coverage.report]
exclude_lines = [
  "no cov",
  "if __name__ == . __main__.:",
  "if TYPE_CHECKING:",
]

```

In comparison with other tools, The CLI [hatch is faster](https://hatch.pypa.io/latest/). This has a huge amount of configurations references to personalized **pyproject.toml.**

Another good feature of Hatch, this allows execution scripts which is similar to NMP package.json scripts

```
[tool.hatch.envs.default.scripts]
cov = "pytest --cov-report=term-missing --cov-config=pyproject.toml --cov=test --cov=tests"
no-cov = "cov --no-cov"
```

The last example allows executing scripts of test coverage and defining them by environment. Managing dependencies in file pyproject.toml is less automated, but that is great at the same time because it avoids adding unnecessary data. Recently the popular framework FastAPI has changed  [changed flit to hatch](https://github.com/tiangolo/fastapi/commit/4267bd1f4f716244a8087e5aa0e68c1de629cfa9#diff-50c86b7ed8ac2cf95bd48334961bf0530cdc77b5a56f852c5c61b89d735fd711)

### [PDM](https://pdm.fming.dev/latest/)

PDM, as described, is a modern Python package and dependency manager supporting the latest PEP standards. One thing to remark is that PDM doesn't use virtualenv, That is because of [PEP 582](https://peps.python.org/pep-0582/%5D)the next code is an example from **pyproject.toml**

```
[project]
name = "pdm-project"
version = "0.1.0"
description = ""
authors = [
    {name = "Jairo Castaneda", email = "me@jairo.com"},
]
dependencies = [
    "fastapi>=0.85.0",
]
requires-python = ">=3.10"
readme = "README.md"
license = {text = "MIT"}
[project.optional-dependencies]

[build-system]
requires = ["pdm-pep517>=1.0.0"]
build-backend = "pdm.pep517.api"

[tool]
[tool.pdm]
[tool.pdm.dev-dependencies]
dev = [
    "pytest>=7.1.3",
]
```

PDM has an incredible console  

![Management dependencies in python](https://jairoandres.com/content/images/2022/10/Untitled.png)
_Figure 3: CLI PDM_

![Management dependencies in python](https://jairoandres.com/content/images/2022/10/Untitled-1.png)
_Figure 4: Folder pypackages which contain libraries installed by PDM_

### Other tools

[**Flit**](https://flit.pypa.io/en/stable/) is other popular tool, but this tool is only for package management. In general, it is used to create and publish packages but need other tool to complement its management dependencies.

**[Bento](https://cournape.github.io/Bento/)** is other tool, like Flit, bento is used for publish and packaging. Bento has a special syntax in a file called _beto.info_

### What tool I should be used?

I have forgotten how many times I read the tittle “modern python package and dependency manager”. Python has a lot of tools for resolver problem with packaging management. But none of these tools are compatible which each other, for instance: the format in _lock file_ (which is very important because of [hashes, pins, and tree of dependencies](https://youtu.be/VWWgkF-0cDQ?t=637)) is totally different in each tool.

In my opinion, **pyproject.toml** is the future to management dependencies in python, i.e., NodeJS uses a file as an entry point to managing dependencies. Choosing the better tool should be a task which takes time to analyze what requirements the project has, and you should have three points in mind.

- Review repository, starts, issues bugs compatibility with your CI/CD tools even some PaaS services don't support **pyproject.toml**
- Review standards, what standard follows? What is the status of the PEP?, is it finalized?  Is it draft? 
- Review who is behind the project? 

### Mentions

**Thanks** to [Jeisson](https://www.facebook.com/jeissonrangelbaez) Rangel for reading drafts and give me feedback of this