Creating A Modern Python Development Environment
Tags: python, programming, tutorial, devops
Published: 2022-04-12 12:59:54

Being a Python developer for the past few years I’ve gathered a list of my favourite packages/tools that I use in almost every project. Developing is a team effort and at some point, someone is going to have to read and/or alter the code you have written. If the code is poorly implemented and the developer has a new feature or bug fix to implement it will likely lead to additional technical debt. Therefore, I like to enforce developers (*including myself *😉) to produce code that is tested, maintainable, readable and most importantly consistent, otherwise, it's rejected. This results in the software development process going more smoothly since they will be working on the same dependable code.

This blog post demonstrates how I set up and configure my Python development environment using pyenv and Poetry, on both Unix and Windows systems to produce the same consistent code, helping prevent technical debt.

## Installing pyenv 🐍

Downloading and installing Python from the official binary limits you to one specific Python version, which isn’t great if you’re working on multiple projects with different Python versions. Therefore, it's better to use [pyenv](https://github.com/pyenv/pyenv), a tool that lets you easily switch between multiple versions of Python.

### Windows Installation

For Windows users, I recommend you install [WSL2](https://docs.microsoft.com/en-us/windows/wsl/install), a [Windows Subsystem that lets developers run a Linux environment without the overhead of a traditional virtual machine or dual-boot setup](https://docs.microsoft.com/en-us/windows/wsl/about#:~:text=The%20Windows%20Subsystem%20for%20Linux%20lets%20developers%20run%20a%20GNU/Linux%20environment%20%2D%2D%20including%20most%20command%2Dline%20tools%2C%20utilities%2C%20and%20applications%20%2D%2D%20directly%20on%20Windows%2C%20unmodified%2C%20without%20the%20overhead%20of%20a%20traditional%20virtual%20machine%20or%20dualboot%20setup.). You can install WSL2 by executing the following command in either PowerShell or Command Prompt:

![WSL2 installation command.](https://cdn-images-1.medium.com/max/2000/1*YLKypOE2mFwP2NCTaLXdVQ.png)

If you don’t want to use WSL2 you can install [pyenv-win](https://github.com/pyenv-win/pyenv-win), *a less modern approach *and skip to the **Set Python Version with pyenv** section.

After installation search for Ubuntu on Windows (WSL terminal) and open it. It will present you with a bash shell and request you provide a username and password. Once they have been set your terminal is ready for tinkering 🔨 !

Before installing pyenv you’re going to need to update your environment and install a few build dependencies *(if you don't already have them). *Open your terminal and execute the following commands:

    > sudo apt update && sudo apt upgrade
    > sudo apt install -y build-essential git curl libexpat1-dev libssl-dev zlib1g-dev libncurses5-dev libbz2-dev liblzma-dev libsqlite3-dev libffi-dev tcl-dev linux-headers-generic libgdbm-dev libreadline-dev tk tk-dev

pyenv is now ready to be installed and configured *(tell your terminal where pyenv is):*

    > curl https://pyenv.run | bash

    > echo '\nexport PATH="$HOME/.pyenv/bin:$PATH"' >> ~/.bashrc
    > echo '\neval "$(pyenv init -)"' >> ~/.bashrc
    > echo '\neval "$(pyenv init --path)"' >> ~/.bashrc

    > source ~/.bashrc

    # Confirm successful installation:
    > pyenv --version
    pyenv 2.2.5 # Your version will likely be newer.

### Set Python Version with pyenv

Once pyenv is installed successfully, set your global Python version using the following commands:

    # Install the required Python version:
    > pyenv install 3.9.0

    # Refresh pyenv since you installed a new Python version:
    > pyenv rehash

    # Set your global Python version:
    > pyenv global 3.9.0

    # You can also set the local Python version, i.e. for a project:
    # > pyenv local 3.9.0

    # Confirm Python version:
    > pyenv version
    3.9.0 (set by /home/wsl/.pyenv/version)

If you require another version of Python you can simply execute the same commands with a different version number 🤗.

## Installing Poetry 📜

Packaging systems and dependency management in Python are rather convoluted and hard to understand for newcomers. Even for seasoned developers, it might be cumbersome at times to create all files needed in a Python project. [Poetry](https://github.com/python-poetry/poetry) helps you declare, manage and install dependencies of Python projects, ensuring you have the right stack everywhere. It automatically creates isolated environments for each project, preventing modifications to dependencies in another project.

    > pyenv shell 3.9.0

    > curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -

    > source ~/.bashrc

    # Confirm installation was successful:
    > poetry --version
    Poetry version 1.1.13 # Your version will likely be newer.

## EditorConfig

[EditorConfig](https://editorconfig.org/) helps maintain consistent coding styles for multiple developers working on the same project across various editors and integrated development environments (IDE). It works by reading the .editorconfig configuration file provided at the top level of a project. [Most IDEs have built-in support](https://editorconfig.org/#pre-installed) but this is a list of [IDEs/editors that require a plugin](https://editorconfig.org/#download).

This is the standard .editorconfig file that I use in all my Python projects:
{% gist https://gist.github.com/ShaneNolan/1bdd307291c729f4f566f90254a1c0b3.js %}
Your environment is now ready to create a modern Python project 🥳.

## Creating and Configuring a Python Project ⚙️

The following instructs how to *manually *create and configure a Python project using pyenv and Poetry.

To create a new Python project you need to specify the Python version using pyenv and then use Poetry to create the project.

    > pyenv shell 3.9.11
    > poetry new PROJECT_NAME # Example testproject
    > cd testproject
    > git init # Initialise the repository

Next, create the .editorconfig configuration provided above. Your project tree structure should look like this:

    > tree

    .
    ├── .editorconfig
    ├── .git
    │   ├── ...
    ├── README.rst
    ├── pyproject.toml
    ├── testproject
    │   └── __init__.py
    └── tests
        ├── __init__.py
        └── test_testproject.py

    12 directories, 22 files

Personally, I prefer Markdown instead of reStructuredText so I rename README.rst to README.md . All application logic goes into the folder named after your project and *tests go into the tests folder*.

![Example of creating a Python project using pyenv and Poetry.](https://cdn-images-1.medium.com/max/3398/1*aweX40JvPGoFG-CXbL6dhg.gif)

## My Essential Python Packages 📦

This section details the majority of development packages (*packages that are only required during the development phase*) I use in Python projects.

### pytest with pytest-cov

The [pytest](https://github.com/pytest-dev/pytest/) framework makes it easy to write small tests, yet scales to support complex functional testing for applications and libraries. I prefer pytest over the inbuilt unittest library because it allows for plugins such as [pytest-xdist](http://pypi.python.org/pypi/pytest-xdist), which enables the execution of multiple tests at once, expediting test runtimes and requires fewer lines of code to produce the same output as unittest. Additionally, [pytest-cov](https://pypi.org/project/pytest-cov/) is a plugin that automatically produces a test coverage report.

![Install development dependencies pytest and pytest-cov with Poetry.](https://cdn-images-1.medium.com/max/2000/1*kNzAYtq0KQtZ2eRJV1Ja9A.png)

### wemake-python-styleguide

The wemake.services team describe [wemake-python-styleguide](https://wemake-python-stylegui.de/en/latest/) as [*the strictest and most opinionated Python linter ever](https://wemake-python-stylegui.de/en/latest/#:~:text=strictest%20and%20most%20opinionated%20Python%20linter%20ever.)*. It's essentially the [flake8](https://github.com/PyCQA/flake8) package, a linting tool that analyses code for defects, with [plugins](https://wemake-python-stylegui.de/en/latest/pages/usage/violations/index.html#external-plugins). Its primary objectives are:

* Enforcing Python 3.6+ usage.

* Significantly reducing the complexity of code and making it more maintainable.

* Enforcing “There should be one — and preferably only one — obvious way to do it” rule to coding and naming styles

* Protect developers from possible errors and enforce best practices.

![Install development dependency wemake-python-styleguide with Poetry.](https://cdn-images-1.medium.com/max/2000/1*9_s4XK2dXtKht1ytGcb5Lw.png)

I’d also recommend the flake8-pytest-style plugin if you’re using pytest.

For existing projects, you can use the package [flakehell](https://github.com/flakehell/flakehell) with the baseline feature to report new violations since integrating a new linter will likely result in hundreds or even thousands of violations. This [guide](https://wemake-python-stylegui.de/en/latest/pages/usage/integrations/flakehell.html) explains how to integrate flakehell and resolve violations over time.

### Mypy

[Mypy](https://github.com/python/mypy) is an optional static type checker that aims to combine the benefits of dynamic (or “duck”) typing and static typing. The benefits of statically typed code are increased readability and maintainability. For an existing code base here's a [guide](https://mypy.readthedocs.io/en/stable/existing_code.html) on how to implement Mypy.

![](https://cdn-images-1.medium.com/max/2000/1*oT-Ai_iPXOA0hVWXbgVa0g.png)

![Before and after statically typed Python code.](https://cdn-images-1.medium.com/max/2000/1*HcJ6c2PvbjKIGDoJ-jEklg.png)

![Install development dependency Mypy with Poetry.](https://cdn-images-1.medium.com/max/2000/1*kRzO55GWEfs2PrtYQB1Ixw.png)

### Safety

A 2021 [security report](https://go.snyk.io/rs/677-THP-415/images/Python%20Insight%20Report.pdf) by Snyk, states 47% of Python projects contain known vulnerabilities. On the bright hand side, almost 87% of known vulnerabilities can be resolved by upgrading the vulnerable package. [Safety](https://github.com/pyupio/safety) checks your installed dependencies for known security vulnerabilities.

![Example of Safety detecting a vulnerable package.](https://cdn-images-1.medium.com/max/3398/1*kyKIpi8f6zU-mj2z_1cVlg.gif)

![Install development dependency Safety with Poetry.](https://cdn-images-1.medium.com/max/2000/1*14LQ0nng0mFXuAbj3TgR8A.png)

### pre-commit

[pre-commit](https://github.com/pre-commit/pre-commit) is a framework for managing and maintaining multi-language pre-commit hooks. For example, a hook that runs pytest must succeed before code can be committed to the repository, preventing code defects. The following pre-commit configuration file is what I use in my Python projects. **It ensures that tests, coverage, stating type checking and vulnerability scanning all pass before code is committed.**

![Example of pre-commit preventing defective code from being committed.](https://cdn-images-1.medium.com/max/3398/1*1Oj11ROFLa5DgvpJqGMhmQ.gif)

This is the .pre-commit-config.yaml file used in the above example:
{% gist https://gist.github.com/ShaneNolan/11f3665cfe8d205c39362684f0b21334.js %}
![Install development dependency pre-commit.](https://cdn-images-1.medium.com/max/2000/1*VPR5SM_voDsUs-eXweL0ug.png)

### Nitpick

It can become tedious maintaining configurations across projects, especially since EditorConfig, pytest-cov, flake8, Mypy and pre-commit all require configuration files. [Nitpick](https://github.com/andreoliwa/nitpick) is a tool and flake8 plugin that enforces the same configurations across multiple language-independent projects, automating this tedious configuration.

![Install development dependency Nitpick with Poetry.](https://cdn-images-1.medium.com/max/2000/1*4uqnkUuzixl0Dpxu5nUMgg.png)

Once installed, add the following line to your pyproject.toml file:

    **[tool.nitpick]**
    style = "https://raw.githubusercontent.com/wemake-services/wemake-python-styleguide/master/styles/nitpick-style-wemake.toml"

You’ll need to create a .gitignore and CHANGELOG.md file to successfully pass Nitpick validation.

![Nitpick configuration setup.](https://cdn-images-1.medium.com/max/2000/1*STIxcv98BWdHNvHuwvWBWg.png)

It will automatically create a setup.cfg file that will contain all the required configurations.

![Nitpick wemake-services installation.](https://cdn-images-1.medium.com/max/3398/1*UufwyUeqD5_DAxjC5viv0g.gif)

Congratulations, you’ve successfully created a modern Python development environment 🎉. If you would like to connect with me you can find me on [Linkedin](https://www.linkedin.com/in/shanenullain/) or [Github](https://github.com/ShaneNolan). Thanks for reading 😊!

## Extra: VSCode WSL2 Integration

If your editor is [VSCode](https://code.visualstudio.com/) then you’re in luck because you can integrate the Linux subsystem easily with its [remote WSL plugin](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-wsl). Once installed open your WSL terminal and execute code . . This will automatically open the VSCode editor within the Linux subsystem. You can read more about it [here](https://code.visualstudio.com/blogs/2019/09/03/wsl2).

## Extra: Automated Project Setup and Configuration

If you’re any way like me, then you must automate a task when you find yourself doing it more than once. I’ve automated the creation and configuration of a modern Python project using [cookiecutter](https://github.com/cookiecutter/cookiecutter). You will need to set up your environment first. Also, nitpick uses my own take on the wemake-services configuration, you can view the [repository](https://github.com/ShaneNolan/python-project-template) here.

    > pyenv shell 3.9.11

    ---------------------------------------------
    # Unix: Specify the name of the project.
    > export PROJECT_NAME="MY_PROJECT_NAME"
    =============================================
    # Windows: Specifcy the name of the project.
    > set PROJECT_NAME="MY_PROJECT_NAME"
    ---------------------------------------------
    
    > python -m pip install cookiecutter &&
    python -m cookiecutter --no-input gh:ShaneNolan/python-project-template \
    project_name=$PROJECT_NAME &&
    (cd $PROJECT_NAME && git init &&
    poetry init --no-interaction --name $PROJECT_NAME &&
    poetry add mypy pytest pytest-cov flake8-pytest-style wemake-python-styleguide safety pre-commit nitpick --dev &&
    poetry run nitpick fix &&
    poetry run pre-commit install) && python -m uninstall cookiecutter

![Example of Automated Project Setup and Configuration.](https://cdn-images-1.medium.com/max/3398/1*GEJV8vWbVOGo4EjelQo1Eg.gif)