Python: Setting Up Project Environment
Tags: python, tutorial
Published: 2020-10-13 17:02:25

> Cover Image By Unsplash

# Introduction

I really love Python! Because it is very simple and easy. So, I'm going to share settings and tools related to Python.

If you want to see all the code needed for setup, check the [repository](https://github.com/jaeheonji/python-boilerplate) I uploaded!

So are you ready for Journey? 🚀 Let's get started!

# Getting Started

## pyenv

The pyenv is a [simple python version management](https://github.com/pyenv/pyenv#installation) tool. it can easily download or change a specific python version.

The pyenv provides a variety of installation methods. Here, we will install it using an automatic installer ([pyenv-installer](https://github.com/pyenv/pyenv-installer))

So, Let's started the installation of pyenv!

```bash
$ curl https://pyenv.run | bash
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   285  100   285    0     0    274      0  0:00:01  0:00:01 --:--:--   274

...

WARNING: seems you still have not added 'pyenv' to the load path.

# Load pyenv automatically by adding
# the following to ~/.bashrc:

export PATH="/home/vagrant/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
```

After the installation is complete, we need to make some configurations.

I mainly use zsh, so I set the environment variable in the `.zshrc` file like this:

```bash
# .zshrc

...

## Pyenv
export PATH="/home/vagrant/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
```

Then apply the environment (`source .zshrc`) or restart the shell.

```bash
vagrant in ~ at arch
➜ pyenv
pyenv 1.2.21
Usage: pyenv <command> [<args>]

Some useful pyenv commands are:
   activate    Activate virtual environment
   commands    List all available pyenv commands
   deactivate   Deactivate virtual environment
   doctor      Verify pyenv installation and development tools to build pythons.
   exec        Run an executable with the selected Python version
   global      Set or show the global Python version(s)
   help        Display help for a command

...
```

Good! Now we can manage python binaries through pyenv. But if you try to install python using pyenv, because of dependency it may fail. In this case, refer to the official [documentation](https://github.com/pyenv/pyenv/wiki#suggested-build-environment).

### Usage

```bash
$ pyenv versions # List all Python versions available to pyenv
* system (set by /home/vagrant/.pyenv/version)

$ pyenv install --list # List all available versions
Available versions:
  2.1.3
  2.2.3
  2.3.7
	...

$ pyenv install 3.9.0 # Install specific python version
Downloading Python-3.9.0.tar.xz...
-> https://www.python.org/ftp/python/3.9.0/Python-3.9.0.tar.xz
Installing Python-3.9.0...
Installed Python-3.9.0 to /home/vagrant/.pyenv/versions/3.9.0

$ pyenv versions
* system (set by /home/vagrant/.pyenv/version)
3.9.0

$ pyenv global 3.9.0 # Set or show the global Python version(s)
$ pyenv versions
system (set by /home/vagrant/.pyenv/version)
* 3.9.0

$ python --version
Python 3.9.0
```

## poetry

[poetry](https://github.com/python-poetry/poetry) is dependency management for Python. Most programming languages have something called a dependency or package manager. For example, Javascript's npm, Rust's cargo, etc.

Of course, `pip` already exists in Python. So why poetry, not pip?

pip has the following problems:

- Dependency resolution
- (non-determinstic) Dependency Locking
- Virtual Environment (Additional application required like `virtualenv`)

poetry is a new dependency management tool that can solve the above problems. So Let's start the poetry with a simple example project!

```bash
$ curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python # Install using script

$ poetry -V # need to configure PATH environment '$HOME/.poetry/bin'
Poetry version 1.1.2
```

After installation poetry, create the sample project folder named `poetry-example` and initialize the project with poetry

```bash
$ poetry new poetry-example
Created package poetry_example in poetry-example

$ cd poetry-example && ls --tree
.
├── poetry_example
│  └── __init__.py
├── pyproject.toml
├── README.rst
└── tests
   ├── __init__.py
   └── test_poetry_example.py
```

Good! Now `poetry-example` is a project that uses poetry! Then, let's install the `requests` package.

```bash
$ poetry add requests # install package
Creating virtualenv poetry-example-mO5WhdNK-py3.9 in /home/meow/.cache/pypoetry/virtualenvs
Using version ^2.24.0 for requests

Updating dependencies
Resolving dependencies... (3.1s)

Writing lock file

Package operations: 14 installs, 0 updates, 0 removals

  • Installing pyparsing (2.4.7)
  • Installing six (1.15.0)
  • Installing attrs (20.2.0)
  • Installing certifi (2020.6.20)
  • Installing chardet (3.0.4)
  • Installing idna (2.10)
  • Installing more-itertools (8.5.0)
  • Installing packaging (20.4)
  • Installing pluggy (0.13.1)
  • Installing py (1.9.0)
  • Installing urllib3 (1.25.10)
  • Installing wcwidth (0.2.5)
  • Installing pytest (5.4.3)
  • Installing requests (2.24.0)

$ ls
poetry.lock  poetry_example  pyproject.toml  README.rst  tests
```

Ok, now we can see the `poetry.lock` file. By default, poetry creates a virtual environment in cache directory. 

```bash
$ pwd
/home/vagrant

$ ls .cache/pypoetry/virtualenvs
poetry-example-mO5WhdNK-py3.9
```

If you want to change the virtualenv directory into local project folder, you can use the `virtualenvs.in-project` [configuration](https://python-poetry.org/docs/configuration/#virtualenvsin-project-boolean).  

## Style Checker & Code Formatter

The Style Checkers & Code Formatters are tools that can help improve code quality. There are many different tools in the Python eco-system, but here I have selected the most used ones.

### black

[Black is the uncompromising Python code formatter](https://github.com/psf/black) provided by the Python Software Foundation. An alternative to black is [yapf](https://github.com/google/yapf), distributed by Google. Both tools have their own strengths and weaknesses, but in my personal opinion, black is a bit easier to use.

```bash
$ pwd
/home/vagrant/Projects/poetry-example

$ poetry add -D black # Install black as development dependency
```

I installed `black` using the `-D` option. This option adds the package as dev dependency. Because in most cases, Style Checker or Code Formatter does not need to be published as part of the project.

Then, create a sample file for testing.

```bash
$ cat poetry_example/main.py
def main():
    print("Hello World");

if __name__ == "__main__":
    main()

$ python poetry_example/main.py
Hello World

$ poetry run black poetry_example/main.py
reformatted poetry_example/main.py
All done! ✨ 🍰 ✨
1 file reformatted.

$ cat poetry_example/main.py
def main():
    print("Hello World")

if __name__ == "__main__":
    main()
```

Did you find the difference before and after using black?

Before executing black, the `main.py` contains **semicolon** after the print statement. Semicolon does not affect Python execution, but Python does not use semicolon except in special cases.

So, after executing black, we can see that the semicolon is gone in the `main.py`

### flake8 & isort

[flake8](https://gitlab.com/pycqa/flake8) is Python style checker based on [PEP8(Python Enhance Proposal 8)](https://www.python.org/dev/peps/pep-0008/). black is a good code formatter. But some items such as Documentation String, black is not provided the check option. flake8 not only helps for lack of black but also can be easily applied with black.

[isort](https://github.com/PyCQA/isort) is also a similar tool to flake8, this tool sorts imports alphabetically, and automatically separated into sections and by type.

So let's use both tools with black.

```bash
$ pwd
/home/vagrant/Projects/poetry-example

$ poetry add -D flake8 isort # Install packages
```

```python
# poetry_example/main.py

from pprint import PrettyPrinter, pformat, isreadable # Expected sorts alphabetically

def Empty():
    raise NotImplemented # Expected Warning - F901

def main():
    pp = PrettyPrinter(indent=4)
    pp.pprint(["Hello", "World"])

if __name__ == "__main__":
    main()
```

Add some test statements and run.

```bash
$ python poetry_example/main.py
['Hello', 'World']

$ poetry run black poetry_example/main.py
All done! ✨ 🍰 ✨
1 file left unchanged.

$ poetry run flake8
./poetry_example/main.py:3:1: F401 'pprint.pformat' imported but unused
./poetry_example/main.py:3:1: F401 'pprint.isreadable' imported but unused
./poetry_example/main.py:7:5: F901 'raise NotImplemented' should be 'raise NotImplementedError'

$ poetry run isort poetry_example/main.py
Fixing /home/meow/Projects/poetry-example/poetry_example/main.py

$ head -5 poetry_example/main.py
from pprint import PrettyPrinter, isreadable, pformat # Changed!
```

## (optional) pipx

There are many applications or packages in the Python eco-system. For example, packages include things like `TensorFlow` and `Django`, and applications include `Jupyter` and `Ansible`.

In most cases, we use `pip` to download applications and packages. This approach creates a ridiculous situation. Because the pip is a general-purpose package installer with no environment isolation.

So, `[pipx](https://github.com/pipxproject/pipx)` is a tool to help solves these situations.

```bash
$ python3 -m pip install --user pipx # install via pip
$ python3 -m pipx ensurepath

# Example
$ pipx install bpytop
installed package bpytop 1.0.42, Python 3.9.0
  These apps are now globally available
    - bpytop
done! ✨ 🌟 ✨

$ which bpytop
/home/vagrant/.local/bin/bpytop
```

# Visual Studio Code

Visual Studio Code is one of my favorite code editors. In particular, I think VSCode is very compatible with Python.

So, I'll show using the sample project how to use the tools I've introduced so far with VSCode. First, let's create a new project.

```bash
$ poetry new python-boilerplate && cd python-boilerplate

$ poetry config virtualenvs.in-project true --local # change virtualenv location to local

$ poetry add -D black flake8 isort

$ touch setup.cfg # for flake8 configuration

$ ls -a
.venv  poetry.lock  poetry.toml  pyproject.toml  python_boilerplate  README.rst  setup.cfg  tests
```

As explained earlier in the poetry section, the poetry automatically creates the `virtualenv` folder in the cache directory.

In these cases, VScode cannot be easily accessed to virtualenv directory. So I used `virtualenvs.in-project` option to make it easier to access the `venv` folder in VSCode.

Now ready to configure settings. first, install the [Python extension for Visual Studio Code](https://marketplace.visualstudio.com/items?itemName=ms-python.python) and create the `.vscode/settings.json` file.

```json
{
    // VSCode
    "[python]": {
        "editor.formatOnSave": true,
        "editor.codeActionsOnSave": {
            "source.organizeImports": true
        }
    },
    // Python Extension
    "python.venvPath": "./.venv",
    "python.linting.pylintEnabled": false,
    "python.linting.flake8Enabled": true,
    "python.formatting.provider": "black",
    "python.linting.enabled": true,
}
```

Good! Now lint will be applied automatically for python files.

## (optional) Type Checker/Hints

As everyone knows, Python is a **dynamically typed language**. These features have pros and cons, but some users want Python to be like a **statically typed** **language.**

To make up for this, Python supports Type Hints ([typing](https://docs.python.org/3/library/typing.html)) since version 3.5. However, the document says:

> The Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as type checkers, IDEs, linters, etc.

So there are several tools that support Type Hint:

- [mypy](https://github.com/python/mypy)
- [pyright](https://github.com/microsoft/pyright)
- [pyre-check](https://github.com/facebook/pyre-check)

All three tools are good for Type Check. But among these, I personally prefer pyright.

### pyright

pyright is a static type checker developed by Microsoft. The biggest reason I prefer pyright is that it can be easily applied through VSCode. Just simply try to install [Pyright extension](https://marketplace.visualstudio.com/items?itemName=ms-pyright.pyright) and creates `pyrightconfig.json`

```json
{
    "include": [
        "python_boilerplate"
    ],
    "exclude": [
        "**/__pycache__"
    ],

    "reportMissingImports": "error",
    "reportMissingTypeStubs": "none"
}
```

This is a sample configuration. Pyright provides a variety of options. For more information, please check the official [documentation](https://github.com/microsoft/pyright/blob/master/docs/configuration.md).

And create the `main.py` for testing

```python
def sayHello(name: str) -> str:
		return 0 # Error!
```

VSCode will display the following problem with the return of that function

```python
Expression of type "Literal[0]" cannot be assigned to return type "str"
  "Literal[0]" is incompatible with "str"
```

# Conclusion

I think the biggest advantage of Python is that anyone can easily program with just only editor. Therefore, the above settings are not required!

However, I hope it will be helpful to anyone who needs the above information.

Thanks for reading 😆