PVP - A Workflow for Python Projects
Tags: python
Published: 2020-06-12 02:07:16

- **I swear I already installed that package twice!!**

- **Which of these environments belong to my project?!**

- **Screw it, I'm installing everything globally!!** 😈

Managing Python projects can be quite the task, especially if you have more than one on your machine. There are countless workflow approaches to managing your Python versions, environments, and project dependencies. This guide is an overview of one such approach that emphasizes explicit control of your Python versions and virtual environments. The tools we'll be using are Pyenv, Virtualenv, and Poetry, which is why I refer to this as the PVP workflow.

___

## Index

- [Requirements](#requirements)

- [Setting Up New Projects](#setting-up-new-projects)
  - [Installing Python Versions](#installing-python-versions)
  - [Creating Virtual Environments](#creating-virtual-environments)
  - [Installing Project Dependencies](#installing-project-dependencies)
  - [Git Started](#git-started)
  - [New Project Summary](#new-project-summary)

- [Working with Existing Projects](#working-with-existing-projects)
  - [Setting Up Existing Projects](#setting-up-existing-projects)
  - [Managing Project Dependencies](#managing-project-dependencies)
  - [Removing Virtual Environments](#removing-virtual-environments)
  - [Removing Python Versions](#removing-python-versions)

- [Conclusion](#conclusion)

___

# Requirements

To follow along with this guide you will need `pyenv`, `pyenv-virtualenv`, and `poetry` installed on your machine. 

- **[Pyenv Prerequisites](https://github.com/pyenv/pyenv/wiki/Common-build-problems)**
  
    Required for `pyenv` to build Python interpreters from source.

- **[Pyenv Installer](https://github.com/pyenv/pyenv-installer)** (Recommended approach)
  
    Will install `pyenv` as well as `pyenv-virtualenv`.

- **[Poetry Installation Page](https://python-poetry.org/docs/#installation)**
  
    Be sure to check out everything `poetry` can do by skimming the docs.

> *If you are a Windows user utilizing Windows Subsystem for Linux, be sure that you are installing these tools on the subsystem rather than Windows.*

___

# Setting Up New Projects

This section of the guide focuses on quickly setting up a brand new Python project. Here we'll be ensuring we have the version of Python we want to use, creating a virtual environment for our project to play in, and installing project dependencies inside that environment.

- [Installing Python Versions](#installing-python-versions)

- [Creating Virtual Environments](#creating-virtual-environments)

- [Installing Project Dependencies](#installing-project-dependencies)

- [Git Started](#git-started)

- [New Project Summary](#new-project-summary)

___

## Installing Python Versions 

Before creating a virtual environment and installing your project dependencies, we should make sure that the Python version you'd like to utilize is installed on your machine. We'll use Pyenv to install and manage different versions of Python. The following commands can be executed from any location on your file system.

1. **Check the versions of Python currently available on your machine.**

    ```python
    pyenv versions
    ```

    This will show you a list of each version of Python that Pyenv has installed on your machine. If the version you want is listed you can skip down to the next section of this guide on [Creating Virtual Environments](#creating-virtual-environments).

2. **Check the versions of Python that Pyenv is able to install on your machine.**

    ```python
    pyenv install --list
    ```

    This command will list out all of the versions of Python that are available for installation. The version you want is most likely one of the simple semantic versioning numbers at the very top of the list (such as 3.8.2).

3. **Install your desired version of Python on your machine.**

    ```python
    # Template:
    pyenv install <python_version>

    # Example:
    pyenv install 3.8.2

    # Verbose Example
    pyenv install -v 3.8.2
    ```

    This process can take about 5 mins so sit tight while it's working. If you want visual feedback on the installation process, add the optional `-v` flag for a verbose printout of the actions Pyenv is taking during installation.

4. **Verify that the desired version was installed.**

    ```python
    pyenv versions
    ```

    This will list out all of the version of Python that Pyenv has installed on your machine. We should now see the version you just installed on the list.

___

## Creating Virtual Environments 

Now that we have our desired version of Python installed on our machine, let's use Virtualenv to create a virtual environment for our project to use. This virtual environment is like a playground containing your desired version of Python and any packages that you install for your project. Within this playground your project can play with reduced fear of interfering with other Python projects on your machine. 

1. **Create a directory for your project and move into it.**

    ```python
    # Template
    mkdir <project_name>
    cd <project_name>

    # Example
    mdkir my-sweet-project
    cd my-sweet-project
    ```

2. **Create a virtual environment for your project and give it a name that matches your project.**

    ```python
    # Template
    pyenv virtualenv <python_version> <project_name>

    # Example
    pyenv virtualenv 3.8.2 my-sweet-project
    ```

    Using this naming convention makes it easy to select the virtual environment containing your dependencies in IDEs like VS Code or Jupyter Lab since the virtual environment's name matches your project's name.

    If you check `pyenv versions` after this step you will now see your new virtual environment is listed as an available version of Python.

3. **Set your new virtual environment as the local version of Python.**

    ```python
    # Template
    pyenv local <virtual_environment_name>

    # Example
    pyenv local my-sweet-project
    ```

    This command will ask Pyenv to create a `.python-version` file in the current directory. This file tells Pyenv that any time we are in this directory, the version of Python we want to use is located in our virtual environment. Now anytime we navigate to this directory, Pyenv quietly switches the Python version to the virtual environment automatically. Whenever we leave this directory, Pyenv will switch the Python version back to the system default, unless it encounters another `.python-version` file that specifies which version to use instead.

    You can double-check `python -V` at this point to verify that the version of Python that is currently recognized matches the version you used to create the virtual environment.

___

## Installing Project Dependencies 

Now that we have a virtual environment utilizing our desired version of Python, we can start managing our new project's dependencies with Poetry.

1. **Initialize your project.**

    ```python
    poetry init

    # Alternatively, if you want to skip the prompts use:
    poetry init -n
    ```

    Poetry will create a `pyproject.toml` file that is used to keep track of some basic project information as well as any dependencies you install for the project.

2. **Install project dependencies.**

    ```python
    # Template
    poetry add <package_name_1> <package_name_2> <...>

    # Example
    poetry add requests jupyterlab requests
    ```

    When Poetry installs these packages, it places them within the virtual environment that was set as the local version of Python. Poetry will also add the name of the package to your project's `.pyproject.toml` file as a dependency. 

3. **Activate Poetry's shell.**

    ```python
    poetry shell
    ```

    To use the dependencies that Poetry has installed, we need to activate Poetry's shell before executing our code. 

    Sometimes it can be unclear whether the shell is active or not. To verify if Poetry's shell is active you can check your environment variables with the following command, `echo $POETRY_ACTIVE`. If the Poetry shell is active it will return a `1`, otherwise it will print a blank line to your terminal.

___

## Git Started

Now that your project is all set up, you probably want to keep track of your work with a version control system. Here we'll be connecting our project to GitHub for safekeeping.

1. **Navigate to the root directory of your project.**

2. **Create a few required files.**

    ```python
    touch .gitignore README.md
    ```

    At minimum, your `README.md` should have a top-level header with your project's name so it is rendered on your GitHub repository's page.

    Be sure to fill in your `.gitignore` file with the appropriate file names, file extensions, and directories to be ignored. [GitIgnore.io](http://gitignore.io/) is a great `.gitignore` generator.

3. **Initialize your project as a Git repository.**

    ```python
    git init
    ```

4. **Create your first commit.**

    ```python
    git add .
    git commit -m "<commit_message>"
    ```

5. **Create a new, empty GitHub repository online.**
    
    Log in to [GitHub](https://www.github.com/) and create a new repository. DO NOT create the repo with a license, `README.md`, or `.gitignore` file. Once the repo has been created you'll see a page with a bunch of commands you can copy/paste into your terminal to finish setting up the repo. We'll use these in the next step.

6. **Create an upstream remote and push your work to GitHub.**
    
    To complete the set up we'll want the two commands listed on your GitHub repository's page under *"... or push an existing repository from the command line"*. You can copy/paste those commands directly into your terminal or type them out if you can spot the HTTPS link to your repo (it ends in .git).

    ```python
    git remote add origin <link_from_github.git>
    git push -u origin master
    ```

    After this you're ready to proceed with your usual Git flow.

___

## New Project Summary

1. **Create Project Directory**

    ```python
    mkdir <project_name>
    cd <project_name>
    ```

2. **Create Virtual Environment**

    ```python
    pyenv virtualenv <python_version> <project_name>
    pyenv local <virtual_environment_name>
    ```

3. **Install Dependencies**

    ```python
    poetry init -n
    poetry add <package_name_1> <package_name_2> <etc>
    ```

4. **Activate Poetry's Shell**

    ```python
    poetry shell

    # (optional) Check if the shell is active
    echo $POETRY_ACTIVE
    ```

5. **Git Started**

    ```python
    # Add your README.md and .gitignore files, then
    git init
    git add .
    git commit -m "<commit_message>"

    # Create new, empty repo on GitHub, then
    git remote add origin <link_from_github.git>
    git push -u origin master
    ```

    ___

# Working with Existing Projects

This section of the guide will focus on how to use the PVP workflow with existing projects. We'll start from the very beginning and assume you are cloning a repository from GitHub on your machine and then move forward from there.

- [Setting Up Existing Projects](#setting-up-existing-projects)

- [Managing Project Dependencies](#managing-project-dependencies)

- [Removing Virtual Environments](#removing-virtual-environments)

- [Removing Python Versions](#removing-python-versions)

___

## Setting Up Existing Projects

This setup process assumes that you are cloning a repository that utilized Poetry for dependency management. Mainly we want to make sure that the repository has a `pyproject.toml` and `poetry.lock` file which are used to notify Poetry which packages (and more specifically which versions) are required as dependencies for the project.

1. **Clone the repository from GitHub to your macine.**

    ```python
    git clone <link_from_github.git>
    ```

2. **Check the required version of Python.**

    ```python
    cd <repository_name>
    cat pyproject.toml
    ```

    The `cat` command will print out the contents of the `pyproject.toml` file. Under `[tool.poetry.dependencies]` you'll see a specification for which versions of Python will work with the project. The example below indicates that the project requires a version of Python that is 3.8 or higher.
    
    ```python
    # Partial example of cat output

    [tool.poetry.dependencies]
    python = "^3.8"
    ```

3. **Check that you have the right version of Python installed.**
    
    ```python
    pyenv versions
    ```
    Use this command to see which versions of Python are available on your system. If you see a version that meets the project requirements you can move on to the next step.

    > *If you do not see a version of Python that meets the project requirements, please follow the [Installing Python Versions](#installing-python-versions) section to identify and install a version that is compatible with the project.*

4. **Create a virtual environment for the project.**

    ```python
    # Template
    pyenv virtualenv <python_version> <project_name>

    # Example
    pyenv virtualenv 3.8.2 my-sweet-project
    ```

5. **Set your virtual environment as the local Python version.**

    ```python
    # Template
    pyenv local <virtual_environment_name>

    # Example
    pyenv local my-sweet-project 
    ```

6. **Install the project's dependencies.**

    ```python
    poetry install
    ```

    Poetry will now read through your `pyproject.toml` to see which dependencies are required by the project. If the project includes a `poetry.lock`, Poetry will also examine that file to see which versions of each dependency are required. 

7. **Shell up and get to work!**

    ```python
    poetry shell
    ```

___

## Managing Project Dependencies

Here are a few basic Poetry commands that will help you install, remove, and inspect dependencies for your projects.

### Adding Project Dependencies

```python
# Adding a single package
poetry add <package_name>

# Adding multiple packages
poetry add <package_name_1> <package_name_2> <...>
```

### Removing Project Dependencies

```python
# Remove a single package
poetry remove <package_name>

# Remove multiple packages
poetry remove <package_name_1> <package_name_2> <...>
```

### Listing Installed Dependencies

```python
poetry show
```

___

## Removing Virtual Environments

When you are no longer examining or working on a project, you may want to perform a little clean up on your machine. Getting rid of the project is as simple as deleting the directory containing the project. It's only one additional step to remove the virtual environment and accompanying installed packages from your machine as well.

```python
# Template
pyenv virtualenv-delete <virtual_environment_name>

# Example
pyenv virtualenv-delete my-sweet-project
```

> *You will be prompted to confirm that you want to delete your virtual environment by entering `'y'` on the terminal.*

___

## Removing Python Versions

Have you installed an experimental build or moved on to the new best thing? At some point you'll probably want to remove a version of Python from your system. Pyenv brought Python into our machine, and so help us, it will take it out.

```python
# Template
pyenv uninstall <python_version>

# Example
pyenv uninstall 3.8.2
```

___

# Conclusion

This is simply one approach out of many towards managing all of the moving pieces associated with development on a Python project. Use aspects that you like, plug in tools that you prefer, and make it your own! There is no one correct answer and I would love to hear about workflows that others are using.