Basic Python project setup
Tags: python, bash
Published: 2024-06-19 09:20:45

---
title: Basic Python project setup
published: true
date: 2024-06-17 22:00:00 UTC
tags: Python,Bash
canonical_url: https://peateasea.de/basic-python-project-setup/
cover_image: https://peateasea.de/assets/images/start-up-python.png
---

*Originally published on [peateasea.de](https://peateasea.de/basic-python-project-setup/).*

Setting up an existing Python project from a fresh clone shouldn’t be a chore. Automate the process with a setup script.

## Simple setup

How do I set up this project again? Do I use virtualenv or just the stdlib’s venv module? How do I install the dependencies? Are the documented setup instructions still up to date?

These are just some of the questions that whiz through my mind when setting up a Python project either from a fresh clone or if I need to start from scratch. One way I make my life easier is by having a setup script which handles all these things for me.

This idea is, of course, not new<sup id="fnref:not-the-first" role="doc-noteref"><a href="#fn:not-the-first" rel="footnote">1</a></sup> and I’ve been using a variation of what I present below for several years. The thing is, I found my solution to be suboptimal: a script called `install-deps` residing in a sub-subdirectory called `devops/bin/`. Although this solution worked it still felt somehow clunky and inefficient. I remember seeing a post from [@b0rk](https://jvns.ca/) a while ago (that I unfortunately can’t find anymore) which mentioned using a simple `setup.sh` script located in the project’s base directory. This seemed like a much better solution and is the pattern I now like to follow.

Here’s what I use:

```shell
#!/bin/bash

# setup.sh - set up virtual environment and install dependencies

# create venv if it doesn't already exist
if [! -d venv]
then
    python3 -m venv venv
fi

# shellcheck source=/dev/null # don't check venv activate script
source venv/bin/activate

pip install -r requirements.txt

# vim: expandtab shiftwidth=4 softtabstop=4
```

Thus, if I’ve moved a project to a new directory (and hence have to rebuild the `venv`) or if I’ve checked out a fresh clone onto a new machine, running

```shell
$ ./setup.sh
```

will get me up to speed quickly and simply.

## Script dissection

For those brave souls who would like more detail, let’s pick the script apart a bit.

### Shebang

```shell
#!/bin/bash
```

The first line is the [shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) line and ensures that we use [bash](https://www.gnu.org/software/bash/) when running the script. Bash has been my shell of choice for over 25 years, so it’s a hard habit to drop. It works well enough for my needs and is still in active development, so there’s been little pressure for me to change to something newer. I can’t say I haven’t tried something else though! Even so, I keep gravitating back to bash. Oh well.

### Quick docs

```shell
# setup.sh - set up virtual environment and install dependencies
```

Next, there’s a quick comment to say what’s getting set up. This will be more helpful in more complex situations where extra info will come in handy. In the basic, simple situation shown here, it’s probably not necessary, although it could be useful background information for onboarding new project members.

### A familiar environment

```shell
# create venv if it doesn't already exist
if [! -d venv]
then
    python3 -m venv venv
fi
```

This snippet creates and initialises the virtual environment directory if it doesn’t already exist. The square brackets test the condition within them and pass a true/false result to the `if` statement. This then handles which code to run depending upon the result it receives. The test condition checks for the absence of a directory (`! -d`; i.e. “not directory exists”) called `venv`. If the directory doesn’t exist, then we initialise the virtual environment within a directory called `venv` by using the [`venv` module from the Python standard library](https://docs.python.org/3/library/venv.html).

Once upon a time, I used to use [virtualenv](https://virtualenv.pypa.io/en/latest/) to create the virtual environment but at some point switched to the `venv` module from the standard library. Although virtualenv does have more features, the standard module is sufficient for my purposes. Thus, I avoid having to install virtualenv separately as an operating-system-level prerequisite before initialising a Python project. Now, Python is often the only OS-level prerequisite, which is a nice simplification.

### Environment activation

```shell
# shellcheck source=/dev/null # don't check venv activate script
source venv/bin/activate
```

To install the Python requirements (the following step), we first need to activate the virtual environment. This is as simple as sourcing the appropriate file.<sup id="fnref:source-is-bash" role="doc-noteref"><a href="#fn:source-is-bash" rel="footnote">2</a></sup>

The comment above the `source` line tells [`shellcheck`](https://www.shellcheck.net/)<sup id="fnref:shellcheck-linter" role="doc-noteref"><a href="#fn:shellcheck-linter" rel="footnote">3</a></sup> not to check the `activate` script. This isn’t my code, hence it doesn’t make any sense to check it for linter issues.

### Dependencies installation

```shell
pip install -r requirements.txt
```

Now we’re ready to do the actual hard work: installing the upstream Python dependencies. This step assumes that a file called `requirements.txt` exists in the base project directory.

Using a single requirements file is fine for small projects, or when starting a new project. Yet, as a project gets larger, it is useful to separate the development- and production-related requirements into separate files. In that case, it’s a good idea to create a `requirements/` _directory_ in the base project directory and put the (appropriately named) requirements files in there. In such a situation the dependencies installation step would look like this:

```shell
pip install -r requirements/base.txt
```

to install the base dependencies only required in the production environment, or

```shell
pip install -r requirements/dev.txt
```

to install the development-related dependencies in addition to the base dependencies.

### Vim standardisation

```shell
# vim: expandtab shiftwidth=4 softtabstop=4
```

The final line is the [`vim`](https://www.vim.org/) coda. This is an old habit but a useful one: it ensures that `vim` expands tabs to spaces and sets how far to indent the code. Although I define this in my main `vim` config, I also find it helpful to specify this information explicitly in source files.

## Always ready to run

One small thing: set the executable bit on the script so that you can run it directly, i.e. without specifying an explicit interpreter. In other words, make the script executable like so:

```shell
$ chmod 755 setup.sh`
```

## Extend as needed; avoid unnecessary docs

With the basic structure in place, one can now extend it to more complex setup situations. This is one of the great things about using a script for this purpose: we push all the gory details and complexity down to a lower level of abstraction. Thus, we put complexity behind a simple interface which does what it says on the box: set things up. This interface is simple, and–when used across multiple projects–consistent, thus reducing cognitive load. Your brain is now free to focus on more interesting things.

Putting these steps into a script makes the setup repeatable and automated; there’s no need to keep detailed setup instructions in a `README` or similar document. By avoiding detailed setup documentation, one avoids such instructions getting out of date. Also, one reduces the risk of human error through missed steps or misspelled commands. The setup documentation is then simply “run the setup script”. In other words: keep it simple. :smiley:

## Summing up

In short, dump any project setup details into a script and automate away your setup documentation.

1. And I’m definitely not the first to have thought of it! [↩](#fnref:not-the-first)

2. [`source`](https://ss64.com/bash/source.html) is a `bash` built in command and is equivalent to `. ` (dot-space) in [POSIX](https://en.wikipedia.org/wiki/POSIX)-compatible shells. I find `source <script-name>` easier to understand than `. <script-name>` and less easy to confuse with script execution, i.e. `.<script-name>`. [↩](#fnref:source-is-bash)

3. `shellcheck` is a [linter](https://en.wikipedia.org/wiki/Lint_(software)) for shell scripts. It’s awesome. You should use it! [↩](#fnref:shellcheck-linter)