Implement your own Python Pre-commit Hooks | by Edward Krueger | Towards Data Science

By: Edward Krueger and Douglas Franklin.
Here is the template for this project. We use poetry for package management. This article is focused on writing custom pre-commit hooks.
github.com
For more introductory information on Python pre-commit Hooks, check out this article.
towardsdatascience.com
Pre-commit-hooks are a subset of Git-hooks. Git-hooks are scripts that run automatically every time a particular action occurs in a Git repository — such as a commit or a push. A “pre-commit hook” runs before a commit takes place.
Pre-commit hooks are often used to make sure code is linted and formatted properly before being published. These hooks are also used for continuous integration, build process, dependency management and testing. However, testing with pre-commit hooks is controversial in the DevOps space and out of our scope here.
With simple bash code, you can craft your own custom hooks for specific needs—for example, continuous integration to a deployed application.
Confusingly, there is also a Python package called pre-commit. Pre-commit is a management tool for pre-commit hooks. It manages the installation and execution hooks before every commit.
We need to install pre-commit as a dev-requirement. For example, with poetry you would run;
poetry add pre-commit --dev
Here is the bash code for our custom hook, create-requirements. This bash script is designed to keep requirements.txt in sync with poetry’s lock file, poetry.lock.
This is useful because if we keep requirements up to date, we can use this file for deployment and avoid any custom build packs.
For example, we can keep a Heroku app deployed by ensuring any changes we make to our virtual environment dependency tree are reflected in our requirements.txt file.
Another reason this is useful is you can make lighter, simpler docker builds by avoiding installing your virtual environment inside the container.
If we have a virtual environment that is out of sync with our requirements.txt this pre-commit hook will catch it, provide an appropriate error message, and create the correct requirements.txt.
Let's take a closer look at our code to see how bash and exit statuses can be used with the package pre-commit to prevent a commit from completing.
On line 1, we can see the bash variable NEW_REQUIREMENTS stores our requirements generated by the package manager we are using, poetry.
Lines 3–9 are a check to see if a file named requirements.txt exists that provides the terminal output of this result. If the file does not exist, we use an exit status of 1 to indicate failure and pre-commit stops the commit.
For the bash shell’s purposes, a command which exits with a zero (0) exit status has succeeded. A non-zero (1–255) exit status indicates failure.
The bash script will finish and ultimately generate a new requirements.txt file because the NEW_REQUIREMENTS will not be equal to the blank REQUIREMENTS during the check on line 11.
On line 11, we store our current requirements.txt file as REQUIREMENTS. Then lines 13–15, check to see if the old version, REQUIREMENTS,is the same as our current virtual environment’s NEW_REQUIREMENTS. If these files are identical, we don’t need to update anything and use exit status 0 to indicate success. This then allows pre-commit to run the next hook or to complete the commit process if create-requirements is our final hook.
However, if REQUIREMENTS and NEW_REQUIREMENTS are not identical, we need to re-write our requirements.txt. This is done on line 18, where we create a new requirements.txt file from poetry’s virtual environment. Notice that after we re-write this file, we use an exit status of 1 so that our script fails and our commit also does.
This failure allows us to run git add requirements.txt to our commit. Our next git commit will rerun our hook, this time passing all of the checks since the new requirements.txt is up to date with our virtual environment.
This process ensures that the virtual environment we are running our app in locally remains consistent with the virtual environment we are running our app in remotely.
Here we run a couple of checks to see how our bash script is working.
Let's imagine there is no requirements.txt file in our repo and we attempt a commit.
According to our bash code, we should get a failure, and requirements.txt does not exist error message. A requirements.txt is then generated in the directory. We can see this by running git status.
Notice we now have an untracked requirements.txt. In practice, one would now:
Run git add requirement.txt
Run git commit -m "add requirements.txt"
Now our commit is completed and is ready to be pushed to git.
If you dont want to commit this check, run git reset HEAD~ to revert the commit.
Let's say we've made a change to our virtual environments dependencies. We can run our pre-commit hooks with the following line.
 poetry run pre-commit run --all-files
This will give the same terminal output as running:
git commit -m “modified requirements.txt”
However, the first line only runs our pre-commit hooks, while the second line runs our hooks to stage a commit.
Notice the new error message that requirements.txt is not up to date!
Once again, we now have an untracked requirements.txt generated by line 18 of our bash script. In practice, one would now:
Run git add requirement.txt
Run git commit -m "update requirements.txt"
The point of pre-commit hooks is to make sure certain things happen or don’t happen before changes to a Git repo occur. In our case, this pre-commit hook makes sure our requirements.txt file exists and is up to date with our current poetry virtual environment. All this is done before we push any code to git and, ultimately, production.
Our custom hook, create-requirements, ensures that whatever environment we are using locally is consistent with the environment we are creating remotely. This is possible with bash code, exit statuses, and pre-commit. Exit statuses that are non zero cause bash scripts to emit an error. These errors therefore can be used to block commits that dont meet our standards.