Getting Started with Python Pre-commit Hooks | by Edward Krueger | Towards Data Science

By: Edward Krueger and Douglas Franklin
Here is the template for this project. We use poetry for package management. This article is focused on pre-commit hooks.
github.com
Pre-commit hooks are a subset of Git hooks. Git-hooks are scripts that run automatically every time a particular event occurs in a Git repository. A “pre-commit hook” runs before a commit takes place.
These Python hooks are considered a method of static analysis. Static code analysis is a method of debugging done by examining code without executing it.
Pre-commit hooks are often used to make sure code is linted and formatted properly before being published. In our project, we use four pre-commit hooks, black, check-large-files, pylint and our custom hookcreate-requirements.
We specify these Python hooks in a YAML file called pre-commit-config.yaml. YAML is a human-readable data-serialization language that is commonly used for configuration files. YAML files contain lists and dictionaries with key/value pairs.
Confusingly, there is also a Python package called pre-commit. Pre-commit is a management tool for pre-commit hooks. This package manages the installation and execution hooks before every commit.
We will need thispre-commit package as a dev-requirement. Additionally, the hook check-added-large-files is from this pre-commit package, as you can see in our .yamlfile later.
Notice that under[tool.poetry.dev-dependecies]you find pre-commit.
Setting up git hooks is surprisingly easy with the package pre-commit. All we need to do is create and format a .pre-commit-config.yaml file within our project. Then the packagepre-commit looks for this .yamlfile when you run:
> pre-commit install
Since Python our Python hooks are stored as YAML in the source code, they are distributed automatically when we upload this file to Git. This makes setting up a project’s pre-commit hooks on a new machine is as easy as cloning the repo and running the above line.
This process is more complicated if you are not using the Python package, pre-commit, and the standard format they have made available to us.
This is a good general use pre-commit-config.yaml. We can see that under repos is a list of dictionaries with key/value pairs to specify hooks and their behaviors. This YAML contains hooks for formatting, a file size filter, a linter, and our custom hook. Each hook has an accompanying id, seen on lines 5, 9, 12, and 18.
This is a standard .yaml format. You can see the repo keys nested within repos. Each of these repo keys is followed by the value (URL) of the required hook code. Lines 2 and 6 have URLs that correspond to Github repositories.
On line 10, we specify that the following hooks arelocal ones. This means that locally we run pylint using the bash code on line 14 and create-requirements with the code on line 20.
Running pylint locally gives us more control over the lining behavior because we can specify how the bash command works. This saves us time with debugging pylint issues and allows for more transparency and customization. Additionally, running pylint locally means you are using the version of pylint in your environment.
It is worth noting that on lines 3 and 7 we use rev to specify versions of the hooks we want to use. These versions may not be in sync with the versions in your virtual environment. You will have to keep these versions in sync manually. If this is an issue, you can run all your hooks locally to ensure the hooks are using the same version you've specified in your virtual environment.
We run poetry run pre-commit install to initialize the git hooks.
Now our pre-commit hooks will run when we commit code.
The terminal output here is straightforward. We see that each hook has a skipped,passed, or failed result.
Black is skipped as there are no changed Python files to run formatting on. The hook check-added-large-files passes because there are no files over 5Mb being committed. pylint fails to find any linting errors in our Python files. Lastly, we notice that our custom hook fails because our requirements.txt file is not in sync with our pyproject.toml.
Let's say we’ve made the changes needed so that all of our pre-commit hooks pass (or skip) and we/ve used git add include the changes in our commit. Now when we run git commit our output looks slightly different.
Since all of our pre-commit Python hooks have passed or skipped the commit will be completed and staged. Note that when a hook fails so does the commit. This is how pre-commit hooks save us from making bad commits when our code is not up to our standards.
Now we can git push and make the changes to our source code knowing that our commit is devoid of syntax errors, unnecessary large files, or an outdated requirements.txt file.
Remember, a“pre-commit hook” runs before a commit takes place. Pre-commit, the Python package, makes it easy for us to manage and run the hooks we need to ensure code is linted and formatted properly before being published.
With minimal setup, pre-commit hooks can save you time by avoiding mistaken commits. Pre-commit hooks like black ensure the code you publish is clean and professional. Pylint shows us errors before we add them to a codebase. The large-file-check warns you before you accidentally commit large files to git. It’s very hard to reverse such an accidental commit, so it’s better to prevent it in advance.
You can even code your own pre-commit hooks as bash scripts for custom needs. For example, you can require tests to run before code is committed.
For more information on using bash to code your own pre-commit hooks check out this article!