Ease your development hurdles with these [M]ake recipes
Tags: make, automation, commands, development
Published: 2021-07-21 07:07:21

## Table of contents

1. [Introduction](#introduction)
2. [Some Recipes/Tasks for your backend projects](#recipes)
  1. [Build the project](#build)
  2. [Clean up the workspace](#clean)
  3. [Run the project](#run)
  4. [Prepare the local environment for development](#setup-dev)
  5. [Reset the database](#reset)
  6. [Freeze and unfreeze the state of the database](#freeze)
  7. [Update non-local environments](#non-local)
  8. [File generation scripts](#generate)
  9. [Run arbitrary command](#command)
  10. [Migrate database](#migrate)
  11. [Check code](#check)
  12. [Bump version](#bump)
  13. [Run unit/acceptance tests](#tests)
  14. [Open the profiling tool](#profiling)
3. [Conclusions](#conclusions)

## Introduction<a name="introduction"></a>

*Make* and other similar expert/automation systems like *PyInvoke* (I highly love this one over other build systems, even over Makefile), *Rake*, *Cake*, etc... avoid the necessity of having a bunch of scripts (bash, python...) to perform certain repetitive tasks. Even then, you often chain them in succession to achieve a certain goal (e.g. you know you have to run _script_1.sh_ before _script_2.sh_, but that dependency is probably not made explicit in the project file structure). These dependencies are difficult to remember as they become more and more complex.

Or you might install a whole bunch of other tools for your project: *poetry*, *gulp* or docker just to name a few. Eventually recalling the arguments required to perform a certain action becomes difficult. **But it's never difficult to recall the action you need to perform**.

Dumb metaphor: It's easier to remember that you need an spanish omelette than remember that you need eggs, potatoes, onions and salt to build such delicious dish. Wouldn't it be easier if you had the power to imagine an omelette and PUFF!, it materializes in front of you, than building the omelette by yourself each and every time? That's where `make` and other derivatives can help you.

Here I present you with several suggested recipes that I use on a daily basis on the backend projects I have been working. The example are expressed in _Makefile_ syntax, but the very idea can be applied to every build system.

## Some Recipes/Tasks for your backend projects<a name="recipes"></a>

### Build the project (`make build`)  <a name="build"></a>

Pretty self explanatory. Every project of a reasonable size and maturity needs to be built, either by compilation, building the docker image, you name it. Therefore, the responsibility of this recipe should be to build/update the project and leave it in a runnable state. Optionally it could run migrations as well to keep the database schema up to date.

Make recipe:
```
build:
        # Put here the commands to compile, install dependencies, build docker images...
```

Invocation:
```
make build
```


### Clean up the workspace (`make clean`)<a name="clean"></a>

This one is the inverse of build. Its responsibility is to clean up the docker images, volumes, generated files... created by the build command.

Make recipe:
```
clean:
        # Clean docker images, dist files, etc...
```

Invocation:
```
make clean
```

### Run the project (`make run`)<a name="run"></a>

This one runs the project. Prior to spinning up the application (running `docker-compose up`, executing the main script, etc.), it could prepare the environment to ensure that the project starts up successfully: invalidating caches, creating an AWS session, running database migrations, etc...

This is the command we tell the frontend team to run after building the application with `make init` to spin up the web server for theirs to run their HTTP requests. 


Make recipe:
```
run:
        # Run your project here

stop:
        # In the same sense that there is a run, it might be interesting for you to have a stop command that kills the process/es that starts up the `run` command.
```

Invocation:
```
make run
make stop
```

### Prepare the local environment for development (`make setup-dev`)<a name="setup-dev"></a>

This one is intended for those that are going to do development over the project. It installs dependencies locally, set up pre-commit hooks, etc... basically setup and install anything required for local development.

Make recipe:
```
setup-dev:
        # Run pre-commit scripts, install dependencies locally...
```

Invocation:
```
make setup-dev
```

### Reset the database (`make reset`)<a name="reset"></a>

Eventually and through your testing or the testing of your colleague developers, the database is going to be heavily polluted, and this command should reset the database to `factory settings` (destroying and creating the local database, re-running the initial data migrations, etc.)

Make recipe:
```
reset: down
        # Run db scripts to purge the DB/s, remove docker volumes...
```

Invocation:
```
make reset
```

### Freeze and unfreeze the state of the database (`make freeze` and `make unfreeze`)<a name="freeze"></a>

I came up with these after some people of my team suggested that it would be cool, from a testing perspective, if we could save a snapshot of the database and then restore it. It proved very useful.

Make recipe:
```
# Example with postgres, leveraging docker

freeze: down
        docker run -v my_postgres_vol:/volume -v /tmp:/backup --rm loomchild/volume-backup backup postgres

unfreeze: down
	docker run -v my_postgres_vol:/volume -v /tmp:/backup --rm loomchild/volume-backup restore postgres
```

Invocation:
```
make freeze
make unfreeze
```

### Update non-local environments like DEV/TEST/UAT/PROD (`make update-remote`)<a name="non-local"></a>

Often non-local environments have different characteristics than your machine (they might not run via docker, for example). Having a recipe to handle migrations and updates to those environments is very useful to avoid forgetting to run something.

Make recipe:
```
update-remote:
        # call here the commands that are intended to be called when you deploy to your remote environment: Database migrations, cache invalidations, scripts to fix something...
```

Invocation:
```
make update-remote
```

### File generation scripts (`make generate-XXXX`)<a name="generate"></a>

Sometimes you might need to generate files in your code (documentation, protobuf code, etc...). These kind of commands are intended for such tasks.

Make recipe:
```
generate-docs:
        # call your doc generation tool (doxygen, sphynx...)
```

Invocation:
```
make generate-docs
```

### Run arbitrary command (`make run-command`)<a name="command"></a>

I work with Django (but this probably applies to other frameworks as well). And wrapping these commands into a recipe is often a good idea to keep the execution centralized.

Make recipe:
```
run-command:
        # e.g. python src/manage.py ($cmd)
```

Invocation:
```
make run-command cmd="XXXXX"
```

### Migrate database (`make migrate`)<a name="migrate"></a>

This recipe usually is a dependency of another of the previously mentioned recipes like `init` and `up`. As the name implies, it applies the migrations to the database.

Make recipe:
```
migrate:
        # Run here the command to execute the migrations
```

Invocation:
```
make check
```

### Check code (`make check`)<a name="check"></a>

Any serious project run some kind of code formatting tool to comply with your company coding guidelines and best practices. This command performs these checks on the code.

Make recipe:
```
check:
    # Example leveraging pre-commit
    pre-commit run --all-files
```

Invocation:
```
make check
```

### Bump version (`make bump-version LEVEL=patch|minor|major`)<a name="bump"></a>

This recipe updates the version of your project. Each time I commit code the pre-commit hook calls this command with the `patch` argument, and each time I deploy to production I manually call it with the `minor|major` argument depending on the type of code change I'm conducting.

Make recipe:
```
bump-patch:
    # Example with bump2version
    poetry run bump2version $(LEVEL)
    git add _version.py
    git add .bumpversion.cfg
```

Invocation:
```
make bump-patch LEVEL=patch
```

### Run unit/acceptance tests (`make run-tests` and `make run-acceptancetests`)<a name="tests"></a>

This recipe runs the tests of my application with some default parameters I like to establish. Sometimes I create other similar recipes to run these very tests but with some quirks, like generating coverage and result reports when running.

Make recipe:
```
run-tests:
        # Call here the command to execute your tests: py.test, NUnit... just make sure that the environment variables needed by your tests are loaded. This is why I like to run my tests with docker compose!
```

Invocation:
```
make run-tests
```

### Open the profiling tool (`make open-profiling`)<a name="profiling"></a>

You might have been profiling some code to find bottlenecks in your application and generated a report you'd like to visualize with tools like `snakeviz` or `pyprof2calltree`. This command will open the latest report generated, ready for visualization.

Make recipe:
```
open-profiling:
        # Example of command, adjust to your preferred visual tool
	pyprof2calltree -i profiling/results/$(FILE).prof -k
```

Invocation:
```
make open-profiling FILE=results
```

## Conclusions<a name="conclusions"></a>

I exposed several recipes that I found useful when developing. Even though I gave some clues of the tooling you could potentially use, it's impossible to provide a detailed implementation since it wildly depends on the programming language you use, the environment, etc... However, what's really important about each recipe is the main objective it tries to achieve, and that's a fact (building the project, running tests...).

I'd like to hear from other fellow developers what other recipes you have come up with that could make our lives a little bit easier. If you have any to share please, feel free to contribute in the comments section!