WIP: Taskfile instead of Makefile?
Tags: tutorial, tooling, automation
Published: 2023-07-31 20:44:55


Recently, I stumbled upon a tool called [Taskfile](https://taskfile.dev/#/). It is a task runner, similar to Makefile, but with many improvements. I decided to try it and see if it can replace my Makefiles.

## Why Taskfile?

First of all, Makefile is a good tool. Today, you can use it on any platform, not just Linux. It is a standard tool for running tasks. It is flexible and powerful. My relation with it is love/hate. It can be tricky to write a good Makefile or to debug it. Reading also can be a challenge. Because of that, when I found out about Taskfile, I decided to try it.

## First impressions

Taskfile is written in Go. It is a single binary that you can download and use. If you can run Go programs, you can run Taskfile. It is a big plus for me. See [docs on how to install it](https://taskfile.dev/#/installation).

Taskfile is a YAML file. Most of us are familiar with YAML, and many tools use it. It is easy to read, write, and parse, and IDEs have good support. Sometimes, it feels like we are programming in YAML. If you do not believe me, check out your CI/CD pipeline. I bet it is written in YAML.

You can use auto-completion if you use ZSH, Bash, Fish, or PowerShell. It is a big plus for me. I like to have auto-completion for my tools. It makes my life easier, especially when you start adding more and more tasks. When you have a lot of tasks, they can be grouped into namespaces, hidden from a user(internal task), global, or local, etc.

## Challenge: automate testing for Klock

[Klock](https://github.com/robert-nemet/klock) is my pet project. It is a `ValidatingAdmissionWebhook` for Kubernetes. It enables locking Kubernetes resources. You can read more here [Klock - Kubernetes locking](https://rnemet.dev/posts/projects/klock/).

One of the things that was on my to-do list was to automate testing. I'm doing it locally because it is easier and faster. But still, it is cumbersome to start the [Kind cluster](https://kind.sigs.k8s.io/), deploy Klock, and run tests. I decided to use Taskfile to automate it.

What I need to do is:

1. Start Kind cluster
2. Install Cert Manager
3. Build K8s manifests
4. Build the Docker image if there are any code/configuration changes
5. Load the image into a Kind cluster
6. Run tests

You can find my initial work on the git branch [migrate_taskfile](https://github.com/robert-nemet/klock/tree/migrate_taskfile). If you compare it with the **main** branch, you'll notice I added a new file called `Taskfile.yml`. It is a main Taskfile. Also, I added the folder `tasks` with two files, `Taskfile.testing.yml` and `Taskfile.tools.yml`. They are imported into the main Taskfile. The idea is to have a main Taskfile that will import other Taskfiles, while other Taskfiles will contain specific tasks.

### Start Kind cluster

First, I need to start the Kind cluster:

```yaml
version: '3'

vars:
  KIND_VERSION: "v0.20.0 go1.20.5"
  CERT_MANAGER_VERSION: v1.12.0
  IMG: controller:latest

tasks:
  init-cluster:
    desc: Create a kind cluster named klock
    preconditions:
      - sh: kind version | grep "{{.KIND_VERSION}}"
        msg: "kind version does not match {{.KIND_VERSION}}. Please install right version of kind"
    cmds:
      - kind create cluster --name klock
    status:
      - kind get clusters | grep klock
```

The slug `tasks` contain a list of tasks. This task is called `init-cluster`. It has a description, preconditions, commands, and status. For any task `name` and `cmds` slugs are mandatory. Others are optional. What they do:

* `cmds` - list of commands that will be executed
* `desc` - description of the task, it will be shown when you run list tasks: `task -l`
* `preconditions` - list of preconditions that must be met before running the task. If any of the preconditions fail, the task will not be executed. In this case, I check if the Kind version is the same as the one I'm using. If not, the task will fail with the message.
* `status` - list of commands that will be run to see if a task can be considered done. That means the task is done if the Kind cluster named `klock` exists. It does not care how a cluster is created. It just checks if it exists.

Notice block `vars`. It is a list of variables that can be used in the Taskfile. In this case, I'm using `KIND_VERSION` to check if the Kind version is the same as the one I'm using. The Taskfile uses [Go template (https://golang.org/pkg/text/template/) to render variables.

### Install Cert Manager

Next, I need to install [Cert Manager](https://cert-manager.io/):

```yaml
  install-cert-manager:
    desc: Install cert-manager    
    deps:
      - init-cluster
    cmds:
      - kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/{{.CERT_MANAGER_VERSION}}/cert-manager.yaml
      - echo "Waiting for cert-manager to be ready" && sleep 25
    status:
      - kubectl -n cert-manager get pods | grep Running | wc -l | grep -q 3
```

The new thing here is the `deps` slug. It is a list of tasks that must be executed before this task. In this case, I need to create a Kind cluster before I can install Cert Manager. Again, I'm using a variable `CERT_MANAGER_VERSION` to install the correct version of Cert Manager.

### Build K8s manifests

To generate manifests, I'm using [kubebuilder tool contrller-gen](https://book.kubebuilder.io/reference/controller-gen.html):

```yaml
 manifests:
    desc: Generate manifests e.g. CRD, RBAC etc.
    cmds:
      - controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
    sources:
      - main.go
      - apis/**/*.go
      - pkg/**/*.go
    generates:
      - config/crd/bases/**/*.yaml
```

This time I'm using `sources` and `generates` slugs:

* `sources` is a list of files that will be used to generate manifests
* `generates` is a list of files that will be generated

In this case, Taskfile will generate a checksum for sources from the previous run. If the checksum is the same, it will not run the task. It will just print out that the task is done. If the checksum is different, it will run the task. It is a nice feature that can save you some time. 

### Deploy Klock to Kind cluster

To deploy Klock to the Kind cluster, I'm using [kustomize](https://kustomize.io/):

```yaml
  deploy:
    desc: Deploy the controller to the kind cluster and wait for it to be ready. Use IMG to specify image name
    deps:
      - install-cert-manager
      - manifests
      - docker-build
    cmds:
      - task: kind-load
        silent: true
      - cd config/manager && kustomize edit set image controller={{.IMG}}
      - kustomize build config/default | kubectl apply -f -
      - echo "Waiting for controller to be ready" && sleep 25
    status:
      - kubectl get pods -n klock-system | grep klock-controller | wc -l | grep -q 1
```

This task depends on `install-cert-manager`, `manifests`, `docker-build`, and `kind-load` tasks. It means that those tasks have to be executed before this task.

But a small catch, `kind-load` can be done only if `docker-build` is done successfully. Other tasks can be executed in parallel. That is why other tasks are in the `deps` slug. Tasks in the `deps` slug will be executed in parallel. Tasks will be executed in order while in the `cmds` slug.

I'm adding a new slug, `silent` to suppress the task output.

This way, I managed to do some tasks in parallel and speed up the process.

### Run tests

To run tests I'm using [KUTTL](https://kuttl.dev/):

```yaml
  ktest:
    desc: Run kuttl tests. Specify image name with IMG
    cmds:
      - task: deploy
        silent: true
      - kubectl kuttl test
```

There is nothing special here. Except I should move the `deploy` task to the `deps` slug. But this is still a work in progress.

### Putting It All Together

My main `Taskfile` looks like this:

```yaml
# https://taskfile.dev

version: '3'

includes:
  tools: tasks/Taskfile.tools.yml
  tests: tasks/Taskfile.testing.yml

tasks:
  default:
    silent: true
    cmds:
      - echo "Welcome to Klock!"
      - task -l
```

I'm including two other `Taskfiles`. One is for tools, and the other is for tests. The `default` task is to print out a welcome message and list all tasks. Invoking the `task` without any arguments, it will run the `default` task:

```bash
$ task
Welcome to Klock!
task: Available tasks for this project:
* tests:cleanup:                    Delete the kind cluster named klock
* tests:deploy:                     Deploy the controller to the kind cluster and wait for it to be ready. Use IMG to specify image name
* tests:docker-build:               Build the docker image, specify image name with IMG
* tests:docker-push:                Push the docker image, specify image name with IMG
* tests:init-cluster:               Create a kind cluster named klock
* tests:install-cert-manager:       Install cert-manager
* tests:kind-load:                  Load the docker image into the kind cluster, specify image name with IMG
* tests:ktest:                      Run kuttl tests. Specify image name with IMG
* tests:manifests:                  Generate manifests e.g. CRD, RBAC etc.
* tests:publish:                    Run tests and publish the docker image. Specify image name with IMG and version with VERSION
* tests:undeploy:                   Undeploy the controller from the kind cluster
* tools:controller-gen:             Download controller-gen locally if necessary.
* tools:create-localbin:            Create the localbin directory
* tools:delete-kustomize:           Delete kustomize
* tools:envtest:                    Download envtest-setup locally if necessary.
* tools:install-kustomize:          Install kustomize
```

To run the tests, I'm using `task tests:ktest IMG=rnemet/klock:test`. It will run the `tests:ktest` task and pass the image name to the task. This image name is propagated to other tasks that need it. As you can see, I grouped tasks into namespaces. I'm using the `tests` namespace for tasks related to tests and the `tools` namespace for tasks related to tools.

## Conclusion

I'm still learning Taskfile. There are better ways to do things. But I'm pleased with the result. Still, I need to migrate other Makefile targets to Taskfile. But the results are promising. Changes are more readable, easier to maintain and to understand. When I migrate the rest of the Makefile targets, I can compare Taskfile and Makefile with more insights. But at the moment, Taskfile is a winner.

Looking into [Taskfile Github repo](https://github.com/go-task/task), I can see that the project is active, and much work is underway. I'm looking forward to new features and improvements.

This is a work in progress. I already have ideas on how to make it better. But I will leave that for another blog post. Meanwhile, you can check Taskfile and let me know what you think. Thanks for reading.

## References

* [Taskfile](https://taskfile.dev/)