Tekton Pipeline — Kubernetes-native pipelines
By: Nico Meisenzahl
Tags: Continuous Integration, Tekton, Continuous Delivery, DevOps, Kubernetes

Tekton Pipeline is a new project which allows you to run your CI/CD pipelines in a Kubernetes-native approach. Tekton Pipelines emerged out of the Knative build project. If you like to know more about Knative I highly recommend you to visit their project website which is available here.

Before we start talking about what’s Kubernetes-native means and how Tekton Pipelines are working I would like to first take a step back and clarify shortly why containerized pipelines are so important and helpful:

Some time ago we started putting our workloads into containers. We did so because of advantages like isolation, dependencies, scalability and immutability. Wouldn’t these be also helpful in your CI/CD pipeline?

Think of “build-hosts” which only provide the tools and dependencies needed for one particular pipeline task. An environment which is looking the same every run and does not have any dependencies of other projects which might cause issues. As well as easily scalable pipelines.

That’s why we need and should use containerized pipelines!

Now, where we briefly talked about containerized pipelines let’s talk about where Tekton Pipeline with its Kubernetes-native approach can help:

Tekton Pipeline allows us to run our containerized pipelines in our existing Kubernetes Clusters. This means we do not need additional machines to run our pipelines and therefore can better utilize our existing ones.

This is greats but, to be honest, that alone does not make Tekton Pipeline unique. Tekton Pipeline goes one step further and also stores everything related to our pipeline within Kubernetes — as Kubernetes resources. This allows us to work with our pipelines as we do with any other resource. Think of a Deployment or Service which you can create and manage using kubectl and YAML files.

How to get started

As mentioned above Tekton Pipeline lives within a Kubernetes Cluster. It is based on 5 Custom Resource Definitions (CRDs), Deployments, Configmaps and Services. You can run the following command to get started:

Besides the above-mentioned resources, it will also create a Namespace, a Pod Security Policy, a Service Account and ClusterRoles. Tekton Pipeline is ready as soon as all Pods in the newly created Namespace (the default name is tekton-pipelines) are ready.

You can, of course, review the above YAML and customize it based on your needs.

If you need to share artifacts or other pipelines resources between your tasks you will need to configure a storage option. You can either use PVCs which will be requested every time needed (Dynamic Volume Provisioning is key!) or Blob storage. You will find more details on this task here.

A first pipeline

So, how does Tekton Pipelines work? I will explain the different resources (Custom Resource Definitions) of Tekton Pipeline using the small example below. The pipeline will build a small Go application, build the depending Image and then push it into a Registry. You will find all the related files here.

First of all, we will create two PipelineResouce definitions which we will use to provide the source Git repository and the target Registry. The Pipeline Resouces are optional but are very helpful to reuse the same pipeline with different sources and targets.



kind: PipelineResource

metadata:

name: git-repo

spec:

type: git

params:

- name: revision

value: master

- name: url

value:

---

apiVersion: tekton.dev/v1alpha1

kind: PipelineResource

metadata:

name: image-registry

spec:

type: image

params:

- name: url

value: registry.gitlab.com/nmeisenzahl/tekton-demo/demo:latest apiVersion: tekton.dev/v1alpha1kind: PipelineResourcemetadata:name: git-repospec:type: gitparams:- name: revisionvalue: master- name: urlvalue: https://gitlab.com/nmeisenzahl/tekton-demo ---apiVersion: tekton.dev/v1alpha1kind: PipelineResourcemetadata:name: image-registryspec:type: imageparams:- name: urlvalue: registry.gitlab.com/nmeisenzahl/tekton-demo/demo:latest

Now we need to create a Task resource to define the steps of our pipeline. You can, of course, define multiple tasks if you need them. In our example, we will use Kaniko to build our Image. The Dockerfile, as well as the app resources, are stored in the Git repository.

apiVersion: tekton.dev/v1alpha1

kind: Task

metadata:

name: build-docker-image

spec:

inputs:

resources:

- name: git-repo

type: git

params:

- name: pathToDockerFile

description: Path to Dockerfile

default: /workspace/git-repo/Dockerfile

- name: pathToContext

description: The build context used by Kaniko

default: /workspace/git-repo

outputs:

resources:

- name: image-registry

type: image

steps:

- name: build-and-push

image: gcr.io/kaniko-project/executor:v0.10.0

env:

- name: "DOCKER_CONFIG"

value: "/builder/home/.docker/"

command:

- /kaniko/executor

args:

- --dockerfile=${inputs.params.pathToDockerFile}

- --destination=${outputs.resources.image-registry.url}

- --context=${inputs.params.pathToContext}

We could now create a TaskRun resource to run an instance of the above task. However, in this example, we use the Pipeline which we can use to combine multiple tasks in a pipeline:

apiVersion: tekton.dev/v1alpha1

kind: Pipeline

metadata:

name: demo-pipeline

spec:

resources:

- name: git-repo

type: git

- name: image-registry

type: image

tasks:

- name: build-docker-image

taskRef:

name: build-docker-image

params:

- name: pathToDockerFile

value: /workspace/git-repo/Dockerfile

- name: pathToContext

value: /workspace/git-repo

resources:

inputs:

- name: git-repo

resource: git-repo

outputs:

- name: image-registry

resource: image-registry

Because we move the image to a registry, you must ensure that the pipeline can authenticate itself by configuring an ImagePullSecrets for the service account used (in our case, the default service account).

We now have everything in place to finally run the pipeline. For that, we need a final definition. A PipelineRun resource:

apiVersion: tekton.dev/v1alpha1

kind: PipelineRun

metadata:

name: demo-pipeline-run-1

spec:

pipelineRef:

name: demo-pipeline

resources:

- name: git-repo

resourceRef:

name: git-repo

- name: image-registry

resourceRef:

name: image-registry

To verify the status of your pipeline run you can execute kubectl get pipelineruns -oyaml .

There is more

Beside the Tekton Pipeline project itself, there is also a project for a CLI which makes you work with your pipelines easier. You can also install a web-based Dashboard to view and manage your pipelines in a browser.

Furthermore, the team is working on another project called Tekton Triggers. This project is pretty new (the first commit was 4 weeks ago) and still work in progress. Tekton Triggers will allow calling Tekton pipelines based on “triggers”. Those could be git commits, git issues or any kind of webhooks. More details are available here.