Building Your Own Kubernetes Operator Easily
By: Emmanuel Sys
Tags: Golang, Kubernetes, DevOps, Containers, Programming

Developing the Job-Watcher Controller

We’ll develop the controller by following the Operator pattern.

Operator?

An Operator is a special kind of Kubernetes controller process that comes with its own custom-resource definition — that is to say, a Kubernetes object. These CRDs allow you, in turn, to extend the functionalities of the Kubernetes API. You can read more about it in the official documentation.

Instead of writing all of the code ourselves, we’ll use the Operator SDK. It’s essentially a code generator to scaffold a fully functional operator.

First, we’ll install the SDK and run a few commands to bootstrap the project.

brew install operator-sdk operator-sdk init job-purger --domain my.github.com --repo github.com/xxx/job-watcher-operator operator-sdk create api --group batch --kind JobWatcher --version v1 --resource true --controller true

Create your CRD

You should now have a whole project created for you. Let’s look first at the api/v1/jobwatch_types.go .

Here we’ve defined our CRD with its specs and status information:

A separate TTL for completed or failed jobs, after which the job will be deleted.

Some namespace and job-name patterns (regex) to identify jobs that may be a candidate for deletion.

The delay between two checks for deletion.

The status information will be the last started and finished times. Note that our Operator is not running concurrently by default.

The special comments like //+kubebuilder will generate constraint code for us.

Implement the reconciliation loop

You can find the full controller code here.

Let’s move to the implementation code. The main logic is in the reconciliation loop. It’ll receive a Request as an argument, which only contains the namespace and the name of a resource.

This method will be called for every object your Operator is concerned with, starting, of course, with our custom JobWatcher CRD. Moreover, the generated Operator comes out of the box with a logger and a Kubernetes client to manipulate objects.

The implemented logic is straightforward:

First, we’ll fetch the JobWatcher object matching the Request . Then, we’ll list and retain namespaces matching one of our namespace patterns. Next, we’ll identify jobs with matching names, and if the job is terminated and the TTL is expired, we’ll delete it. Finally, we’ll update the status information for our CRD.

Discussion

Reconcile is called either when one of your CRDs change or if the returned ctrl.Result isn’t empty (or an error is returned). We exploit this behavior to reschedule the call to the Reconcile function at the frequency given in the CRD spec.

The cluster role and binding are automatically created for you if you specify the generator comments: //+kubebuilder:rbac:groups:resources:verbs .

In the SetupWithManager method, we indicate the Operator is managing the JobWatcher CRD through the For method. Other methods allow you to be notified for other objects:

Owns() for child objects you’re creating.

for child objects you’re creating. Watches() for any objects in the cluster.

For example, if you add Owns(&kbatch.Job{}) , your reconciliation loop will be called for every jobs that’s created, deleted, or modified — and with your CRD as its owner. The input Request parameter of the Reconcile method will be the owner instance of the job. Of course, here we’re not creating jobs, so it’s just for sake of example.

More relevant to us, if you add a call to the Watches method in the builder for the Job kind, you’ll be notified for every change in a job in the cluster. You can (and should) filter the events you’re interested in by setting appropriate options.

Finally, don’t forget to update the status of your CRD at the end of the loop.