# Run Jobs

This is the multi-page printable view of this section. Click here to print.Return to the regular view of this page.Run JobsRun Jobs using parallel processing.1: Running Automated Tasks with a CronJob2: Coarse Parallel Processing Using a Work Queue3: Fine Parallel Processing Using a Work Queue4: Indexed Job for Parallel Processing with Static Work Assignment5: Job with Pod-to-Pod Communication6: Parallel Processing using Expansions7: Handling retriable and non-retriable pod failures with Pod failure policy1 - Running Automated Tasks with a CronJobThis page shows how to run automated tasks using Kubernetes CronJob object.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesCreating a CronJobCron jobs require a config file. Here is a manifest for a CronJob that runs a simple demonstration task every minute:application/job/cronjob.yaml apiVersion: batch/v1 kind: CronJob metadata: name: hello spec: schedule: "* * * * *" jobTemplate: spec: template: spec: containers: - name: hello image: busybox:1.28 imagePullPolicy: IfNotPresent command: - /bin/sh - -c - date; echo Hello from the Kubernetes cluster restartPolicy: OnFailure Run the example CronJob by using this command:kubectl create -f https://k8s.io/examples/application/job/cronjob.yaml The output is similar to this:cronjob.batch/hello created After creating the cron job, get its status using this command:kubectl get cronjob hello The output is similar to this:NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE hello */1 * * * * False 0 <none> 10s As you can see from the results of the command, the cron job has not scheduled or run any jobs yet. Watch for the job to be created in around one minute:kubectl get jobs --watch The output is similar to this:NAME COMPLETIONS DURATION AGE hello-4111706356 0/1 0s hello-4111706356 0/1 0s 0s hello-4111706356 1/1 5s 5s Now you've seen one running job scheduled by the "hello" cron job. You can stop watching the job and view the cron job again to see that it scheduled the job:kubectl get cronjob hello The output is similar to this:NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE hello */1 * * * * False 0 50s 75s You should see that the cron job hello successfully scheduled a job at the time specified in LAST SCHEDULE. There are currently 0 active jobs, meaning that the job has completed or failed.Now, find the pods that the last scheduled job created and view the standard output of one of the pods.Note:The job name is different from the pod name.# Replace "hello-4111706356" with the job name in your system pods=$(kubectl get pods --selector=job-name=hello-4111706356 --output=jsonpath={.items[*].metadata.name}) Show the pod log:kubectl logs $pods The output is similar to this:Fri Feb 22 11:02:09 UTC 2019 Hello from the Kubernetes cluster Deleting a CronJobWhen you don't need a cron job any more, delete it with kubectl delete cronjob <cronjob name>:kubectl delete cronjob hello Deleting the cron job removes all the jobs and pods it created and stops it from creating additional jobs. You can read more about removing jobs in garbage collection.2 - Coarse Parallel Processing Using a Work QueueIn this example, you will run a Kubernetes Job with multiple parallel worker processes.In this example, as each pod is created, it picks up one unit of work from a task queue, completes it, deletes it from the queue, and exits.Here is an overview of the steps in this example:Start a message queue service. In this example, you use RabbitMQ, but you could use another one. In practice you would set up a message queue service once and reuse it for many jobs.Create a queue, and fill it with messages. Each message represents one task to be done. In this example, a message is an integer that we will do a lengthy computation on.Start a Job that works on tasks from the queue. The Job starts several pods. Each pod takes one task from the message queue, processes it, and exits.Before you beginYou should already be familiar with the basic, non-parallel, use of Job.You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesYou will need a container image registry where you can upload images to run in your cluster.This task example also assumes that you have Docker installed locally.Starting a message queue serviceThis example uses RabbitMQ, however, you can adapt the example to use another AMQP-type message service.In practice you could set up a message queue service once in a cluster and reuse it for many jobs, as well as for long-running services.Start RabbitMQ as follows:# make a Service for the StatefulSet to use kubectl create -f https://kubernetes.io/examples/application/job/rabbitmq/rabbitmq-service.yaml service "rabbitmq-service" created kubectl create -f https://kubernetes.io/examples/application/job/rabbitmq/rabbitmq-statefulset.yaml statefulset "rabbitmq" created Testing the message queue serviceNow, we can experiment with accessing the message queue. We will create a temporary interactive pod, install some tools on it, and experiment with queues.First create a temporary interactive Pod.# Create a temporary interactive container kubectl run -i --tty temp --image ubuntu:22.04 Waiting for pod default/temp-loe07 to be running, status is Pending, pod ready: false ... [ previous line repeats several times .. hit return when it stops ] ... Note that your pod name and command prompt will be different.