MongoDB Backups with Kubernetes Jobs
Tags: kubernetes, mongodb, databases, systemadministration
Published: 2018-12-30 22:29:45

---
title: MongoDB Backups with Kubernetes Jobs
published: true
description: Using Kubernetes Jobs to create and automate persistent data backups
tags: kubernetes, mongodb, databases, system-administration
---

I recently began [running RocketChat on Kubernetes](https://dev.to/jmarhee/running-rocketchat-and-mongodb-on-kubernetes-with-statefulsets-431o)--a key component of this deployment was a MongoDB replica set. A best practice for running MongoDB reliably is, both, replication __and__ regular backups, and Kubernetes provides accessible interfaces for both approaches. 

In my case, I wanted regular and one-off backup capability, and the Kubernetes `Jobs` resource provided me a quick way to do this. I wanted my job pod to write out these dumpfiles to a persistent data store, so I first setup a `PersistentVolume` and accompanying `Claim` to that volume:

```yaml
kind: PersistentVolume
apiVersion: v1
metadata:
  name: mongo-dump-pv-volume
  labels:
    type: local
    app: mongo-dump
spec:
  storageClassName: manual
  capacity:
    storage: 50Gi
  accessModes:
    - ReadWriteMany
  hostPath:
    path: "/mnt/kube-data/mongo-dumps"
    type: DirectoryOrCreate
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: mongo-dump-pv-claim
  labels:
    app: postgres
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 50Gi
```
If you use a cloud provider like AWS or Azure, a PersistentVolume can provision block storage, so if your provider does make [durability guarantees](https://aws.amazon.com/ebs/features/) then your data volume is that much more persistent. The above example just uses `hostPath` volumes, so will persist for the lifecycle of the host that path resides on. 

Your Job, itself, will have a podspec, much like other Kubernetes resources like `Deployments` where you make requests for resources, and it'll look like this:

```yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: mongodb-backup
  labels:
    app: mongo-dump
spec:
  backoffLimit: 5
  activeDeadlineSeconds: 100
  template:
    spec:
      containers:
      - name: mongodump
        image: mongo
        command: ["mongodump","--host","mongo-service:27017","--db","your_db"]
        volumeMounts:
          - mountPath: dump
            name: mongo-dumps
      volumes:
      - name: mongo-dumps
        persistentVolumeClaim:
          claimName: mongo-dump-pv-claim
      restartPolicy: OnFailure
```

You'll see we're attaching the volume as we might normally, and then run the `mongodump` command, which will write out to the mount path, `dump`. 

If you have authentication enabled on your Mongo service, or use a SaaS like MongoDB Atlas, you can use `Secrets` like you might normally to pass through credentials being stored securely, not in the Job spec itself:

```yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: mongodb-backup
  labels:
    app: mongo-dump
spec:
  backoffLimit: 5
  activeDeadlineSeconds: 100
  template:
    spec:
      containers:
      - name: mongodump
        image: mongo
        env:
          - name: MONGO_CONN_STRING
            valueFrom:
              secretKeyRef:
                name: mongo-auth
                key: connstring
          - name: MONGO_DB
            value: "my_db"
        command: ["mongodump","--host","$MONGO_CONN_STRING","--db","$MONGO_DB"]
        volumeMounts:
          - mountPath: dump
            name: mongo-dumps
      volumes:
      - name: mongo-dumps
        persistentVolumeClaim:
          claimName: mongo-dump-pv-claim
      restartPolicy: Never
```

After you apply this Job spec, you can monitor your progress:

```
kubectl get pods -l app=mongo-dump
```
then monitor the logs for that pod name:

```
kubectl logs $POD_NAME
```

If you see your job failing, you can use the `restartPolicy` to define behavior in this area; for example, in the declaration above, it will not restart, but you can, for example, use `OnFailure` to attempt a retry, and use other options available to to define retires, backoff timing, etc.

[CronJobs](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#creating-a-cron-job) are another type of Job supported in Kubernetes presently (since 1.7), and in the link above, you'll see that the spec is similar, but includes your typical [Cron syntax](https://crontab.guru/) for defining when you'd like the job run, and related behavior.