Redis: running Master-Slave replication in Kubernetes
Tags: redis, database, kubernetes, tutorial
Published: 2020-09-29 11:43:25

---
title: Redis: running Master-Slave replication in Kubernetes
published: true
date: 2020-09-29 06:20:01 UTC
tags: redis,database,kubernetes,tutorial
canonical_url: https://rtfm.co.ua/en/redis-running-master-slave-replication-in-kubernetes/
---

![](https://cdn-images-1.medium.com/max/1024/1*qhdhG6g9cJRKcGcZnUa-kA.png)

The task is to spin up a Redis instance in a Kubernetes cluster.

Will use the Master-Slave replication setup with Sentinels for monitoring and failover operations.

Check the [Redis: replication, part 2 — Master-Slave replication, and Redis Sentinel](https://rtfm.co.ua/en/redis-replication-part-2-master-slave-replication-and-redis-sentinel/) post for more details.

### Redis cluster vs Redis replication

See [Redis: replication, part 1 — an overview. Replication vs Sharding. Sentinel vs Cluster. Redis topology.](https://rtfm.co.ua/en/redis-replication-part-1-overview-replication-vs-sharding-sentinel-vs-cluster-redis-topology/) and [Choose between Redis Helm Chart and Redis Cluster Helm Chart](https://github.com/bitnami/charts/tree/master/bitnami/redis#choose-between-redis-helm-chart-and-redis-cluster-helm-chart).

In short:

- _Replica_ — includes a Redis Master instance that performs read-write operations and copies data to its Redis Slaves instance(s) which serves Read-only operations. During this, such a Salve can be promoted to the Master’s role if its Master fails.
- _Cluster_ — have a sense when your Redis have more data than your server’s RAM. The Cluster can use [Sharding](https://rtfm.co.ua/redis-replikaciya-chast-1-obzor-replication-vs-sharding-sentinel-vs-cluster-topologiya-redis/#Sharding) and a client requesting a piece of data will be redirected to a node that keeps that data.

### Ways to run Redis in Kubernetes

Let’s see how we can perform the task — to run a Redis with replication in a Kubernetes cluster.

- manual setup  —  see the [How to Create a Primary-Replica Redis Cluster in Kubernetes](https://tech.ringieraxelspringer.com/blog/cloud/how-to-create-a-primary-replica-redis-cluster-in-kubernetes/r8lt028)
- Redis Operator  —  [redis-operator](https://github.com/spotahome/redis-operator)
- Helm chart which the Redis Cluster  — [https://bitnami.com/stack/redis-cluster](https://bitnami.com/stack/redis-cluster)
- Helm with Redis Master-Slave replication  — [https://bitnami.com/stack/redis](https://bitnami.com/stack/redis) (our choice in this post)

In our current case, we don’t need to worry about data’s persistence as our Redis will be used as a cache service only so we don’t need for a [Kubernetes PersistentVolume](https://rtfm.co.ua/kubernetes-persistentvolume-i-persistentvolumeclaim-obzor-i-primery/).

### Helm chart deploy

At first, we will run Redis services from the chart, will take a short look at them, and then will proceed to the available [parameters](https://github.com/bitnami/charts/tree/master/bitnami/redis#parameters).

Add the Bitnami repository to your Helm:

```
$ helm repo add bitnami https://charts.bitnami.com/bitnami
“bitnami” has been added to your repositories
```

Deploy the Redis chart:

```
$ helm install backend-redis bitnami/redis
NAME: backend-redis
LAST DEPLOYED: Tue Sep 22 14:48:02 2020
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:

\*\* Please be patient while the chart is being deployed \*\*

Redis can be accessed via port 6379 on the following DNS names from within your cluster:

backend-redis-master.default.svc.cluster.local for read/write operations
backend-redis-slave.default.svc.cluster.local for read-only operations

To get your password run:

export REDIS\_PASSWORD=$(kubectl get secret — namespace default backend-redis -o jsonpath=”{.data.redis-password}” | base64 — decode)

To connect to your Redis server:

1. Run a Redis pod that you can use as a client:

kubectl run — namespace default backend-redis-client — rm — tty -i — restart=’Never’ \
 — env REDIS\_PASSWORD=$REDIS\_PASSWORD \
 — image docker.io/bitnami/redis:6.0.8-debian-10-r0 — bash

2. Connect using the Redis CLI:

redis-cli -h backend-redis-master -a $REDIS\_PASSWORD
redis-cli -h backend-redis-slave -a $REDIS\_PASSWORD

To connect to your database from outside the cluster execute the following commands:

kubectl port-forward — namespace default svc/backend-redis-master 6379:6379 &
redis-cli -h 127.0.0.1 -p 6379 -a $REDIS\_PASSWORD
```

Get the password which was generated during the chart’s deployment:

```
$ kubectl get secret — namespace default backend-redis -o jsonpath=”{.data.redis-password}” | base64 — decode
TySS43UhAW
```

Run the[port-forward](https://rtfm.co.ua/en/kubernetes-clusterip-vs-nodeport-vs-loadbalancer-services-and-ingress-an-overview-with-examples/#kubectl_portforward) to connect to the Redis Master instance:

```
$ kubectl port-forward — namespace default svc/backend-redis-master 6379:6379
Forwarding from [::1]:6379 -> 6379
Forwarding from 127.0.0.1:6379 -> 6379
```

Connect:

```
$ redis-cli -h 127.0.0.1 -p 6379 -a TySS43UhAW
Warning: Using a password with ‘-a’ or ‘-u’ option on the command line interface may not be safe.
127.0.0.1:6379>
```

_“It works!” (с)_

Check which Services do we have here:

```
$ kk get svc redis-service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEredis-service NodePort 172.20.119.242 <none> 6379:32445/TCP 117d
```

Its type is the `NodePort` while we need for a `LoadBalancer`.

Also, we need for Redis Sentinel, which is off by default:

> sentinel.enabled Enable sentinel containers `false`

Okay, let’s go to the options to enable Sentinel and configure a LoadBalancer.

#### Redis Options

Create a list of options that can be useful for you.

In my case this will be:

- global.redis.password
- metrics:
- metrics.enabled
- metrics.serviceMonitor.enabled
- metrics.serviceMonitor.namespace
- Persistent:
- master.persistence.enabled
- slave.persistence.enabled
- Service
- master.service.type
- master.service.annotations
- slave.service.type
- slave.service.annotations
- Sentinel
- sentinel.enabled
- sentinel.service.type — need for LBso clients can ask for Master/Slaves
- sentinel.service.annotations

Create a new file `~/Temp/redis-opts.yaml` to keep our desired parameters:

```
global:
  redis:
    password: "blablacar"

metrics:
  enabled: true
  serviceMonitor:
    enabled: true
    namespace: "monitoring"

master:
  persistence:
    enabled: false
  service:
    type: LoadBalancer
    annotations: 
      kubernetes.io/ingress.class: alb
      alb.ingress.kubernetes.io/scheme: internal 

slave:
  persistence:
    enabled: false
  service:
    type: LoadBalancer
    annotations:
      kubernetes.io/ingress.class: alb
      alb.ingress.kubernetes.io/scheme: internal

sentinel:
  enabled: true
  service:
    type: LoadBalancer
    annotations: 
      kubernetes.io/ingress.class: alb
      alb.ingress.kubernetes.io/scheme: internal
```

Update the deployment with `-f` to specify our parameters file:

```
$ helm upgrade — install backend-redis bitnami/redis -f ~/Temp/redis-opts.yaml
```

Check the password:

```
$ kubectl get secret — namespace default backend-redis -o jsonpath=”{.data.redis-password}” | base64 — decode
blablacar
```

And load-balancer:

```
$ kk get svc -l app=redis
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

backend-redis LoadBalancer 172.20.204.235 af5e1294a4a73426692c7e25f7bb947d-915967.us-east-2.elb.amazonaws.com 
6379:30647/TCP,26379:32523/TCP 80s

backend-redis-headless ClusterIP None <none> 6379/TCP,26379/TCP 80s

backend-redis-metrics ClusterIP 172.20.66.2 <none> 9121/TCP 80s
```

Connect:

```
$ redis-cli -h af5e1294a4a73426692c7e25f7bb947d-915967.us-east-2.elb.amazonaws.com -a blablacar
Warning: Using a password with ‘-a’ or ‘-u’ option on the command line interface may not be safe.\
af5e1294a4a73426692c7e25f7bb947d-915967.us-east-2.elb.amazonaws.com:6379>
```

Works, okay.

But why the LoadBalancer is Public when it was set to be _internal_?

This because the `alb.ingress.kubernetes.io/scheme: internal` is used for the [ALB Ingress Controller](https://rtfm.co.ua/en/aws-elastic-kubernetes-service-running-alb-ingress-controller/) while the chart creates a simple Kubernetes Service with the LoadBalancer type which will create an AWS Classic Load Balancer.

Read the documentation — [https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer](https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer) and update our annotations: instead of the “`alb.ingress.kubernetes.io/scheme: internal`" - specify the "`service.beta.kubernetes.io/aws-load-balancer-internal: "true"`":

```
...
master:
  service:
    type: LoadBalancer
    annotations: 
      service.beta.kubernetes.io/aws-load-balancer-internal: "true"

slave:
  service:
    type: LoadBalancer
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-internal: "true"

sentinel:
  enabled: true
  service:
    type: LoadBalancer
    annotations: 
      service.beta.kubernetes.io/aws-load-balancer-internal: "true"
```

Update the deployment and check again:

```
$ kk get svc -l app=redis
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

backend-redis LoadBalancer 172.20.178.72 internal-\*\*\*-1786016015.us-east-2.elb.amazonaws.com 6379:31192/TCP,26379:32239/TCP 17s\

backend-redis-headless ClusterIP None <none> 6379/TCP,26379/TCP 17s

backend-redis-metrics ClusterIP 172.20.35.163 <none> 9121/TCP 17s
```

Now, make a Write operation — must be served by the Redis Master instance:

```
$ admin@bttrm-dev-app-1:~$ redis-cli -h internal-\*\*\*-1786016015.us-east-2.elb.amazonaws.com -p 6379 -a blablacar SET testkey testvalue
OK
```

And a Read operation:

```
admin@bttrm-dev-app-1:~$ redis-cli -h internal-\*\*\*-1786016015.us-east-2.elb.amazonaws.com -p 6379 -a blablacar GET testkey
“testvalue”
```

Replication status:

```
admin@bttrm-dev-app-1:~$ redis-cli -h internal-\*\*\*-1786016015.us-east-2.elb.amazonaws.com -p 6379 -a blablacar info replication
Replication
role:master
connected\_slaves:1
slave0:ip=10.3.50.119,port=6379,state=online,offset=144165,lag=1
…
```

And Sentinel status:

```
admin@bttrm-dev-app-1:~$ redis-cli -h internal-\*\*\*-1786016015.us-east-2.elb.amazonaws.com -p 26379 -a blablacar info sentinel
Sentinel
sentinel\_masters:1
sentinel\_tilt:0
sentinel\_running\_scripts:0
sentinel\_scripts\_queue\_length:0
sentinel\_simulate\_failure\_flags:0
master0:name=mymaster,status=ok,address=10.3.33.107:6379,slaves=1,sentinels=2
```

Although write operations need to be performed after getting the Master’s address via Sentinels:

```
admin@bttrm-dev-app-1:~$ redis-cli -h internal-\*\*\*-1786016015.us-east-2.elb.amazonaws.com -p 26379 -a blablacar sentinel get-master-addr-by-name mymaster
1) “10.3.33.107”
r2) “6379”
```

Check the [documentation](https://github.com/bitnami/charts/tree/master/bitnami/redis/#master-slave-with-sentinel).

Done.

_Originally published at_ [_RTFM: Linux, DevOps, and system administration_](https://rtfm.co.ua/en/redis-running-master-slave-replication-in-kubernetes/)_._

* * *