Copy Redis Data to New Server
Tags: redis, devops, ruby
Published: 2022-08-04 19:28:08

---
title: Copy Redis Data to New Server
published: true
description: 
tags: redis, devops, ruby

# cover_image: https://direct_url_to_image.jpg
# Use a ratio of 100:42 for best results.
---

Sometimes we need to copy redis data to new instance

I know 3 simple ways to do it:
* continuous migration tools
* simple script to copy data
* redis master-slave replication

Other options, more complex:
* Redis HA & sentinel solution - it will require rebuild redis
* https://github.com/Netflix/dynomite - replication and routing

## redis-migrate-tool

https://github.com/vipshop/redis-migrate-tool

It monitors changes in source server/cluster and apply them in destination server. Useful when we migrate app that actively writes to redis to use new redis instance. There are other similar tools, but this one works and easy to configure.

Check README for supported types, It's written in C, easier to run it with docker

```bash
docker pull ahmadgh21/redis-migrate-tool
docker run -it ahmadgh21/redis-migrate-tool bash
# inside container
mkdir /tmp
chmod 777 /tmp
apt-get update
apt install nano
nano /etc/redis-migrate-tool/rmt.conf
```
Config example
```toml
[source]
type: single
hash: fnv1a_64
hash_tag: "{}"
distribution: ketama
servers :
- <source IP>:6379 # <--- change this

[target]
type: single
servers:
- <destination IP>:6379 # <--- change this

[common]
listen: 0.0.0.0:8888
```

run migrate-tool inside container
```
/opt/redis-migrate-tool -c /etc/redis-migrate-tool/rmt.conf
```

## Make own script

This script simply copy all keys from server A to server B (it doesn't support all data types, find it in code)

```ruby
require 'redis'

puts "Connecting to redis://#{ARGV[0]} and redis://#{ARGV[1]}"

redis_src = Redis.new(url: "redis://#{ARGV[0]}")
redis_dest = Redis.new(url: "redis://#{ARGV[1]}")

dest_keys = redis_dest.keys("*")

redis_src.keys("*").each do |key|
  next if dest_keys.include?(key)
  type = redis_src.type(key)

  if type == "hash"
    value = redis_src.hgetall(key)
    puts "HASH   #{key} : #{value}"
    redis_dest.mapped_hmset(key, value)
  elsif type == "list"
    values = redis_src.lrange(key, 0, -1)
    puts "LIST.  #{key} : #{values}"
    redis_dest.lpush(key, values)
  elsif type == "set"
    values = redis_src.smembers(key)
    puts "SET    #{key} : #{values}"
    redis_dest.sadd(key, values)
  else
    data = redis_src.get(key)
    puts "STRING #{key} : #{data}"
    redis_dest.set(key, data)
  end
end

puts "Complete"
```

Make sure you have ruby redis library installed (`gem install redis`)

Run the script

```bash
ruby redis_copy.rb source_IP destination_IP
```

## Setup redis replication

I think this is most proper option but also require more preparation

on master we need to update config

```
protected-mode no
requirepass 1234abcd
```

on replica server we need to put in config:
```
replicaof <master ip> 6379
masterauth 1234abcd
```

restart master, then replica, check logs

check status in redis-cli
```
INFO replication
```

more docs:
* https://redis.io/docs/manual/replication/
* https://www.vultr.com/docs/how-to-set-up-a-redis-replication-cluster
* https://medium.com/dlt-labs-publication/how-to-setup-redis-replication-c9cc89ba6c03

## Pros and Cons

**Simple script** - It copy values once, can be used for not critical data such as cache, rate limit, and for rarely updated data such as configuration

I usually run it once before updating apps to connect to new server, and once again after to make sure all data is in sync

**redis-migrate-tool** - Constantly replicates all data from source to destination. For apps that constantly write updates to redis, e.g. user sessions, temporary important values.

But it doesn't replicate writes from destination server back to source. To avoid situation when one process write/update to new server and other process can't find it in old server, we need to restart all our apps at one time with updated config. (I usually scale app down to 1 replica during migration)

**redis replication** - replica is not writable, migration may require short downtime or we need to modify code to handle it smoothly. In our code we can try to save data to new server, and if got "not writable" error then save to old server

---

Have a smooth redis migration to everyone who's doing it now :)