antirez/redis — Issue #189: Replication: partial resync after link goes down then up.
Author: antirez
Created: 2011-11-14 12:25:26+00:00
State: open
Labels: feature request, more design needed, waiting user input

This is an excerpt from a message I wrote in the mailing list explaining the possible solution for this problem.
Basically now after master and slave connection goes down a full resync is needed. Redis is able to resync in a very short time but with big datasets and short link down times it is still not very good to perform a full resync, so in the future I want to fix this definitely. The following is a description of the solution I've in mind:

if I understand correctly what you want is faster synchronization
between master and slaves after the replication link is down?
You propose your solution (that is useful in other context, for a
possible unification of RDB and AOF in the future), but we are
implementing a better replication resync with a different approach
than the one proposed.

Basically when a slave is no longer connected the master will stil
accumulate writes for him instead of freeing the client structure
associated with the slave.

Then the slave will reconnect again, but will issue an optional "SYNC
" command, specifying the absolute offset of all the master ->
slave chat of the previous connection. If we have still the buffer in
memory (this depends on the amount of time and on the amount of data
written on the server while the connection is down, since after some
time we have to gave up and free the client structure) we start a
partial synchronization.

Also in order to do that we need to be sure that the slave instance
chatting with us is the same as before. In order to do that we'll
assign an unique Redis instance ID to every instance at every run.
This instance ID will be communicated in the course of SYNC requests.