antirez/redis — pull request #1686 by yossigo (2014-04-15 17:03:17+00:00)
Title: Volatile (non-replicating) scripts

I know the issue of script replication was discussed a lot in the past, but this may be another point of view and possibly a significant improvement in a few cases.

We came across the following scenario:
- One Redis master with multiple slaves (1-10), used for data availability and potentially load balancing of read-only operations.
- Clients update data on master Redis every second.
- LUA needs to perform potentially intensive aggregation of that data and produce summary records.

We want to optimize:
- Avoid replicating the per-second data updates to all slaves, as we simply don't need them (nobody reads them from slaves, and they get re-written every second anyway so nothing to lose).
- Avoid running the intensive LUA code on every slave, run it once in the master and just make the resulting aggregated records available to all slaves.

It seems like this can be a relatively common real-world optimization case, so I came up with a proposed mechanism for volatile scripts. It has two parts:

VEVAL and VEVALSHA commands which allow executing LUA scripts in Volatile Mode. When scripts execute in this mode, they are not replicated to slaves as we assume nothing they do is worth replication.

Essentially this introduces a potentially interesting concept of "volatile keys", but for now we don't mark them in any way and simply don't replicate these changes.

When our script finally needs to create the aggregated record, it calls a new redis.nvcall() LUA function which is identical to call/pcall in all aspects but one - commands performed through it are replicated to slaves (as discrete Redis commands) if data was changed.