linkedin/sensei — comment on commit 2ca216ed21e0b982e3ba4a2c5fbc1f8ea6beb4cc
File:  line 0
By chtyim:

Hi Josh,

Sure. It's good to know that there is plan to fix Sensei to use Norbert correct.

For the approach I take, merging all partitions into one ring is only to reduce the number of lookup per routing (1 lookup vs N lookups for N partitions). When there is changes to node list (add/remove) or partitions, a new ConsistentHashLoadBalancer instance would be created, so no keep iterating (rolling) situation will arise. The promise of the merge is that for clients not routed to the down node won't get affected. For partitions that are affected, clients routed to the down node will evenly distribution (sort of, by the hashing function) among other replicas for that partition.

E.g.
Node 1 has partitions 1,2,3
Node 2 has partitions 2,3
Node 3 has partitions 1,2,3

So the ring might looks like:

1 -> 1,1,2 (p1 handled by node1, p2 handled by node1, p3 handled by node2)
5 -> 3,2,1
20 -> 3,1,2
55 -> 1,3,3
….

Now, if node 2 dies, a new ring will generate and it will look like (notice that routing for partition 1 won't change. Also routing for partition 2 and 3 also won't change for points that were not routed to node 2).

1 -> 1,1,1 (node 2 for p3 replace with the next one down the ring, I.e. Node 1)
5 -> 3,1,1
20 -> 3,1,3
55 -> 1,3,3

You can refer to the "testConsisitentHashRemove" test case for more details.

Terence