antirez/redis — pull request #448 by parrish (2012-04-11 06:15:43+00:00)
Title: Added ZSET command ZDIFFSTORE

I've posted to an existing topic on the [google group](https://groups.google.com/forum/?fromgroups#!topic/redis-db/8X9pZ2EIhlU) (pending approval) to explain our use case, but I'll go ahead and duplicate it here.

We use Redis to select the next object to be shown to a user for classification.

Two things are important in this selection:

1. Users are shown objects only once (unique selection)
2. Objects are scored based upon prior classifications (the score is updated in real time) and users should be shown the most highly scored object they haven't seen.

Currently we do the following:

- Store many keys for object scores (`objects_score_`)
- Store a set of object ids seen by a user (`seen_objects_for_`)
- Store a set of object ids available for classification (`objects`)
- Remove the diff set after selection is done (since the stored order from the score is invalidated shortly after creation)

Then perform:

```
sdiffstore user__unseen_objects objects seen_objects_for_user_
sort user__unseen_objects DESC BY objects_score_* LIMIT 0 1
del user__unseen_objects
```

Given we have over 600,000 users and well over 10,000,000 objects between our projects, staying responsive while handling rates above 100 requests/second is non-trivial. Optimizing this specific use-case is pretty critical for us.

Please feel free to suggest changes, revisions, etc.

Thanks!

-Michael