mperham/sidekiq — comment on commit 1a028c4a2f41ef7458e34aa634bba0575cc4127e
File: lib/sidekiq/middleware.rb line 94
By mperham:

RE: msg, if the msg is 100k, the set member will be 100k. If you use a hash, the set member will be 32 bytes not matter what the message size is. You just need the set member to be a unique fingerprint of the msg.

RE: leak, the network could die and then when the srem fails, that set member is now there forever. Redis would need to support expiring set members or we'd need to support a collection of sets, e.g. one set every 15 minutes, that we can manually expire the entire contents after so much time.