form26/sidekiq-unique-jobs — pull request #4 by kemper-blinq (2012-09-27 18:17:54+00:00)
Title: Fix multiple bugs, cleaned up dependencies, and added a feature

We found that the gem you published would not release processed jobs
so they could be enqueued a second time. The problem was that the data
that was used to generate the md5 value in the server middleware was
not the same as the client middleware. We fixed that and added a test
with both of them interacting.

The way that hashing was done was insufficient as well; it would try
to hash based on just the arguments passed to sidekiq. Without
considering class name and queue name, you can never be sure that the
job is unique. We changed the hash logic to consider class, queue, and
args.

You had added logic that created very large expiration values for the
keys in redis for delayed/scheduled sidekiq jobs. Not sure what you
had in mind there, but made it so that the key expires
HASH_KEY_EXPIRATION seconds after it's scheduled to run. This makes it
comparable to what happens when you enqueue a job to be processed
immediately.

The final fix/cleanup was that the gem was written to require too
much. The gemfile duplicated dependencies of sidekiq that don't need
to be declared, and the middleware.rb file required sidekiq components
that would actually load the middleware files. This causes problems.
For instance, the middleware loads celluloid, and celluloid starts a
worker thread. That worker thread causes the exit code of specs to
always be successful making our ci server always report passing specs.
We cleaned up those dependency/load issues.

We also added the ability to use a value other than
HASH_KEY_EXPIRATION when configuring a job. This is passed in as a
sidekiq_option called 'unique_job_expiration'. We updated the readme
to reflect that option.

Unfortunately, one of the specs we added fails periodically; I'm not sure why, but it seems like the multi command causes some wierdness. I didn't really change that code, but I'm hoping it's not a sign of a larger problem.