pressly/qmd — pull request #10 by dkua (2014-06-24 13:54:07+00:00)
Title: Dedup

This is for issue #5 the duplicated message problem. The solution uses Redis as a locking mechanism whereas handlers who consume the message will check for and create a key similar to `Job #:id`. All handlers after the first will be locked out until 900 seconds (15 minutes) until the key expires.

900 seconds is how long NSQ will allow a message to be in the system by default. If the NSQ `max-message-timeout` was increased or our `JOB_TIMEOUT` was decreased then it would be possible for QMD to have duplicate messages in the system. Clock skew could also be an issue if there is a huge skew.