resque/resque — pull request #1207 by Sirupsen (2014-04-24 02:42:47+00:00)
Title: Pulse

Commits need to be cleaned up, but that'll be done later.

At Shopify, we run Resque workers in Docker containers across a lot of different servers. Each container, and therefore worker, thus has its own hostname. If a worker is shutdown abnormally, and doesn't get to unregister itself, there's no longer anything to clean up after it because the old logic works on a per host basis.

This takes a different approach, where each worker sends heartbeats at a specified interval. When new workers start, instead of looking for dead workers on the same host—it looks for workers which haven't sent a heartbeat for a while and unregisters them.

We've run this in production on 100s of workers restarting many times per day during deploys, and it's worked great.

Note that this is a PR to the 1-x-stable branch.