resque/resque — pull request #1155 by davetron5000 (2013-12-12 15:45:50+00:00)
Title: Avoid masking exceptions that cause unregistering

When unregistering a worker, there are various calls to Redis to try to fail the job that's in progress, however if said job is failing due to an unhandled exception but we then get an exception accessing Redis itself, the original exception is raised, making it impossible to figure out what else might've gone wrong and how to fix it.

FWIW, I see this very frequently and the log message "Failed to start worker {ex.message}" is never rendered.

To show how masking the real exception is a problem, consider the following situation:

1. Queue worker to charge someone $50
2. Will contacting the gateway, we loose the network
3. Job fails with "Connection Timed Out"
4. Exception bubbles up and Resque unregisters the worker, with the timeout exception as the value `exception` passed to `unregister_worker`.
5. When checking to see if still processing a job, timeout to Redis (which you could imagine might happen if Redis is cloud-hosted and the network issue were on the network where workers were running)

In the current implementation, we don't see the timeout to our credit card processor, and we don't see what job was even running, so we can't event see that there was a problem charging someone $50

The change below would favor the original exception to be raised, instead of the Redis Timeout, in this case.

Let me know if there is a better way to do this.