collectiveidea/delayed_job — pull request #661 by denisahearn (2014-05-23 14:08:02+00:00)
Title: Added error handling when job fails permanently and failure callback rai...

...ses an exception

Here is our situation that prompted the changes in this pull request:

1. Configuration

Delayed::Worker.destroy_failed_jobs = false
Delayed::Worker.max_attempts = 1

2. We have a custom job class that has a perform method that is failing, causing the job to fail.

3. The custom job class implements the failure callback. A bug in the callback causes an exception to be raised.

4. The "failed" method in worker.rb does not have any error handling logic in it to catch the exception, so the line of code that calls job.fail! doesn't happen. As a result, the job gets "stuck" in the queue. The unhandled exception also causes the worker to stop/exit.