Ruby background processing  worker error handling comparison
Tags: ruby, sidekiq, resque
Published: 2021-08-06 08:25:50

![Photo of a coffee shop worker by Valeria Boltneva](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r19f27nfqxut2yxjfovg.jpg)

How do Ruby background job processing gems do their error handling? 

I have had to look into this, as I have my own background job gem called [Belated](https://rubygems.org/gems/belated), so I want to know a good way to do it in my own gem. 

A word about Belated: it runs in a different process, but uses [dRuby](https://github.com/ruby/drb) to connect your main app and Belated. 

So, currently, in Belated I am just rescuing `StandardError`. 
But what happens if someone passes a job with perform method like below? (failing RSpec test)

```ruby
it 'will not crash with syntax errors' do
  expect { 
    @worker.job_list.push(
      proc { raise SyntaxError }
    )
    sleep 0.01
  }.not_to raise_error
end
```

This raises an error (SyntaxError: compile error).

So let's look at how other job libraries deal with this! 

## Sidekiq

Sidekiq is pretty much the de-facto job processing gem for Ruby. At my day job, we depend on Sidekiq to run business critical jobs having to with live streams.  

Sidekiq seems to rescue Exception, but it also has a manager class to handle these exceptions:

```ruby
def run
  process_one until @done
  @mgr.processor_stopped(self)
rescue Sidekiq::Shutdown
  @mgr.processor_stopped(self)
rescue Exception => ex
  @mgr.processor_died(self, ex)
end
```

If you rescue Exception only, you end up rescuing too much, so Sidekiq is also taking care of properly handling shutdown by rescuing `Sidekiq::Shutdown` first. 

A very clever approach with really nice logging too!

[Source](https://github.com/mperham/sidekiq/blob/e8df351f0a6b222807ecf2e05a98149187b02e48/lib/sidekiq/processor.rb#L72)

## Resque

Resque, like Sidekiq, needs Redis to function. It's an older gem, still in use in many places, but Resque seems to be quite a bit slower than Sidekiq. (See [this article to learn more](https://dev.to/molly/switching-from-resque-to-sidekiq-3b04))

Resque also rescues `Exception`, and then checks if that exception is shutdown. 

```ruby
def work(interval = 5.0, &block)
  interval = Float(interval)
  # skipping a lot of code here...
rescue Exception => exception
  return if exception.class == SystemExit && !@child && run_at_exit_hooks
  log_with_severity :error, "Failed to start worker : #{exception.inspect}"
  unregister_worker(exception)
  run_hook :worker_exit
end
``` 

I have to say I'm not completely sure how this is working, but the gist of it should be that in case there is something so wrong that we get a syntax error the worker will shutdown. 
Correct me if I'm wrong! 

[Source](https://github.com/resque/resque/blob/master/lib/resque/worker.rb#L251)

## GoodJob

GoodJob is an interesting new gem that uses PostgreSQL to handle connecting the processes. 

It seems to just rescue StandardError. 

```ruby
def execute
  # code that executes the job
  rescue StandardError => e
    ExecutionResult.new(value: nil, unhandled_error: e)
  end
end
```

This makes handling shutdown much easier. 
GoodJob, like Sidekiq, allows you to have an error handler that sends your errors to a service like Airbreak, or maybe emails them to you using an exception handling gem. 
But something like a SyntaxError will probably not get reported. (Probably quite a rare case though... and feel free to correct me if I'm wrong!)

[Source](https://github.com/bensheldon/good_job/blob/main/lib/good_job/job.rb#L329)

## Conclusion

So... what to do with Belated? 
I think the right thing would be to just rescue Exception, but when you rescue exception, you have to also take into account shutdown. You don't want to make it impossible to actually close the program with CTRL + C. 

Sidekiq's approach looks really promising, but as excepted, also the most demanding one. So something like it... but simpler. For now, Belated will just crash if you feed it a proc with a syntax error in it. 

And here's what I ended up doing:

```ruby
rescue Exception => e
  case e.class
  when Interrupt, SignalException
    raise e
  else
    e.inspect
  end
end
```
Just check if I actually want to rescue the error. Normally rescuing Exception is overdoing it, and you get a Rubocop warning for it too, but in a background job process you want to keep the worker working even if there is a syntax error in the job is processing.