Refactor to include less stuff directly in ActiveJob::Retry but use Retrier classes

Refactor all the things! This should make it epicly easy to customise.
Seems not, there's a spec failure if this line is removed.