Sidekiq (+ some cleanup)

Ohai Sidekiq

### Todo

* [ ] Write a migration guide
* [ ] Find out sane defaults (probably out of scope of this PR)
* [ ] Have more people test this.