Fixing jenkins failures

I'm always reluctant to add sleeps into the code (even if it is a nicely implemented one that limits the length of the sleep).
I prefer the `backOffTo` rather than the `every`, e.g. `.backOffTo(Duration.millis(20))`.

I'm ok with merging this because it improves the race. But the race still exists, and may well still bite us on apache jenkins where the machines are low powered so things could take more than 200ms.