Refactor worker to use the new retry stuff