Add retry support

cc @ccare, @malcyL, @mmmmmrob 

I have a really fuzzy head today, so please give this a sanity check before merging. Thanks :)