Support (configurable) retries

Fixes #103 
Is this not settable via the config file?