Retry when there are too many participants. Make the retrys configurable
Fixes for retry logic

* Make sure setting retries to 0 will still run
* Sleep before retrying instead of after. Otherwise the final sleep is
pointless
* Make log messages a little more helpful