[RECOVERY] Make recovery delay configurable

Today we wait 500ms before we retry a recovery if the target node is not ready.
This happens if the source starts the recovery before the target has
processed the clusterstate moving the target shard into the right state.
This can cause a 500ms delay each time it happens while the shard is ready
way earlier on the target node. This commit makes this delay configurable
to mainly speed up test processing and shard allocation in tests.