Stop smashing retry_keys, since removing the dict comprehension resulted in ignoring existing keys.
Use a proper const rather than overriding the meaning of 'False' or 'True' to make code easier to read.
Reimplement a full sweep per retry of all the hosts, rather than backing off per host in a cluster.
Avoid thread-leaks which can obscure debug output if a test fails by ensuring every test joins its thread.
Make it more obvious if assumptions are broken in the test harness.