Prevent an exception with large num_retries.

If the calculated wait time is infinite, don't try to do
further calculations on it to prevent it from becoming NaN.
This could happen if num_retries becomes too large (>1024).