Don't reset retries when we find a new subdir. Fixes #39