I'm not sure that this is correct. My understanding is that a given test could change on each non-keyframe run. So the test result might already in the queue, and should not count against numTestsLeft, as other test results might not yet be loaded.

I might however misunderstand how this works, didn't look too deeply into it. Could you add some comments to clarify the reasoning?