Make test more resilient against scheduling decisions.

Should bring the atom buildbots back to life.