Workaround for the test_parallel_wait test

This test sometimes failed because of the thread switching.