Fix concurrency bug in PrepWorkerManagerTest