Make test more deterministic

This should resolve potential race condition issues