Add sleepers to reflect real directory behavior

This test fix is in preparation for the async library, which doesn't necessarily notify of directory changes all at once within nanoseconds. It could take a few milliseconds though