[misc] Fix flickering test by improving the test isolation.