Use with statement instead of addCleanup

Fixes failing tests on windows.