Cleanup compiled directory for test case