Don't reuse compile dirs, delete compile dir when done