Remove build directory

Rely on the temp dir providied by the operating system instead.