Clean up temp directories

Used to not trigger deferred cleanup because of calling syscall.Exec.