Be more cautious when deleting the temp directory