Remove the temporary pykit directories