Remove temp dir on error.

If error occurs remove the temp directory we created. Also
simplify code for setting pth.