Cleanup temporary files during directory removal (fixes #919)

Seems to fix it.