Don't die if dir isn't deleted

If we fail to rmdir (perhaps a permission problem), don't die. This
silently continues and does leave empty folders behind, but stops things
from breaking until the solution is really found.