improved directory cleanup

Using os.removedirs now, so it is possible that the walk will end up on
empty directories. Be sure to check that the path exists still since
removedirs might have removed it.