Properly remove directories on overwrite