Cleanup the source tree in move mode even for ignored files