Remove file as a result of renaming