Remove file that was renamed