Actually delete files which were moved in #1807