Delete files that were readded by the last merge