Fix invalid removing of ignored files