Remove files that are supposed to be ignored