Remove files that should be ignored