Labels conflict when loading a local CSV file.

## Describe the bug
I am trying to load a local CSV file with a separate file containing label names. It is successfully loaded for the first time, but when I try to load it again, there is a conflict between provided labels and the cached dataset info. Disabling caching globally and/or using `download_mode="force_redownload"` did not help.

## Steps to reproduce the bug
```python
load_dataset('csv', data_files='data/my_data.csv',
                        features=Features(text=Value(dtype='string'),
                                          label=ClassLabel(names_file='data/my_data_labels.txt')))
```
`my_data.csv` file has the following structure:
```
text,label
"example1",0
"example2",1
...
```
and the `my_data_labels.txt` looks like this:
```
label1
label2
...
```

## Expected results
Successfully loaded dataset.

## Actual results
```python
  File "/usr/local/lib/python3.8/site-packages/datasets/load.py", line 1706, in load_dataset
    ds = builder_instance.as_dataset(split=split, ignore_verifications=ignore_verifications, in_memory=keep_in_memory)
  File "/usr/local/lib/python3.8/site-packages/datasets/builder.py", line 766, in as_dataset
    datasets = utils.map_nested(
  File "/usr/local/lib/python3.8/site-packages/datasets/utils/py_utils.py", line 261, in map_nested
    mapped = [
  File "/usr/local/lib/python3.8/site-packages/datasets/utils/py_utils.py", line 262, in <listcomp>
    _single_map_nested((function, obj, types, None, True))
  File "/usr/local/lib/python3.8/site-packages/datasets/utils/py_utils.py", line 197, in _single_map_nested
    return function(data_struct)
  File "/usr/local/lib/python3.8/site-packages/datasets/builder.py", line 797, in _build_single_dataset
    ds = self._as_dataset(
  File "/usr/local/lib/python3.8/site-packages/datasets/builder.py", line 872, in _as_dataset
    return Dataset(fingerprint=fingerprint, **dataset_kwargs)
  File "/usr/local/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 638, in __init__
    inferred_features = Features.from_arrow_schema(arrow_table.schema)
  File "/usr/local/lib/python3.8/site-packages/datasets/features/features.py", line 1242, in from_arrow_schema
    return Features.from_dict(metadata["info"]["features"])
  File "/usr/local/lib/python3.8/site-packages/datasets/features/features.py", line 1271, in from_dict
    obj = generate_from_dict(dic)
  File "/usr/local/lib/python3.8/site-packages/datasets/features/features.py", line 1076, in generate_from_dict
    return {key: generate_from_dict(value) for key, value in obj.items()}
  File "/usr/local/lib/python3.8/site-packages/datasets/features/features.py", line 1076, in <dictcomp>
    return {key: generate_from_dict(value) for key, value in obj.items()}
  File "/usr/local/lib/python3.8/site-packages/datasets/features/features.py", line 1083, in generate_from_dict
    return class_type(**{k: v for k, v in obj.items() if k in field_names})
  File "<string>", line 7, in __init__
  File "/usr/local/lib/python3.8/site-packages/datasets/features/features.py", line 776, in __post_init__
    raise ValueError("Please provide either names or names_file but not both.")
ValueError: Please provide either names or names_file but not both.
```

## Environment info
- `datasets` version: 1.18.0
- Python version: 3.8.2