jplu (CONTRIBUTOR) commented on 2020-04-15 07:52:18:
Ok, there are some news, most good than bad :laughing: 

The dataset script now became:
```python
import csv

import nlp


class Bbc(nlp.GeneratorBasedBuilder):
    VERSION = nlp.Version("1.0.0")

    def __init__(self, **config):
        self.train = config.pop("train", None)
        self.validation = config.pop("validation", None)
        super(Bbc, self).__init__(**config)

    def _info(self):
        return nlp.DatasetInfo(builder=self, description="bla", features=nlp.features.FeaturesDict({"id": nlp.int32, "text": nlp.string, "label": nlp.string}))

    def _split_generators(self, dl_manager):
        return [nlp.SplitGenerator(name=nlp.Split.TRAIN, gen_kwargs={"filepath": self.train}),
                nlp.SplitGenerator(name=nlp.Split.VALIDATION, gen_kwargs={"filepath": self.validation})]

    def _generate_examples(self, filepath):
        with open(filepath) as f:
            reader = csv.reader(f, delimiter=',', quotechar="\"")
            lines = list(reader)[1:]

            for idx, line in enumerate(lines):
                yield idx, {"id": idx, "text": line[1], "label": line[0]}

```

And the dataset folder becomes:
```
.
├── bbc
│   ├── bbc.py
│   └── data
│       ├── test.csv
│       └── train.csv
```
I can load the dataset by using the keywords arguments like this:
```python
import nlp
dataset = nlp.load("bbc", builder_kwargs={"train": "bbc/data/train.csv", "validation": "bbc/data/test.csv"})
```

That was the good part ^^ Because it took me some time to understand that the script itself is put in cache in `datasets/src/nlp/datasets/some-hash/bbc.py` which is very difficult to discover without checking the source code. It means that doesn't matter the changes you do to your original script it is taken into account. I think instead of doing a hash on the name (I suppose it is the name), a hash on the content of the script itself should be a better solution.

Then by diving a bit in the code I found the `force_reload` parameter [here](https://github.com/huggingface/datasets/blob/master/src/nlp/load.py#L50) but the call of this `load_dataset` method is done with the `builder_kwargs` as seen [here](https://github.com/huggingface/datasets/blob/master/src/nlp/load.py#L166) which is ok until the call to the builder is done as the builder do not have this `force_reload` parameter. To show as example, the previous load becomes:
```python
import nlp
dataset = nlp.load("bbc", builder_kwargs={"train": "bbc/data/train.csv", "validation": "bbc/data/test.csv", "force_reload": True})
```
Raises
```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jplu/dev/jplu/datasets/src/nlp/load.py", line 283, in load
    dbuilder: DatasetBuilder = builder(path, name, data_dir=data_dir, **builder_kwargs)
  File "/home/jplu/dev/jplu/datasets/src/nlp/load.py", line 170, in builder
    builder_instance = builder_cls(**builder_kwargs)
  File "/home/jplu/dev/jplu/datasets/src/nlp/datasets/84d638d2a8ca919d1021a554e741766f50679dc6553d5a0612b6094311babd39/bbc.py", line 12, in __init__
    super(Bbc, self).__init__(**config)
TypeError: __init__() got an unexpected keyword argument 'force_reload'
```
So yes the cache is refreshed with the new script but then raises this error.