ahmedfarazsyk/datasets-github-issues huggingface dataset:
[Dataset] `NonMatchingChecksumError` due to an update in the LinCE benchmark data
Author: gaguilar
State: closed

Hi,

There is a `NonMatchingChecksumError` error for the `lid_msaea` (language identification for Modern Standard Arabic - Egyptian Arabic) dataset from the LinCE benchmark due to a minor update on that dataset. 

How can I update the checksum of the library to solve this issue? The error is below and it also appears in the [nlp viewer](https://huggingface.co/nlp/viewer/?dataset=lince&config=lid_msaea):

```python
import nlp
nlp.load_dataset('lince', 'lid_msaea')
```

Output:
```
NonMatchingChecksumError: ['https://ritual.uh.edu/lince/libaccess/eyJ1c2VybmFtZSI6ICJodWdnaW5nZmFjZSBubHAiLCAidXNlcl9pZCI6IDExMSwgImVtYWlsIjogImR1bW15QGVtYWlsLmNvbSJ9/lid_msaea.zip']
Traceback:
File "/home/sasha/streamlit/lib/streamlit/ScriptRunner.py", line 322, in _run_script
    exec(code, module.__dict__)
File "/home/sasha/nlp-viewer/run.py", line 196, in <module>
    dts, fail = get(str(option.id), str(conf_option.name) if conf_option else None)
File "/home/sasha/streamlit/lib/streamlit/caching.py", line 591, in wrapped_func
    return get_or_create_cached_value()
File "/home/sasha/streamlit/lib/streamlit/caching.py", line 575, in get_or_create_cached_value
    return_value = func(*args, **kwargs)
File "/home/sasha/nlp-viewer/run.py", line 150, in get
    builder_instance.download_and_prepare()
File "/home/sasha/.local/share/virtualenvs/lib-ogGKnCK_/lib/python3.7/site-packages/nlp/builder.py", line 432, in download_and_prepare
    download_config.force_download = download_mode == FORCE_REDOWNLOAD
File "/home/sasha/.local/share/virtualenvs/lib-ogGKnCK_/lib/python3.7/site-packages/nlp/builder.py", line 469, in _download_and_prepare
File "/home/sasha/.local/share/virtualenvs/lib-ogGKnCK_/lib/python3.7/site-packages/nlp/utils/info_utils.py", line 36, in verify_checksums
    raise NonMatchingChecksumError(str(bad_urls))
```

Thank you in advance!

@lhoestq 

Comments: Hi @gaguilar 

If you want to take care of this, it very simple, you just need to regenerate the `dataset_infos.json` file as indicated [in the doc](https://huggingface.co/nlp/share_dataset.html#adding-metadata) by [installing from source](https://huggingface.co/nlp/installation.html#installing-from-source) and running the following command from the root of the repo:
```bash
python nlp-cli test ./datasets/lince --save_infos --all_configs
```
And then you can open a pull-request with the updated json file.

Otherwise we'll do it sometime this week.

Hi @thomwolf 

Thanks for the details! I just created a PR with the updated `dataset_infos.json` file (#550).

Thanks for updating the json file. Closing this one
`list_datasets()` is broken.
Author: ashutosh-dwivedi-e3502
State: closed

version = '0.4.0'

`list_datasets()` is broken. It results in the following error : 

```
In [3]: nlp.list_datasets()
Out[3]: ---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
~/.virtualenvs/san-lgUCsFg_/lib/python3.8/site-packages/IPython/core/formatters.py in __call__(self, obj)
    700                 type_pprinters=self.type_printers,
    701                 deferred_pprinters=self.deferred_printers)
--> 702             printer.pretty(obj)
    703             printer.flush()
    704             return stream.getvalue()

~/.virtualenvs/san-lgUCsFg_/lib/python3.8/site-packages/IPython/lib/pretty.py in pretty(self, obj)
    375                 if cls in self.type_pprinters:
    376                     # printer registered in self.type_pprinters
--> 377                     return self.type_pprinters[cls](obj, self, cycle)
    378                 else:
    379                     # deferred printer

~/.virtualenvs/san-lgUCsFg_/lib/python3.8/site-packages/IPython/lib/pretty.py in inner(obj, p, cycle)
    553                 p.text(',')
    554                 p.breakable()
--> 555             p.pretty(x)
    556         if len(obj) == 1 and type(obj) is tuple:
    557             # Special case for 1-item tuples.

~/.virtualenvs/san-lgUCsFg_/lib/python3.8/site-packages/IPython/lib/pretty.py in pretty(self, obj)
    392                         if cls is not object \
    393                                 and callable(cls.__dict__.get('__repr__')):
--> 394                             return _repr_pprint(obj, self, cycle)
    395
    396             return _default_pprint(obj, self, cycle)

~/.virtualenvs/san-lgUCsFg_/lib/python3.8/site-packages/IPython/lib/pretty.py in _repr_pprint(obj, p, cycle)
    698     """A pprint that just redirects to the normal repr function."""
    699     # Find newlines and replace them with p.break_()
--> 700     output = repr(obj)
    701     lines = output.splitlines()
    702     with p.group():

~/.virtualenvs/san-lgUCsFg_/lib/python3.8/site-packages/nlp/hf_api.py in __repr__(self)
    110
    111     def __repr__(self):
--> 112         single_line_description = self.description.replace("\n", "")
    113         return f"nlp.ObjectInfo(id='{self.id}', description='{single_line_description}', files={self.siblings})"
    114

AttributeError: 'NoneType' object has no attribute 'replace'
```

Comments: Thanks for reporting !
This has been fixed in #475 and the fix will be available in the next release

What you can do instead to get the list of the datasets is call

```python
print([dataset.id for dataset in nlp.list_datasets()])
```

Thanks @lhoestq .
wmt download speed example
Author: sshleifer
State: closed

Continuing from the slack 1.0 roadmap thread w @lhoestq , I realized the slow downloads is only a thing sometimes. Here are a few examples, I suspect there are multiple issues. All commands were run from the same gcp us-central-1f machine.

```
import nlp
nlp.load_dataset('wmt16', 'de-en')
```
Downloads at 49.1 KB/S

Whereas 
```
pip install gdown # download from google drive
!gdown https://drive.google.com/uc?id=1iO7um-HWoNoRKDtw27YUSgyeubn9uXqj
```
Downloads at 127 MB/s. (The file is a copy of wmt-en-de raw).


```
nlp.load_dataset('wmt16', 'ro-en')
```
goes at 27 MB/s, much faster. 

if we wget the same data from s3 is the same download speed, but ¼ the file size:
```
wget https://s3.amazonaws.com/datasets.huggingface.co/translation/wmt_en_ro_packed_200_rand.tgz
```

Finally,
```
nlp.load_dataset('wmt19', 'zh-en')
```
Starts fast, but broken. (duplicate of #493 )




Comments: Thanks for creating the issue :)
The download link for wmt-en-de raw looks like a mirror. We should use that instead of the current url.
Is this mirror official ?

Also it looks like for `ro-en` it tried to download other languages. If we manage to only download the one that is asked it'd be cool

Also cc @patrickvonplaten 

Mirror is not official.

Shall we host the files ourselves or it is fine to use this mirror in your opinion ?

Should we add an argument in `load_dataset` to override some URL with a custom URL (e.g. mirror) or a local path?

This could also be used to provide local files instead of the original files as requested by some users (e.g. when you made a dataset with the same format than SQuAD and what to use it instead of the official dataset files).

@lhoestq I think we should host it ourselves. I'll put the subset of wmt (without preprocessed files) that we need on s3 and post a link over the weekend.

Is there a solution yet? The download speed is still too slow. 60-70kbps download for wmt16 and around 100kbps for wmt19. @sshleifer 

I'm working on mirror links which will provide high download speed :)
See https://github.com/huggingface/datasets/issues/1892

Resolved via https://github.com/huggingface/datasets/pull/1912
[BUG] Metrics throwing new error on master since 0.4.0
Author: jbragg
State: closed

The following error occurs when passing in references of type `List[List[str]]` to metrics like bleu.
Wasn't happening on 0.4.0 but happening now on master.

```
  File "/usr/local/lib/python3.7/site-packages/nlp/metric.py", line 226, in compute
    self.add_batch(predictions=predictions, references=references)
  File "/usr/local/lib/python3.7/site-packages/nlp/metric.py", line 242, in add_batch
    batch = self.info.features.encode_batch(batch)
  File "/usr/local/lib/python3.7/site-packages/nlp/features.py", line 527, in encode_batch
    encoded_batch[key] = [encode_nested_example(self[key], cast_to_python_objects(obj)) for obj in column]
  File "/usr/local/lib/python3.7/site-packages/nlp/features.py", line 527, in <listcomp>
    encoded_batch[key] = [encode_nested_example(self[key], cast_to_python_objects(obj)) for obj in column]
  File "/usr/local/lib/python3.7/site-packages/nlp/features.py", line 456, in encode_nested_example
    raise ValueError("Got a string but expected a list instead: '{}'".format(obj))
```

Comments: Update - maybe this is only failing on bleu because I was not tokenizing inputs to the metric

Closing - seems to be just forgetting to tokenize. And found the helpful discussion in huggingface/evaluate#105
add MLDoc dataset
Author: jxmorris12
State: open
Labels: name: dataset request

Hi,

I am recommending that someone add MLDoc, a multilingual news topic classification dataset.

- Here's a link to the Github: https://github.com/facebookresearch/MLDoc
- and the paper: http://www.lrec-conf.org/proceedings/lrec2018/pdf/658.pdf

Looks like the dataset contains news stories in multiple languages that can be classified into four hierarchical groups: CCAT (Corporate/Industrial), ECAT (Economics), GCAT (Government/Social) and MCAT (Markets). There are 13 languages: Dutch, French, German, Chinese, Japanese, Russian, Portuguese, Spanish, Latin American Spanish, Italian, Danish, Norwegian, and Swedish

Comments: Any updates on this?

This request is still an open issue waiting to be addressed by any community member, @GuillemGSubies.
Version of numpy to use the library
Author: isspek
State: closed

Thank you so much for your excellent work! I would like to use nlp library in my project. While importing nlp, I am receiving the following error `AttributeError: module 'numpy.random' has no attribute 'Generator'` Numpy version in my project is 1.16.0. May I learn which numpy version is used for the nlp library.

Thanks in advance.

Comments: Seems like this method was added in 1.17. I'll add a requirement on this.

Thank you so much. After upgrading the numpy library, it worked.
Converting TensorFlow dataset example
Author: saareliad
State: closed

Hi,
I want to use TensorFlow datasets with this repo, I noticed you made some conversion script,
can you give a simple example of using it?

Thanks


Comments: Do you want to convert a dataset script to the tfds format ?
If so, we currently have a comversion script nlp/commands/convert.py but it is a conversion script that goes from tfds to nlp.
I think it shouldn't be too hard to do the changes in reverse (at some manual adjustments).
If you manage to make it work in reverse, feel free to open a PR to share it with the community :)

In our docs: [Using a Dataset with PyTorch/Tensorflow](https://huggingface.co/docs/datasets/torch_tensorflow.html).
TypeError: Receiver() takes no arguments
Author: sebastiantomac
State: closed

I am trying to load a wikipedia data set

```
import nlp
from nlp import load_dataset

dataset = load_dataset("wikipedia", "20200501.en", split="train", cache_dir=data_path, beam_runner='DirectRunner')
#dataset = load_dataset('wikipedia', '20200501.sv', cache_dir=data_path, beam_runner='DirectRunner')
```

This fails in the apache beam runner. 

```
Traceback (most recent call last):
  File "D:/ML/wikiembedding/gpt2_sv.py", line 36, in <module>
    dataset = load_dataset("wikipedia", "20200501.en", split="train", cache_dir=my_cache_dir, beam_runner='DirectRunner')
  File "C:\Users\seto\AppData\Local\Programs\Python\Python38\lib\site-packages\nlp\load.py", line 548, in load_dataset
    builder_instance.download_and_prepare(
  File "C:\Users\seto\AppData\Local\Programs\Python\Python38\lib\site-packages\nlp\builder.py", line 462, in download_and_prepare
    self._download_and_prepare(
  File "C:\Users\seto\AppData\Local\Programs\Python\Python38\lib\site-packages\nlp\builder.py", line 969, in _download_and_prepare
    pipeline_results = pipeline.run()
  File "C:\Users\seto\AppData\Local\Programs\Python\Python38\lib\site-packages\apache_beam\pipeline.py", line 534, in run
    return self.runner.run_pipeline(self, self._options)
....
  File "C:\Users\seto\AppData\Local\Programs\Python\Python38\lib\site-packages\apache_beam\runners\worker\bundle_processor.py", line 218, in process_encoded
    self.output(decoded_value)
  File "C:\Users\seto\AppData\Local\Programs\Python\Python38\lib\site-packages\apache_beam\runners\worker\operations.py", line 332, in output
    cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value)
  File "C:\Users\seto\AppData\Local\Programs\Python\Python38\lib\site-packages\Cython\Shadow.py", line 167, in cast
    return type(*args)
TypeError: Receiver() takes no arguments

```

This is run on a Windows 10 machine with python 3.8. I get the same error loading the swedish wikipedia dump.

Comments: Which version of Apache Beam do you have (can you copy your full environment info here)?

apache-beam==2.23.0
nlp==0.4.0

For me this was resolved by running the same python script on Linux (or really WSL). 

Do you manage to run a dummy beam pipeline with python on windows ? 
You can test a dummy pipeline with [this code](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/wordcount_minimal.py)

If you get the same error, it means that the issue comes from apache beam.
Otherwise we'll investigate what went wrong here

Still, same error, so I guess it is on apache beam then. 
Thanks for the investigation.

Thanks for trying
Let us know if you find clues of what caused this issue, or if you find a fix
Errors when I use 
Author: mchari
State: closed

I tried the following example code from https://huggingface.co/deepset/roberta-base-squad2 and got errors 
I am using **transformers 3.0.2** code .


from transformers.pipelines import pipeline
from transformers.modeling_auto import AutoModelForQuestionAnswering
from transformers.tokenization_auto import AutoTokenizer

model_name = "deepset/roberta-base-squad2"

nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
QA_input = {
    'question': 'Why is model conversion important?',
    'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.'
}
res = nlp(QA_input)

The errors are :

res = nlp(QA_input)
  File ".local/lib/python3.6/site-packages/transformers/pipelines.py", line 1316, in __call__
    for s, e, score in zip(starts, ends, scores)
  File ".local/lib/python3.6/site-packages/transformers/pipelines.py", line 1316, in <listcomp>
    for s, e, score in zip(starts, ends, scores)
KeyError: 0



Comments: Looks like an issue with 3.0.2 transformers version. Works fine when I use "master" version of transformers.
Caching doesn't work for map (non-deterministic)
Author: wulu473
State: closed

The caching functionality doesn't work reliably when tokenizing a dataset. Here's a small example to reproduce it. 

```python
import nlp
import transformers

def main():
    ds = nlp.load_dataset("reddit", split="train[:500]")

    tokenizer = transformers.AutoTokenizer.from_pretrained("gpt2")

    def convert_to_features(example_batch):
        input_str = example_batch["body"]
        encodings = tokenizer(input_str, add_special_tokens=True, truncation=True)
        return encodings

    ds = ds.map(convert_to_features, batched=True)

if __name__ == "__main__":
    main()
```

Roughly 3/10 times, this example recomputes the tokenization.

Is this expected behaviour?

Comments: Thanks for reporting !

To store the cache file, we compute a hash of the function given in `.map`, using our own hashing function.
The hash doesn't seem to stay the same over sessions for the tokenizer.
Apparently this is because of the regex at `tokenizer.pat` is not well supported by our hashing function.

I'm working on a fix

Thanks everyone. Works great now.

Hi. I believe the fix was for the nlp library. Is there a solution to handle compiled regex expressions in .map() with the caching. I want to run a simple regex pattern on a big dataset, but I am running into the issue of compiled expression not being cached. 

Instead of opening a new issue, I thought I would put my query here. Let me know if a new issue would be more suitable. Thanks

Hi @MaveriQ! This fix is also included in the `datasets` library. Can you provide a reproducer?
nlp.Features does not distinguish between nullable and non-nullable types in PyArrow schema
Author: jarednielsen
State: closed

Here's the code I'm trying to run:

```python
dset_wikipedia = nlp.load_dataset("wikipedia", "20200501.en", split="train", cache_dir=args.cache_dir)
dset_wikipedia.drop(columns=["title"])
dset_wikipedia.features.pop("title")
dset_books = nlp.load_dataset("bookcorpus", split="train", cache_dir=args.cache_dir)
dset = nlp.concatenate_datasets([dset_wikipedia, dset_books])
```

This fails because they have different schemas, despite having identical features.

```python
assert dset_wikipedia.features == dset_books.features # True
assert dset_wikipedia._data.schema == dset_books._data.schema # False
```

The Wikipedia dataset has 'text: string', while the BookCorpus dataset has 'text: string not null'. Currently I hack together a working schema match with the following line, but it would be better if this was handled in Features themselves.

```python
dset_wikipedia._data = dset_wikipedia.data.cast(dset_books._data.schema)
```


Comments: In 0.4.0, the assertion in `concatenate_datasets ` is on the features, and not the schema.
Could you try to update `nlp` ?

Also, since 0.4.0, you can use `dset_wikipedia.cast_(dset_books.features)` to avoid the schema cast hack.

Or maybe the assertion comes from elsewhere ?

I'm using the master branch. The assertion failure comes from the underlying `pa.concat_tables()`, which is in the pyarrow package. That method does check schemas.

Since `features.type` does not contain information about nullable vs non-nullable features, the `cast_()` method won't resolve the schema mismatch. There is information in a schema which is not stored in features.

I'm doing a refactor of type inference in #363 . Both text fields should match after that

By default nullable will be set to True

It should be good now. I was able to run

```python
>>> from nlp import concatenate_datasets, load_dataset
>>>
>>> bookcorpus = load_dataset("bookcorpus", split="train")
>>> wiki = load_dataset("wikipedia", "20200501.en", split="train")
>>> wiki.remove_columns_("title")  # only keep the text
>>>
>>> assert bookcorpus.features.type == wiki.features.type
>>> bert_dataset = concatenate_datasets([bookcorpus, wiki])
```

Thanks!
No 0.4.0 release on GitHub
Author: jarednielsen
State: closed

0.4.0 was released on PyPi, but not on GitHub. This means [the documentation](https://huggingface.co/nlp/) is still displaying from 0.3.0, and that there's no tag to easily clone the 0.4.0 version of the repo.

Comments: I did the release on github, and updated the doc :)
Sorry for the delay

Thanks!
Loading preprocessed Wikipedia dataset requires apache_beam
Author: jarednielsen
State: closed

Running 

`nlp.load_dataset("wikipedia", "20200501.en", split="train", dir="/tmp/wikipedia")`

gives an error if apache_beam is not installed, stemming from

https://github.com/huggingface/nlp/blob/38eb2413de54ee804b0be81781bd65ac4a748ced/src/nlp/builder.py#L981-L988

This succeeded without the dependency in version 0.3.0. This seems like an unnecessary dependency to process some dataset info if you're using the already-preprocessed version. Could it be removed?
PAWS dataset first item is header
Author: jxmorris12
State: closed

```
import nlp
dataset = nlp.load_dataset('xtreme', 'PAWS-X.en')
dataset['test'][0]
```

prints the following

```
{'label': 'label', 'sentence1': 'sentence1', 'sentence2': 'sentence2'}
```

dataset['test'][0] should probably be the first item in the dataset, not just a dictionary mapping the column names to themselves. Probably just need to ignore the first row in the dataset by default or something like that.
rotten tomatoes movie review dataset taken down
Author: jxmorris12
State: closed

In an interesting twist of events, the individual who created the movie review seems to have left Cornell, and their webpage has been removed, along with the movie review dataset (http://www.cs.cornell.edu/people/pabo/movie-review-data/rt-polaritydata.tar.gz). It's not downloadable anymore.

Comments: found a mirror: https://storage.googleapis.com/seldon-datasets/sentence_polarity_v1/rt-polaritydata.tar.gz

fixed in #484 

Closing this one. Thanks again @jxmorris12 for taking care of this :)
Export TFRecord to GCP bucket
Author: astariul
State: closed

Previously, I was writing TFRecords manually to GCP bucket with : `with tf.io.TFRecordWriter('gs://my_bucket/x.tfrecord')`

Since `0.4.0` is out with the `export()` function, I tried it. But it seems TFRecords cannot be directly written to GCP bucket.

`dataset.export('local.tfrecord')` works fine,  
but `dataset.export('gs://my_bucket/x.tfrecord')` does not work. 

There is no error message, I just can't find the file on my bucket...

---

Looking at the code, `nlp` is using `tf.data.experimental.TFRecordWriter`, while I was using `tf.io.TFRecordWriter`.  

**What's the difference between those 2 ? How can I write TFRecords files directly to GCP bucket ?**

@jarednielsen @lhoestq 

Comments: Nevermind, I restarted my python session and it worked fine...

---

I had an authentification error, and I authenticated from another terminal. After that, no more error but it was not working. Restarting the sessions makes it work :)