ahmedfarazsyk/datasets-github-issues huggingface dataset:
Overview.ipynb throws exceptions with nlp 0.4.0
Author: mandy-li
State: closed

with nlp 0.4.0, the TensorFlow example in Overview.ipynb throws the following exceptions:


---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-5-48907f2ad433> in <module>
----> 1 features = {x: train_tf_dataset[x].to_tensor(default_value=0, shape=[None, tokenizer.max_len]) for x in columns[:3]}
      2 labels = {"output_1": train_tf_dataset["start_positions"].to_tensor(default_value=0, shape=[None, 1])}
      3 labels["output_2"] = train_tf_dataset["end_positions"].to_tensor(default_value=0, shape=[None, 1])
      4 tfdataset = tf.data.Dataset.from_tensor_slices((features, labels)).batch(8)

<ipython-input-5-48907f2ad433> in <dictcomp>(.0)
----> 1 features = {x: train_tf_dataset[x].to_tensor(default_value=0, shape=[None, tokenizer.max_len]) for x in columns[:3]}
      2 labels = {"output_1": train_tf_dataset["start_positions"].to_tensor(default_value=0, shape=[None, 1])}
      3 labels["output_2"] = train_tf_dataset["end_positions"].to_tensor(default_value=0, shape=[None, 1])
      4 tfdataset = tf.data.Dataset.from_tensor_slices((features, labels)).batch(8)

AttributeError: 'numpy.ndarray' object has no attribute 'to_tensor'

Comments: Thanks for reporting this issue

There was a bug where numpy arrays would get returned instead of tensorflow tensors.
This is fixed on master.

I tried to re-run the colab and encountered this error instead:

```
AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'to_tensor'
```

This is because the dataset returns a Tensor and not a RaggedTensor.
But I think we should always return a RaggedTensor unless the length of the sequence is fixed (it that case they can be stack into a Tensor).

Hi, I got another error (on Colab):

```python
# You can read a few attributes of the datasets before loading them (they are python dataclasses)
from dataclasses import asdict

for key, value in asdict(datasets[6]).items():
    print('👉 ' + key + ': ' + str(value))

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-6-b8ace6c227a2> in <module>()
      2 from dataclasses import asdict
      3 
----> 4 for key, value in asdict(datasets[6]).items():
      5     print('👉 ' + key + ': ' + str(value))

/usr/local/lib/python3.6/dist-packages/dataclasses.py in asdict(obj, dict_factory)
   1008     """
   1009     if not _is_dataclass_instance(obj):
-> 1010         raise TypeError("asdict() should be called on dataclass instances")
   1011     return _asdict_inner(obj, dict_factory)
   1012 

TypeError: asdict() should be called on dataclass instances
```

Indeed we'll update the cola with the new release coming up this week.
test_load_real_dataset when config has BUILDER_CONFIGS that matter
Author: marcotcr
State: closed

It a dataset has custom `BUILDER_CONFIGS` with non-keyword arguments (or keyword arguments with non default values), the config is not loaded during the test and causes an error.
I think the problem is that `test_load_real_dataset` calls `load_dataset` with `data_dir=temp_data_dir` ([here](https://github.com/huggingface/nlp/blob/master/tests/test_dataset_common.py#L200)). This causes [this line](https://github.com/huggingface/nlp/blob/master/src/nlp/builder.py#L201) to always be false because `config_kwargs` is not `None`. [This line](https://github.com/huggingface/nlp/blob/master/src/nlp/builder.py#L222) will be run instead, which doesn't use `BUILDER_CONFIGS`.

For an example, you can try running the test for lince:
` RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_real_dataset_lince`
which yields
> E           TypeError: __init__() missing 3 required positional arguments: 'colnames', 'classes', and 'label_column'

Comments: The `data_dir` parameter has been removed. Now the error is `ValueError: Config name is missing`

As mentioned in #470 I think we can have one test with the first config of BUILDER_CONFIGS, and another test that runs all of the configs in BUILDER_CONFIGS

This was fixed in #527 

Closing this one, but feel free to re-open if you have other questions
DEFAULT_TOKENIZER import error in sacrebleu
Author: idoh
State: closed

Latest Version 0.3.0

When loading the metric "sacrebleu" there is an import error due to the wrong path
![image](https://user-images.githubusercontent.com/5303103/88633063-2c5e5f00-d0bd-11ea-8ca8-4704dc975433.png)


Comments: This issue was resolved by #447
Keep loading old file even I specify a new file in load_dataset
Author: joshhu
State: closed
Labels: name: dataset bug

I used load a file called 'a.csv' by 
```
dataset = load_dataset('csv', data_file='./a.csv')
```
And after a while, I tried to load another csv called 'b.csv'
```
dataset = load_dataset('csv', data_file='./b.csv')
```
However, the new dataset seems to remain the old 'a.csv' and not loading new csv file.

Even worse, after I load a.csv, the load_dataset function keeps loading the 'a.csv' afterward. 

Is this a cache problem?


Comments: Same here !

This is the only fix I could come up with without touching the repo's code.
```python
from nlp.builder import FORCE_REDOWNLOAD
dataset = load_dataset('csv', data_file='./a.csv', download_mode=FORCE_REDOWNLOAD, version='0.0.1')
```
You'll have to change the version each time you want to load a different csv file.
If you're willing to add a ```print```, you can go to ```nlp.load``` and add ```print(builder_instance.cache_dir)``` right before the ```return ds``` in the ```load_dataset``` method. It'll print the cache folder, and you'll just have to erase it (and then you won't need the change here above).
Cannot unpickle saved .pt dataset with torch.save()/load()
Author: vegarab
State: closed

Saving a formatted torch dataset to file using `torch.save()`. Loading the same file fails during unpickling:

```python
>>> import torch
>>> import nlp

>>> squad = nlp.load_dataset("squad.py", split="train")
>>> squad
Dataset(features: {'source_text': Value(dtype='string', id=None), 'target_text': Value(dtype='string', id=None)}, num_rows: 87599)
>>> squad = squad.map(create_features, batched=True)
>>> squad.set_format(type="torch", columns=["source_ids", "target_ids", "attention_mask"])
>>> torch.save(squad, "squad.pt")

>>> squad_pt = torch.load("squad.pt")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/vegarab/.conda/envs/torch/lib/python3.7/site-packages/torch/serialization.py", line 593, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/home/vegarab/.conda/envs/torch/lib/python3.7/site-packages/torch/serialization.py", line 773, in _legacy_load
    result = unpickler.load()
  File "/home/vegarab/.conda/envs/torch/lib/python3.7/site-packages/nlp/splits.py", line 493, in __setitem__
    raise ValueError("Cannot add elem. Use .add() instead.")
ValueError: Cannot add elem. Use .add() instead.
```
where `create_features` is a function that tokenizes the data using `batch_encode_plus` and returns a Dict with `input_ids`, `target_ids` and `attention_mask`. 
```python
def create_features(batch):
    source_text_encoding = tokenizer.batch_encode_plus(
        batch["source_text"],
        max_length=max_source_length,
        pad_to_max_length=True,
        truncation=True)

    target_text_encoding = tokenizer.batch_encode_plus(
        batch["target_text"],
        max_length=max_target_length,
        pad_to_max_length=True,
        truncation=True)

    features = {
        "source_ids": source_text_encoding["input_ids"],
        "target_ids": target_text_encoding["input_ids"],
        "attention_mask": source_text_encoding["attention_mask"]
    }

    return features
```

I found a similar issue in [issue 5267 in the huggingface/transformers repo](https://github.com/huggingface/transformers/issues/5267) which was solved by downgrading to `nlp==0.2.0`. That did not solve this problem, however. 

Comments: This seems to be fixed in a non-released version. 

Installing nlp from source
```
git clone https://github.com/huggingface/nlp
cd nlp
pip install .
```
solves the issue.
[Suggestion] Glue Diagnostic Data with Labels 
Author: ggbetz
State: open
Labels: name: Dataset discussion

Hello! First of all, thanks for setting up this useful project!

I've just realised you provide the the [Glue Diagnostics Data](https://huggingface.co/nlp/viewer/?dataset=glue&config=ax) without labels, indicating in the `GlueConfig` that you've only a test set.

Yet, the data with labels is available, too (see also [here](https://gluebenchmark.com/diagnostics#introduction)):

https://www.dropbox.com/s/ju7d95ifb072q9f/diagnostic-full.tsv?dl=1 

Have you considered incorporating it?
Issues: Adding a FAISS or Elastic Search index to a Dataset
Author: nsankar
State: closed

It seems the DPRContextEncoder, DPRContextEncoderTokenizer cited[ in this documentation](https://huggingface.co/nlp/faiss_and_ea.html) is not implemented ? It didnot work with the standard nlp installation . Also, I couldn't find or use it with the latest nlp install from github in Colab.  Is  there any dependency on the latest PyArrow 1.0.0 ? Is it yet to be made generally available ?

Comments: `DPRContextEncoder` and `DPRContextEncoderTokenizer` will be available in the next release of `transformers`.

Right now you can experiment with it by installing `transformers` from the master branch.
You can also check the docs of DPR [here](https://huggingface.co/transformers/master/model_doc/dpr.html).

Moreover all the indexing features will also be available in the next release of `nlp`.

@lhoestq Thanks for the info 

@lhoestq  I tried installing transformer from the master branch. Python imports for DPR again didnt' work.  Anyways, Looking forward to trying it in the next release of nlp 

@nsankar have you tried with the latest version of the library?

@yjernite it worked. Thanks
New Datasets: IWSLT15+, ITTB
Author: sshleifer
State: open
Labels: name: dataset request

**Links:**
[iwslt](https://pytorchnlp.readthedocs.io/en/latest/_modules/torchnlp/datasets/iwslt.html)
Don't know if that link is up to date.

[ittb](http://www.cfilt.iitb.ac.in/iitb_parallel/)
**Motivation**: replicate mbart finetuning results (table below)
![image](https://user-images.githubusercontent.com/6045025/88490093-0c1c8c00-cf67-11ea-960d-8dcaad2aa8eb.png)


For future readers, we already have the following language pairs in the wmt namespaces:

```
wmt14: ['cs-en', 'de-en', 'fr-en', 'hi-en', 'ru-en']
wmt15: ['cs-en', 'de-en', 'fi-en', 'fr-en', 'ru-en']
wmt16: ['cs-en', 'de-en', 'fi-en', 'ro-en', 'ru-en', 'tr-en']
wmt17: ['cs-en', 'de-en', 'fi-en', 'lv-en', 'ru-en', 'tr-en', 'zh-en']
wmt18: ['cs-en', 'de-en', 'et-en', 'fi-en', 'kk-en', 'ru-en', 'tr-en', 'zh-en']
wmt19: ['cs-en', 'de-en', 'fi-en', 'gu-en', 'kk-en', 'lt-en', 'ru-en', 'zh-en', 'fr-de']
```

Comments: Thanks Sam, we now have a very detailed tutorial and template on how to add a new dataset to the library. It typically take 1-2 hours to add one. Do you want to give it a try ?
The tutorial on writing a new dataset loading script is here: https://huggingface.co/nlp/add_dataset.html
And the part on how to share a new dataset is here: https://huggingface.co/nlp/share_dataset.html

Hi @sshleifer, I'm trying to add IWSLT using the link you provided but the download urls are not working. Only `[en, de]` pair is working. For others language pairs it throws a `404` error.
Google Colab - load_dataset - PyArrow exception
Author: nsankar
State: closed

With latest PyArrow 1.0.0 installed, I get the following exception   . Restarting colab has the same issue

ImportWarning: To use `nlp`, the module `pyarrow>=0.16.0` is required, and the current version of `pyarrow` doesn't match this condition. If you are running this in a Google Colab, you should probably just restart the runtime to use the right version of `pyarrow`.

The error goes only when I install version 0.16.0 
i.e.  !pip install pyarrow==0.16.0

Comments: Indeed, we’ll make a new PyPi release next week to solve this. Cc @lhoestq 

+1! this is the reason our tests are failing at [TextAttack](https://github.com/QData/TextAttack) 

(Though it's worth noting if we fixed the version number of pyarrow to 0.16.0 that would fix our problem too. But in this case we'll just wait for you all to update)

Came to raise this issue, great to see other already have and it's being fixed so soon!

As an aside, since no one wrote this already, it seems like the version check only looks at the second part of the version number making sure it is >16, but pyarrow newest version is 1.0.0 so the second past is 0!

> Indeed, we’ll make a new PyPi release next week to solve this. Cc @lhoestq

Yes definitely

please fix this on pypi! @lhoestq 

Is this issue fixed ?

We’ll release the new version later today. Apologies for the delay.

I just pushed the new version on pypi :)

Thanks for the update.
ImportWarning for pyarrow 1.0.0
Author: HanGuo97
State: closed

The following PR raised ImportWarning at `pyarrow ==1.0.0` https://github.com/huggingface/nlp/pull/265/files

Comments: This was fixed in #434 
We'll do a release later this week to include this fix.
Thanks for reporting

I dont know if the fix was made but the problem is still present : 
Instaled with pip : NLP 0.3.0 // pyarrow 1.0.0 
OS : archlinux with kernel zen 5.8.5

Yes it was fixed in `nlp>=0.4.0`
You can update with pip

Sorry, I didn't got the updated version, all is now working perfectly thanks
[FEATURE REQUEST] Multiprocessing with for dataset.map, dataset.filter
Author: timothyjlaurent
State: closed
Labels: name: enhancement

It would be nice to be able to speed up `dataset.map` or `dataset.filter`. Perhaps this is as easy as sharding the dataset sending each shard to a process/thread/dask pool and using the new `nlp.concatenate_dataset()` function to join them all together?

Comments: Yes that's definitely something we plan to add ^^

Yes, that would be nice. We could take a look at what tensorflow `tf.data` does under the hood for instance.

So `tf.data.Dataset.map()` returns a `ParallelMapDataset` if `num_parallel_calls is not None` [link](https://github.com/tensorflow/tensorflow/blob/2b96f3662bd776e277f86997659e61046b56c315/tensorflow/python/data/ops/dataset_ops.py#L1623).

There, `num_parallel_calls` is turned into a tensor and and fed to `gen_dataset_ops.parallel_map_dataset` where it looks like tensorflow takes over.

We could start with something simple like a thread or process pool that `imap`s over some shards.
 

Multiprocessing was added in #552 . You can set the number of processes with `.map(..., num_proc=...)`. It also works for `filter`

Closing this one, but feel free to reo-open if you have other questions

@lhoestq Great feature implemented! Do you have plans to add it to official tutorials [Processing data in a Dataset](https://huggingface.co/docs/datasets/processing.html?highlight=save#augmenting-the-dataset)? It took me sometime to find this parallel processing api.

Thanks for the heads up !

I just added a paragraph about multiprocessing:
https://huggingface.co/docs/datasets/master/processing.html#multiprocessing
Something is wrong with WMT 19 kk-en dataset
Author: ChenghaoMou
State: open
Labels: name: dataset bug

The translation in the `train` set does not look right:

```
>>>import nlp
>>>from nlp import load_dataset
>>>dataset = load_dataset('wmt19', 'kk-en')
>>>dataset["train"]["translation"][0]
{'kk': 'Trumpian Uncertainty', 'en': 'Трамптық белгісіздік'}
>>>dataset["validation"]["translation"][0]
{'kk': 'Ақша-несие саясатының сценарийін қайта жазсақ', 'en': 'Rewriting the Monetary-Policy Script'}
```
from_dict delete?
Author: hackerxiaobai
State: closed

AttributeError: type object 'Dataset' has no attribute 'from_dict'

Comments: `from_dict` was added in #350 that was unfortunately not included in the 0.3.0 release. It's going to be included in the next release that will be out pretty soon though.
Right now if you want to use `from_dict` you have to install the package from the master branch
```
pip install git+https://github.com/huggingface/nlp.git
```

> `from_dict` was added in #350 that was unfortunately not included in the 0.3.0 release. It's going to be included in the next release that will be out pretty soon though.
> Right now if you want to use `from_dict` you have to install the package from the master branch
> 
> ```
> pip install git+https://github.com/huggingface/nlp.git
> ```
OK, thank you.
Is there a way to download only NQ dev?
Author: tholor
State: closed

Maybe I missed that in the docs, but is there a way to only download the dev set of natural questions (~1 GB)? 
As we want to benchmark QA models on different datasets, I would like to avoid downloading the 41GB of training data. 

I tried
```
dataset = nlp.load_dataset('natural_questions', split="validation", beam_runner="DirectRunner")
```
But this still triggered a big download of presumably the whole dataset. Is there any way of doing this or are splits / slicing options only available after downloading?

Thanks!

Comments: Unfortunately it's not possible to download only the dev set of NQ.

I think we could add a way to download only the test set by adding a custom configuration to the processing script though.

Ok, got it. I think this could be a valuable feature - especially for large datasets like NQ, but potentially also others. 
For us, it will in this case make the difference of using the library or keeping the old downloads of the raw dev datasets.  
However, I don't know if that fits into your plans with the library and can also understand if you don't want to support this.

I don't think we could force this behavior generally since the dataset script authors are free to organize the file download as they want (sometimes the mapping between split and files can be very much nontrivial) but we can add an additional configuration for Natural Question indeed as @lhoestq indicate.
train_test_split error: 'dict' object has no attribute 'deepcopy'
Author: morganmcg1
State: closed

`train_test_split` is giving me an error when I try and call it:

`'dict' object has no attribute 'deepcopy'`

## To reproduce

```
dataset = load_dataset('glue', 'mrpc', split='train')
dataset = dataset.train_test_split(test_size=0.2)
```

## Full Stacktrace
```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-12-feb740dbec9a> in <module>
      1 dataset = load_dataset('glue', 'mrpc', split='train')
----> 2 dataset = dataset.train_test_split(test_size=0.2)

~/anaconda3/envs/fastai2_me/lib/python3.7/site-packages/nlp/arrow_dataset.py in train_test_split(self, test_size, train_size, shuffle, seed, generator, keep_in_memory, load_from_cache_file, train_cache_file_name, test_cache_file_name, writer_batch_size)
   1032                     "writer_batch_size": writer_batch_size,
   1033                 }
-> 1034                 train_kwargs = cache_kwargs.deepcopy()
   1035                 train_kwargs["split"] = "train"
   1036                 test_kwargs = cache_kwargs.deepcopy()

AttributeError: 'dict' object has no attribute 'deepcopy'
```

Comments: It was fixed in 2ddd18d139d3047c9c3abe96e1e7d05bb360132c.
Could you pull the latest changes from master @morganmcg1 ?

Thanks @lhoestq, works fine now!
MissingBeamOptions for Wikipedia 20200501.en
Author: mitchellgordon95
State: closed

There may or may not be a regression for the pre-processed Wikipedia dataset. This was working fine 10 commits ago (without having Apache Beam available):

```
nlp.load_dataset('wikipedia', "20200501.en", split='train')
```

And now, having pulled master, I get:

```
Downloading and preparing dataset wikipedia/20200501.en (download: 16.99 GiB, generated: 17.07 GiB, total: 34.06 GiB) to /home/hltcoe/mgordon/.cache/huggingface/datasets/wikipedia/20200501.en/1.0.0/76b0b2747b679bb0ee7a1621e50e5a6378477add0c662668a324a5bc07d516dd...
Traceback (most recent call last):
  File "scripts/download.py", line 11, in <module>
    fire.Fire(download_pretrain)
  File "/home/hltcoe/mgordon/.conda/envs/huggingface/lib/python3.6/site-packages/fire/core.py", line 138, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/home/hltcoe/mgordon/.conda/envs/huggingface/lib/python3.6/site-packages/fire/core.py", line 468, in _Fire
    target=component.__name__)
  File "/home/hltcoe/mgordon/.conda/envs/huggingface/lib/python3.6/site-packages/fire/core.py", line 672, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "scripts/download.py", line 6, in download_pretrain
    nlp.load_dataset('wikipedia', "20200501.en", split='train')
  File "/exp/mgordon/nlp/src/nlp/load.py", line 534, in load_dataset
    save_infos=save_infos,
  File "/exp/mgordon/nlp/src/nlp/builder.py", line 460, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/exp/mgordon/nlp/src/nlp/builder.py", line 870, in _download_and_prepare
    "\n\t`{}`".format(usage_example)
nlp.builder.MissingBeamOptions: Trying to generate a dataset using Apache Beam, yet no Beam Runner or PipelineOptions() has been provided in `load_dataset` or in the builder arguments. For big datasets it has to run on large-scale data processing tools like Dataflow, S
park, etc. More information about Apache Beam runners at https://beam.apache.org/documentation/runners/capability-matrix/
If you really want to run it locally because you feel like the Dataset is small enough, you can use the local beam runner called `DirectRunner` (you may run out of memory).
Example of usage:
        `load_dataset('wikipedia', '20200501.en', beam_runner='DirectRunner')`
```

Comments: Fixed. Could you try again @mitchellgordon95 ?
It was due a file not being updated on S3.

We need to make sure all the datasets scripts get updated properly @julien-c 

Works for me! Thanks.

I found the same issue with almost any language other than English. (For English, it works). Will someone need to update the file on S3 again?

This is because only some languages are already preprocessed (en, de, fr, it) and stored on our google storage.
We plan to have a systematic way to preprocess more wikipedia languages in the future.

For the other languages you have to process them on your side using apache beam. That's why the lib asks for a Beam runner.
Memory issue when doing select
Author: lhoestq
State: closed

As noticed in #389, the following code loads the entire wikipedia in memory.

```python
import nlp
w = nlp.load_dataset("wikipedia", "20200501.en", split="train")
w.select([0])
```

This is caused by [this line](https://github.com/huggingface/nlp/blob/master/src/nlp/arrow_dataset.py#L626) for some reason, that tries to serialize the function with all the wikipedia data with it.

It's not the case with `.map` or `.filter`.
However functions that are based on `.select` like `.shuffle`, `.shard`, `.train_test_split`, `.sort` are affected.