rIsHu009/github-issues-updated huggingface dataset:
Title: trec dataset unavailable 
Author: rabeehk (CONTRIBUTOR)
State: closed
Created: 2020-11-22T08:09:36Z

Hi
when I try to load the trec dataset I am getting these errors, thanks for your help

`datasets.load_dataset("trec",  split="train")
`
```
  File "<stdin>", line 1, in <module>
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/load.py", line 611, in load_dataset
    ignore_verifications=ignore_verifications,
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/builder.py", line 476, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/builder.py", line 531, in _download_and_prepare
    split_generators = self._split_generators(dl_manager, **split_generators_kwargs)
  File " /idiap/home/rkarimi/.cache/huggingface/modules/datasets_modules/datasets/trec/ca4248481ad244f235f4cf277186cad2ee8769f975119a2bbfc41b8932b88bd7/trec.py", line 140, in _split_generators
    dl_files = dl_manager.download_and_extract(_URLs)
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/utils/download_manager.py", line 254, in download_and_extract
    return self.extract(self.download(url_or_urls))
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/utils/download_manager.py", line 179, in download
    num_proc=download_config.num_proc,
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/utils/py_utils.py", line 225, in map_nested
    _single_map_nested((function, obj, types, None, True)) for obj in tqdm(iterable, disable=disable_tqdm)
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/utils/py_utils.py", line 225, in <listcomp>
    _single_map_nested((function, obj, types, None, True)) for obj in tqdm(iterable, disable=disable_tqdm)
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/utils/py_utils.py", line 163, in _single_map_nested
    return function(data_struct)
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/utils/file_utils.py", line 308, in cached_path
    use_etag=download_config.use_etag,
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/utils/file_utils.py", line 477, in get_from_cache
    raise ConnectionError("Couldn't reach {}".format(url))
ConnectionError: Couldn't reach http://cogcomp.org/Data/QA/QC/train_5500.label
```

Comments:
This was fixed in #740 
Could you try to update `datasets` and try again ?

This has been fixed in datasets 1.1.3
Title: Add IndicGLUE dataset and Metrics
Author: sumanthd17 (CONTRIBUTOR)
State: closed
Created: 2020-11-20T17:09:34Z

Added IndicGLUE benchmark for evaluating models on 11 Indian Languages. The descriptions of the tasks and the corresponding paper can be found [here](https://indicnlp.ai4bharat.org/indic-glue/)

- [x] Followed the instructions in CONTRIBUTING.md
- [x] Ran the tests successfully 
- [x] Created the dummy data

Comments:
thanks ! merging now
Title: terminate called after throwing an instance of 'google::protobuf::FatalException'
Author: rabeehk (CONTRIBUTOR)
State: closed
Created: 2020-11-20T12:56:24Z

Hi
I am using the dataset "iwslt2017-en-nl", and after downloading it I am getting this error when trying to evaluate it on T5-base with seq2seq_trainer.py in the huggingface repo could you assist me please? thanks 


100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 63/63 [02:47<00:00,  2.18s/it][libprotobuf FATAL /sentencepiece/src/../third_party/protobuf-lite/google/protobuf/repeated_field.h:1505] CHECK failed: (index) >= (0): 
terminate called after throwing an instance of 'google::protobuf::FatalException'
  what():  CHECK failed: (index) >= (0): 
run_t5_base_eval.sh: line 19:  5795 Aborted

Comments:
Loading the iwslt2017-en-nl config of iwslt2017 works fine on my side. 
Maybe you can open an issue on transformers as well ? And also add more details about your environment (OS, python version, version of transformers and datasets etc.)

closing now, figured out this is because the max length of decoder was set smaller than the input_dimensions. thanks
Title: [Feature Request] Add optional parameter in text loading script to preserve linebreaks
Author: jncasey (NONE)
State: closed
Created: 2020-11-19T23:51:31Z
Labels: enhancement

I'm working on a project about rhyming verse using phonetic poetry and song lyrics, and line breaks are a vital part of the data. 

I recently switched over to use the datasets library when my various corpora grew larger than my computer's memory. And so far, it is SO great. 

But the first time I processed all of my data into a dataset, I hadn't realized the text loader script was processing the source files line-by-line and stripping off the newlines. 

Once I caught the issue, I made my own data loader by modifying one line in the default text loader (changing `batch = batch.splitlines()` to `batch = batch.splitlines(True)` inside `_generate_tables`). And so I'm all set as far as my project is concerned.

But if my use case is more general, it seems like it'd be pretty trivial to add a kwarg to the default text loader called keeplinebreaks or something, which would default to False and get passed to `splitlines()`.

Comments:
Hi ! Thanks for your message.
Indeed it's a free feature we can add and that can be useful.
If you want to contribute, feel free to open a PR to add it to the text dataset script :)

Resolved via #1913.
Title: Consistent metric outputs
Author: lhoestq (MEMBER)
State: closed
Created: 2020-11-18T18:05:59Z
Labels: transfer-to-evaluate

To automate the use of metrics, they should return consistent outputs.
In particular I'm working on adding a conversion of metrics to keras metrics.
To achieve this we need two things:
- have each metric return dictionaries of string -> floats since each keras metrics should return one float
- define in the metric info the different fields of the output dictionary

In this PR I'm adding these two features.
I also fixed a few bugs in some metrics

#867 needs to be merged first

Comments:
I keep this PR in stand-by for next week's datasets sprint. If the next release is 2.0.0 then we can include it given that it's breaking for many metrics

Metrics are deprecated in `datasets` and `evaluate` should be used instead: https://github.com/huggingface/evaluate
Title: OSCAR from Inria group
Author: jchwenger (NONE)
State: closed
Created: 2020-11-18T14:40:54Z
Labels: dataset request

## Adding a Dataset
- **Name:** *OSCAR* (Open Super-large Crawled ALMAnaCH coRpus), multilingual parsing of Common Crawl (separate crawls for many different languages), [here](https://oscar-corpus.com/).
- **Description:** *OSCAR or Open Super-large Crawled ALMAnaCH coRpus is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the goclassy architecture.*
- **Paper:** *[here](https://hal.inria.fr/hal-02148693)*
- **Data:** *[here](https://oscar-corpus.com/)*
- **Motivation:** *useful for unsupervised tasks in separate languages. In an ideal world, your team would be able to obtain the unshuffled version, that could be used to train GPT-2-like models (the shuffled version, I suppose, could be used for translation).*

I am aware that you do offer the "colossal" Common Crawl dataset already, but this has the advantage to be available in many subcorpora for different languages.

Comments:
PR is already open here : #348 
The only thing remaining is to compute the metadata of each subdataset (one per language + shuffled/unshuffled).
As soon as #863 is merged we can start computing them. This will take a bit of time though

Grand, thanks for this!
Title: Have Trouble importing `datasets`
Author: forest1988 (CONTRIBUTOR)
State: closed
Created: 2020-11-18T08:04:41Z

I'm failing to import transformers (v4.0.0-dev), and tracing the cause seems to be failing to import datasets.

I cloned the newest version of datasets (master branch), and do `pip install -e .`.

Then, `import datasets` causes the error below.

```
~/workspace/Clone/datasets/src/datasets/utils/file_utils.py in <module>
    116 sys.path.append(str(HF_MODULES_CACHE))
    117 
--> 118 os.makedirs(HF_MODULES_CACHE, exist_ok=True)
    119 if not os.path.exists(os.path.join(HF_MODULES_CACHE, "__init__.py")):
    120     with open(os.path.join(HF_MODULES_CACHE, "__init__.py"), "w"):

~/.pyenv/versions/anaconda3-2020.07/lib/python3.8/os.py in makedirs(name, mode, exist_ok)
    221             return
    222     try:
--> 223         mkdir(name, mode)
    224     except OSError:
    225         # Cannot rely on checking for EEXIST, since the operating system 

FileNotFoundError: [Errno 2] No such file or directory: '<MY_HOME_DIRECTORY>/.cache/huggingface/modules'
```

The error occurs in `os.makedirs` in `file_utils.py`, even though `exist_ok = True` option is set.
(I use Python 3.8, so `exist_ok` is expected to work.)

I've checked some environment variables, and they are set as below.

```
*** NameError: name 'HF_MODULES_CACHE' is not defined
*** NameError: name 'hf_cache_home' is not defined
*** NameError: name 'XDG_CACHE_HOME' is not defined
```

Should I set some environment variables before using this library?
And, do you have any idea why "No such file or directory" occurs even though the `exist_ok = True` option is set?

Thank you in advance.

Comments:
I'm sorry, this was a problem with my environment.
Now that I have identified the cause of environmental dependency, I would like to fix it and try it.
Excuse me for making a noise.
Title: Add clear_cache parameter in the test command
Author: lhoestq (MEMBER)
State: closed
Created: 2020-11-17T17:52:29Z

For certain datasets like OSCAR #348 there are lots of different configurations and each one of them can take a lot of disk space.

I added a `--clear_cache` flag to the `datasets-cli test` command to be able to clear the cache after each configuration test to avoid filling up the disk. It should enable an easier generation for the `dataset_infos.json` file for OSCAR.

Comments:
Title: Integrate file_lock inside the lib for better logging control
Author: lhoestq (MEMBER)
State: closed
Created: 2020-11-16T15:13:39Z

Previously the locking system of the lib was based on the file_lock package. However as noticed in #812 there were too many logs printed even when the datasets logging was set to warnings or errors.

For example
```python
import logging
logging.basicConfig(level=logging.INFO)

import datasets
datasets.set_verbosity_warning()
datasets.load_dataset("squad")
```
would still log the file lock events:
```
INFO:filelock:Lock 5737989232 acquired on /Users/quentinlhoest/.cache/huggingface/datasets/44801f118d500eff6114bfc56ab4e6def941f1eb14b70ac1ecc052e15cdac49d.85f43de978b9b25921cb78d7a2f2b350c04acdbaedb9ecb5f7101cd7c0950e68.py.lock
INFO:filelock:Lock 5737989232 released on /Users/quentinlhoest/.cache/huggingface/datasets/44801f118d500eff6114bfc56ab4e6def941f1eb14b70ac1ecc052e15cdac49d.85f43de978b9b25921cb78d7a2f2b350c04acdbaedb9ecb5f7101cd7c0950e68.py.lock
INFO:filelock:Lock 4393489968 acquired on /Users/quentinlhoest/.cache/huggingface/datasets/_Users_quentinlhoest_.cache_huggingface_datasets_squad_plain_text_1.0.0_1244d044b266a5e4dbd4174d23cb995eead372fbca31a03edc3f8a132787af41.lock
INFO:filelock:Lock 4393489968 released on /Users/quentinlhoest/.cache/huggingface/datasets/_Users_quentinlhoest_.cache_huggingface_datasets_squad_plain_text_1.0.0_1244d044b266a5e4dbd4174d23cb995eead372fbca31a03edc3f8a132787af41.lock
INFO:filelock:Lock 4393490808 acquired on /Users/quentinlhoest/.cache/huggingface/datasets/_Users_quentinlhoest_.cache_huggingface_datasets_squad_plain_text_1.0.0_1244d044b266a5e4dbd4174d23cb995eead372fbca31a03edc3f8a132787af41.lock
Reusing dataset squad (/Users/quentinlhoest/.cache/huggingface/datasets/squad/plain_text/1.0.0/1244d044b266a5e4dbd4174d23cb995eead372fbca31a03edc3f8a132787af41)
INFO:filelock:Lock 4393490808 released on /Users/quentinlhoest/.cache/huggingface/datasets/_Users_quentinlhoest_.cache_huggingface_datasets_squad_plain_text_1.0.0_1244d044b266a5e4dbd4174d23cb995eead372fbca31a03edc3f8a132787af41.lock
```

With the integration of file_lock in the library, the ouput is much cleaner:
```
Reusing dataset squad (/Users/quentinlhoest/.cache/huggingface/datasets/squad/plain_text/1.0.0/1244d044b266a5e4dbd4174d23cb995eead372fbca31a03edc3f8a132787af41)
```

Since the file_lock package is only a 450 lines file I think it's fine to have it inside the lib.

Fix #812

Comments:
Title: Add SemEval-2010 task 8
Author: JoelNiklaus (CONTRIBUTOR)
State: closed
Created: 2020-11-16T14:57:57Z

Hi,
I don't know how to add dummy data, since I create the validation set out of the last 1000 examples of the train set. If you have a suggestion, I am happy to implement it.
Cheers,
Joel

Comments:
Added dummy data and encoding to open(). Now everything should be fine, hopefully :)
Title: Use pandas reader in csv
Author: lhoestq (MEMBER)
State: closed
Created: 2020-11-16T14:05:45Z

The pyarrow CSV reader has issues that the pandas one doesn't (see #836 ).
To fix that I switched to the pandas csv reader.
The new reader is compatible with all the pandas parameters to read csv files.
Moreover it reads csv by chunk in order to save RAM, while the pyarrow one loads everything in memory.

Fix #836 
Fix #794 

Breaking: now all the parameters to read to csv file can be used in the `load_dataset` kwargs when loading csv, and the previous pyarrow objects `pyarrow.csv.ReadOptions`, `pyarrow.csv.ParseOptions` and `pyarrow.csv.ConvertOptions` are not used anymore.

Comments:
Title: Fix kor nli csv reader
Author: lhoestq (MEMBER)
State: closed
Created: 2020-11-16T09:53:41Z

The kor_nli dataset had an issue with the csv reader that was not able to parse the lines correctly. Some lines were merged together for some reason.
I fixed that by iterating through the lines directly instead of using a csv reader.
I also changed the feature names to match the other NLI datasets (i.e. use "premise", "hypothesis", "label" features)

Fix #821

Comments:
Title: Load amazon dataset
Author: bhavitvyamalik (CONTRIBUTOR)
State: closed
Created: 2020-11-13T08:34:24Z

Hi,
I was going through amazon_us_reviews dataset and found that example API usage given on website is different from the API usage while loading dataset. 

Eg. what API usage is on the [website](https://huggingface.co/datasets/amazon_us_reviews) 
```
from datasets import load_dataset
dataset = load_dataset("amazon_us_reviews")
```
How it is when I tried (the error generated does point me to the right direction though)
```
from datasets import load_dataset
dataset = load_dataset("amazon_us_reviews", 'Books_v1_00')
``` 
Also, there is some issue with formatting as it's not showing bullet list in description with new line. Can I work on it?

Comments:
Thanks for reporting !
We plan to show information about the different configs of the datasets on the website, with the corresponding `load_dataset` calls.

Also I think the bullet points formatting has been fixed
Title: Add HoVer multi-hop fact verification dataset
Author: yjernite (MEMBER)
State: closed
Created: 2020-11-12T19:55:46Z
Labels: dataset request

## Adding a Dataset
- **Name:** HoVer
- **Description:** https://twitter.com/YichenJiang9/status/1326954363806429186 contains 20K claim verification examples
- **Paper:** https://arxiv.org/abs/2011.03088
- **Data:** https://hover-nlp.github.io/
- **Motivation:** There are still few multi-hop information extraction benchmarks (HotpotQA, which dataset wase based off, notwithstanding)

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).

Comments:
Hi @yjernite  I'm new but wanted to contribute. Has anyone already taken this problem and do you think it is suitable for newbies?

Hi @tenjjin! This dataset is still up for grabs! Here's the link with the guide to add it. You should play around with the library first (download and look at a few datasets), then follow the steps here:

https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md

Closed by #1399
Title: use_custom_baseline still produces errors for bertscore
Author: penatbater (NONE)
State: closed
Created: 2020-11-12T11:44:32Z
Labels: metric bug

`metric = load_metric('bertscore')`
`a1 = "random sentences"`
`b1 = "random sentences"`
`metric.compute(predictions = [a1], references = [b1], lang = 'en')`

`Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/stephen_chan/.local/lib/python3.6/site-packages/datasets/metric.py", line 393, in compute
    output = self._compute(predictions=predictions, references=references, **kwargs)
  File "/home/stephen_chan/.cache/huggingface/modules/datasets_modules/metrics/bertscore/361e597a01a41d6cf95d94bbfb01dea16261687abc0c6c74cc9930f80488f363/bertscore.py", line 108, in _compute
    hashcode = bert_score.utils.get_hash(model_type, num_layers, idf, rescale_with_baseline)
TypeError: get_hash() missing 1 required positional argument: 'use_custom_baseline'`

Adding 'use_custom_baseline = False' as an argument produces this error

`Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/stephen_chan/.local/lib/python3.6/site-packages/datasets/metric.py", line 393, in compute
    output = self._compute(predictions=predictions, references=references, **kwargs)
TypeError: _compute() got an unexpected keyword argument 'use_custom_baseline'`

This is on Ubuntu 18.04, Python 3.6.9, datasets version 1.1.2

Comments:
Thanks for reporting ! That's a bug indeed
If you want to contribute, feel free to fix this issue and open a PR :)

This error is because of a mismatch between `datasets` and `bert_score`. With `datasets=1.1.2` and `bert_score>=0.3.6` it works ok. So `pip install -U bert_score` should fix the problem. 

Thanks for the heads up @pvl and for the PR as well :)

Hello everyone,

I think the  problem is not solved: 

```
from datasets import load_metric
metric=load_metric('bertscore')
metric.compute(
    predictions=predictions,
    references=references,
    lang='fr',
    rescale_with_baseline=True
)
TypeError: get_hash() missing 2 required positional arguments: 'use_custom_baseline' and 'use_fast_tokenizer'
```
This code is produced using `Python 3.6.9 datasets==1.1.2 and bert_score==0.3.10`

Hi ! This has been fixed by https://github.com/huggingface/datasets/pull/2770, we'll do a new release soon to make the fix available :)

In the meantime please use an older version of `bert_score`
Title: How to enable `.map()` pre-processing pipelines to support multi-node parallelism?
Author: shangw-nvidia (NONE)
State: open
Created: 2020-11-12T02:04:38Z

Hi,

Currently, multiprocessing can be enabled for the `.map()` stages on a single node. However, in the case of multi-node training, (since more than one node would be available) I'm wondering if it's possible to extend the parallel processing among nodes, instead of only 1 node running the `.map()` while the other node is waiting for it to finish?

Thanks!

Comments:
Right now multiprocessing only runs on single node.

However it's probably possible to extend it to support multi nodes. Indeed we're using the `multiprocess` library from the `pathos` project to do multiprocessing in `datasets`, and `pathos` is made to support parallelism on several nodes. More info about pathos [on the pathos repo](https://github.com/uqfoundation/pathos).

If you're familiar with pathos or if you want to give it a try, it could be a nice addition to the library :)

Curious to hear if anything on that side changed or if you suggestions to do it changed @lhoestq :)

For our use-case, we are entering the regime where trading a few more instances to save a few days would be nice :)

Currently for multi-node setups we're mostly going towards a nice integration with Dask. But I wouldn't exclude exploring `pathos` more at one point

What is the current status of this feature? Is multi-node parallelism supported for `.map` function?

Hi ! `datasets` main focus in single node data processing, but you can manually `.shard()` a dataset to run `map()` on each dataset shard in a distributed setup !

Otherwise you can also explore `datatrove` which supports SLURM for distributed data processing, or tools like spark / dask
Title: XSum dataset missing spaces between sentences
Author: loganlebanoff (NONE)
State: open
Created: 2020-11-11T00:34:43Z

I noticed that the XSum dataset has no space between sentences. This could lead to worse results for anyone training or testing on it. Here's an example (0th entry in the test set):

`The London trio are up for best UK act and best album, as well as getting two nominations in the best song category."We got told like this morning 'Oh I think you're nominated'", said Dappy."And I was like 'Oh yeah, which one?' And now we've got nominated for four awards. I mean, wow!"Bandmate Fazer added: "We thought it's best of us to come down and mingle with everyone and say hello to the cameras. And now we find we've got four nominations."The band have two shots at the best song prize, getting the nod for their Tynchy Stryder collaboration Number One, and single Strong Again.Their album Uncle B will also go up against records by the likes of Beyonce and Kanye West.N-Dubz picked up the best newcomer Mobo in 2007, but female member Tulisa said they wouldn't be too disappointed if they didn't win this time around."At the end of the day we're grateful to be where we are in our careers."If it don't happen then it don't happen - live to fight another day and keep on making albums and hits for the fans."Dappy also revealed they could be performing live several times on the night.The group will be doing Number One and also a possible rendition of the War Child single, I Got Soul.The charity song is a  re-working of The Killers' All These Things That I've Done and is set to feature artists like Chipmunk, Ironik and Pixie Lott.This year's Mobos will be held outside of London for the first time, in Glasgow on 30 September.N-Dubz said they were looking forward to performing for their Scottish fans and boasted about their recent shows north of the border."We just done Edinburgh the other day," said Dappy."We smashed up an N-Dubz show over there. We done Aberdeen about three or four months ago - we smashed up that show over there! Everywhere we go we smash it up!"`

Comments:
Title: CNN/Dailymail Dataset Card
Author: mcmillanmajora (CONTRIBUTOR)
State: closed
Created: 2020-11-10T23:56:43Z

Link to the card page: https://github.com/mcmillanmajora/datasets/tree/cnn_dailymail_card/datasets/cnn_dailymail

One of the questions this dataset brings up is how we want to handle versioning of the cards to mirror versions of the dataset. The different versions of this dataset are used for different tasks (which may not be reflected in the versions that we currently have in the repo?), but it's only the structure that's changing rather than the content in this particular case, at least between versions 2.0.0 and 3.0.0.

Comments:
Title: AlloCiné dataset card
Author: mcmillanmajora (CONTRIBUTOR)
State: closed
Created: 2020-11-10T21:19:53Z

Link to the card page: https://github.com/mcmillanmajora/datasets/blob/allocine_card/datasets/allocine/README.md

There wasn't as much information available for this dataset, so I'm wondering what's the best way to address open questions about the dataset. For example, where did the list of films that the dataset creator used come from?

I'm also wondering how best to go about talking about limitations when so little is known about the data.

Comments:
Title: Wikipedia postprocessing
Author: bminixhofer (NONE)
State: closed
Created: 2020-11-10T17:26:38Z

Hi, thanks for this library!

Running this code:

```py
import datasets
wikipedia = datasets.load_dataset("wikipedia", "20200501.de")
print(wikipedia['train']['text'][0])
```

I get:

```
mini|Ricardo Flores Magón
mini|Mexikanische Revolutionäre, Magón in der Mitte anführend, gegen die Diktatur von Porfirio Diaz, Ausschnitt des Gemälde „Tierra y Libertad“ von Idelfonso Carrara (?) von 1930.

Ricardo Flores Magón (* 16. September 1874 in San Antonio Eloxochitlán im mexikanischen Bundesstaat Oaxaca; † 22. November 1922 im Bundesgefängnis Leavenworth im US-amerikanischen Bundesstaat Kansas) war als Journalist, Gewerkschafter und Literat ein führender anarchistischer Theoretiker und Aktivist, der die revolutionäre mexikanische Bewegung radikal beeinflusste. Magón war Gründer der Partido Liberal Mexicano und Mitglied der Industrial Workers of the World.

Politische Biografie 
Journalistisch und politisch kämpfte er und sein Bruder sehr kompromisslos gegen die Diktatur Porfirio Diaz. Philosophisch und politisch orientiert an radikal anarchistischen Idealen und den Erfahrungen seiner indigenen Vorfahren bei der gemeinschaftlichen Bewirtschaftung des Gemeindelandes, machte er die Forderung „Land und Freiheit“ (Tierra y Libertad) populär. Besonders Francisco Villa und Emiliano Zapata griffen die Forderung Land und Freiheit auf. Seine Philosophie hatte großen Einfluss auf die Landarbeiter. 1904 floh er in die USA und gründete 1906 die Partido Liberal Mexicano. Im Exil lernte er u. a. Emma Goldman kennen. Er verbrachte die meiste Zeit seines Lebens in Gefängnissen und im Exil und wurde 1918 in den USA wegen „Behinderung der Kriegsanstrengungen“ zu zwanzig Jahren Gefängnis verurteilt. Zu seinem Tod gibt es drei verschiedene Theorien. Offiziell starb er an Herzversagen. Librado Rivera, der die Leiche mit eigenen Augen gesehen hat, geht davon aus, dass Magón von einem Mitgefangenen erdrosselt wurde. Die staatstreue Gewerkschaftszeitung CROM veröffentlichte 1923 einen Beitrag, nachdem Magón von einem Gefängniswärter erschlagen wurde.
mini|Die Brüder Ricardo (links) und Enrique Flores Magón (rechts) vor dem Los Angeles County Jail, 1917

[...]
```

so some Markup like `mini|` is still left. Should I run another parser on this text before feeding it to an ML model or is this a known imperfection of parsing Wiki markup?

Apologies if this has been asked before.

Comments:
Hi @bminixhofer ! Parsing WikiMedia is notoriously difficult: this processing used [mwparserfromhell](https://github.com/earwig/mwparserfromhell) which is pretty good but not perfect.

As an alternative, you can also use the Wiki40b dataset which was pre-processed using an un-released Google internal tool

Ok, thanks! I'll try the Wiki40b dataset.

If anyone else is concerned about this, `wiki40b` does indeed seem very well cleaned.
Title: [GEM] add WikiLingua cross-lingual abstractive summarization dataset
Author: yjernite (MEMBER)
State: closed
Created: 2020-11-10T17:00:43Z
Labels: dataset request

## Adding a Dataset
- **Name:** WikiLingua
- **Description:** The dataset includes ~770k article and summary pairs in 18 languages from WikiHow. The gold-standard article-summary alignments across languages were extracted by aligning the images that are used to describe each how-to step in an article.
- **Paper:** https://arxiv.org/pdf/2010.03093.pdf
- **Data:** https://github.com/esdurmus/Wikilingua
- **Motivation:** Included in the GEM shared task. Multilingual.

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).

Comments:
Hey @yjernite. This is a very interesting dataset. Would love to work on adding it but I see that the link to the data is to a gdrive folder. Can I just confirm wether dlmanager can handle gdrive urls or would this have to be a manual dl?

Hi @KMFODA ! A version of WikiLingua is actually already accessible in the [GEM dataset](https://huggingface.co/datasets/gem)

You can use it for example to load the French to English translation with:
```python
from datasets import load_dataset
wikilingua = load_dataset("gem", "wiki_lingua_french_fr")
```

Closed by https://github.com/huggingface/datasets/pull/1807
Title: [GEM] add ASSET text simplification dataset
Author: yjernite (MEMBER)
State: closed
Created: 2020-11-10T16:56:30Z
Labels: dataset request

## Adding a Dataset
- **Name:** ASSET
- **Description:** ASSET is a crowdsourced
multi-reference corpus for assessing sentence simplification in English where each simplification was produced by executing several rewriting transformations.
- **Paper:** https://www.aclweb.org/anthology/2020.acl-main.424.pdf
- **Data:** https://github.com/facebookresearch/asset
- **Motivation:** Included in the GEM shared task

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).

Comments:
Title: [GEM] add WikiAuto text simplification dataset
Author: yjernite (MEMBER)
State: closed
Created: 2020-11-10T16:53:23Z
Labels: dataset request

## Adding a Dataset
- **Name:** WikiAuto
- **Description:** Sentences in English Wikipedia and their corresponding sentences in Simple English Wikipedia that are written with simpler grammar and word choices. A lot of lexical and syntactic paraphrasing. 
- **Paper:** https://www.aclweb.org/anthology/2020.acl-main.709.pdf
- **Data:** https://github.com/chaojiang06/wiki-auto
- **Motivation:** Included in the GEM shared task

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).

Comments:
Title: [GEM] Add WebNLG dataset
Author: yjernite (MEMBER)
State: closed
Created: 2020-11-10T16:46:48Z
Labels: dataset request

## Adding a Dataset
- **Name:** WebNLG
- **Description:** WebNLG consists of Data/Text pairs where the data is a set of triples extracted from DBpedia and the text is a verbalisation of these triples (16,095 data inputs and 42,873 data-text pairs). The data is available in English and Russian
- **Paper:** https://www.aclweb.org/anthology/P17-1017.pdf
- **Data:** https://webnlg-challenge.loria.fr/download/
- **Motivation:** Included in the GEM shared task, multilingual

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).

Comments:
Title: [GEM] add ToTTo Table-to-text dataset
Author: yjernite (MEMBER)
State: closed
Created: 2020-11-10T16:38:34Z
Labels: dataset request

## Adding a Dataset
- **Name:** ToTTo
- **Description:** ToTTo is an open-domain English table-to-text dataset with over 120,000 training examples that proposes a controlled generation task: given a Wikipedia table and a set of highlighted table cells, produce a one-sentence description.
- **Paper:** https://arxiv.org/abs/2004.14373
- **Data:** https://github.com/google-research-datasets/totto
- **Motivation:** Included in the GEM shared task

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).

Comments:
closed via #1098
Title: [GEM] add Schema-Guided Dialogue
Author: yjernite (MEMBER)
State: closed
Created: 2020-11-10T16:33:44Z
Labels: dataset request

## Adding a Dataset
- **Name:** The Schema-Guided Dialogue Dataset
- **Description:** The Schema-Guided Dialogue (SGD) dataset consists of over 20k annotated multi-domain, task-oriented conversations between a human and a virtual assistant. These conversations involve interactions with services and APIs spanning 20 domains, ranging from banks and events to media, calendar, travel, and weather.
- **Paper:** https://arxiv.org/pdf/2002.01359.pdf https://arxiv.org/pdf/2004.15006.pdf
- **Data:** https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
- **Motivation:** Included in the GEM shared task

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).

Comments:
Title: Add writer_batch_size attribute to GeneratorBasedBuilder
Author: lhoestq (MEMBER)
State: closed
Created: 2020-11-10T15:28:19Z

As specified in #741 one would need to specify a custom ArrowWriter batch size to avoid filling the RAM. Indeed the defaults buffer size is 10 000 examples but for multimodal datasets that contain images or videos we may want to reduce that.

Comments:
Title: [GEM] MultiWOZ dialogue dataset
Author: yjernite (MEMBER)
State: closed
Created: 2020-11-10T14:57:50Z
Labels: dataset request

## Adding a Dataset
- **Name:** MultiWOZ (Multi-Domain Wizard-of-Oz)
- **Description:** 10k annotated human-human dialogues. Each dialogue consists of a goal, multiple user and system utterances as well as a belief state. Only system utterances are annotated with dialogue acts – there are no annotations from the user side.
- **Paper:** https://arxiv.org/pdf/2007.12720.pdf
- **Data:** https://github.com/budzianowski/multiwoz
- **Motivation:** Will likely be part of the GEM shared task

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).

Comments:
Hi @yjernite can I help in adding this dataset? 

I am excited about this because this will be my first contribution to the datasets library as well as to hugginface.

Resolved via https://github.com/huggingface/datasets/pull/979
Title: [GEM] Add E2E dataset
Author: yjernite (MEMBER)
State: closed
Created: 2020-11-10T14:50:40Z
Labels: dataset request

## Adding a Dataset
- **Name:** E2E NLG dataset (for End-to-end natural language generation)
- **Description:**a dataset for training end-to-end, datadriven natural language generation systems in the restaurant domain, the datasets consists of 5,751 dialogue-act Meaning Representations (structured data) and 8.1 reference free-text utterances per dialogue-act on average
- **Paper:** https://arxiv.org/pdf/1706.09254.pdf https://arxiv.org/abs/1901.07931
- **Data:** http://www.macs.hw.ac.uk/InteractionLab/E2E/#data
- **Motivation:** This dataset will likely be included in the GEM shared task

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).

Comments:
Title: Add accuracy, precision, recall and F1 metrics
Author: jplu (CONTRIBUTOR)
State: closed
Created: 2020-11-10T13:50:35Z

This PR adds several single metrics, namely:

- Accuracy
- Precision
- Recall
- F1

They all uses under the hood the sklearn metrics of the same name. They allow different useful features when training a multilabel/multiclass model:
- have a macro/micro/per label/weighted/binary/per sample score
- score only the selected labels (usually what we call the positive labels) and ignore the negative ones. For example in case of a Named Entity Recognition task, positive labels are (`PERSON`, `LOCATION` or `ORGANIZATION`) and the negative one is `O`.

Comments:
Title: datasets freezes 
Author: rabeehkarimimahabadi (NONE)
State: closed
Created: 2020-11-10T05:10:19Z
Labels: dataset bug

Hi, I want to load these two datasets and convert them to Dataset format in torch and the code freezes for me, could you have a look please? thanks 

dataset1 = load_dataset("squad", split="train[:10]")
dataset1 = dataset1.set_format(type='torch', columns=['context', 'answers', 'question'])

dataset2 = load_dataset("imdb", split="train[:10]")
dataset2 = dataset2.set_format(type="torch", columns=["text", "label"])
print(len(dataset1))

Comments:
Pytorch is unable to convert strings to tensors unfortunately.
You can use `set_format(type="torch")` on columns that can be converted to tensors, such as token ids.

This makes me think that we should probably raise an error or at least a warning when one tries to create pytorch tensors out of text columns

Ultimately, we decided to return a list instead of an error when formatting a string column with the format type `"torch"`.

If you think an error would be more appropriate, please open a new issue.
Title: Make save function use deterministic global vars order
Author: lhoestq (MEMBER)
State: closed
Created: 2020-11-09T18:12:03Z

The `dumps` function need to be deterministic for the caching mechanism.
However in #816 I noticed that one of dill's method to recursively check the globals of a function may return the globals in different orders each time it's used. To fix that I sort the globals by key in the `globs` dictionary.
I had to add a rectified `save_function` to the saving functions registry of the Pickler to make it work.

This should fix #816

Comments:
Sorry, asking for help here, but the dill thread stop around 2013. Is it possible to use dill deterministically? I tried to monkeypatch the solution presented here into dill, but I suppose it requires forking their project.

Hi ! What we did was to subclass `dill`'s Pickler to fix the non-deterministic behaviors, and it's been working fine. A fork should also do the job
Title: Fix type hints pickling in python 3.6
Author: lhoestq (MEMBER)
State: closed
Created: 2020-11-09T16:27:47Z

Type hints can't be properly pickled in python 3.6. This was causing errors the `run_mlm.py` script from `transformers` with python 3.6

However Cloupickle proposed a [fix](https://github.com/cloudpipe/cloudpickle/pull/318/files) to make it work anyway.
The idea is just to implement the pickling/unpickling of parameterized type hints. There is one detail though: since in python 3.6 we can't use `isinstance` on type hints, then we can't use pickle saving functions registry directly. Therefore we just wrap the `save_global` method of the Pickler.

This should fix https://github.com/huggingface/transformers/issues/8212 for python 3.6 and make `run_mlm.py` support python 3.6

cc @sgugger

Comments:
Title: Add MRQA dataset
Author: VictorSanh (CONTRIBUTOR)
State: closed
Created: 2020-11-09T15:52:19Z
Labels: dataset request

## Adding a Dataset
- **Name:** MRQA
- **Description:** Collection of different (subsets of) QA datasets all converted to the same format to evaluate out-of-domain generalization (the datasets come from different domains, distributions, etc.). Some datasets are used for training and others are used for evaluation. This dataset was collected as part of MRQA 2019's shared task 
- **Paper:** https://arxiv.org/abs/1910.09753
- **Data:** https://github.com/mrqa/MRQA-Shared-Task-2019
- **Motivation:** Out-of-domain generalization is becoming (has become) a de-factor evaluation for NLU systems

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).

Comments:
Done! cf #1117 and #1022
Title: [Caching] Dill globalvars() output order is not deterministic and can cause cache issues.
Author: lhoestq (MEMBER)
State: closed
Created: 2020-11-09T15:01:20Z

Dill uses `dill.detect.globalvars` to get the globals used by a function in a recursive dump. `globalvars` returns a dictionary of all the globals that a dumped function needs. However the order of the keys in this dict is not deterministic and can cause caching issues.

To fix that one could register an implementation of dill's `save_function` in the `datasets` pickler that sorts the globals keys before dumping a function.

Comments:
To show the issue:
```
python -c "from datasets.fingerprint import Hasher; a=[]; func = lambda : len(a); print(Hasher.hash(func))"
```
doesn't always return the same ouput since `globs` is a dictionary with "a" and "len" as keys but sometimes not in the same order
Title: Joining multiple datasets 
Author: rabeehkarimimahabadi (NONE)
State: closed
Created: 2020-11-08T16:19:30Z
Labels: dataset request

Hi
I have multiple iterative datasets from your library with different size and I want to join them in a way that each datasets is sampled equally, so smaller datasets more, larger one less, could you tell me how to implement this in pytorch? thanks

Comments:
found a solution here  https://discuss.pytorch.org/t/train-simultaneously-on-two-datasets/649/35, closed for now, thanks
Title: How to implement DistributedSampler with datasets 
Author: rabeehkarimimahabadi (NONE)
State: closed
Created: 2020-11-08T15:27:11Z
Labels: dataset request

Hi,
I am using your datasets to define my dataloaders, and I am training finetune_trainer.py in huggingface repo on them.
I need a distributedSampler to be able to train the models on TPUs being able to distribute the load across the TPU cores. Could you tell me how I can implement the distribued sampler when using datasets in which datasets are iterative? To give you more context, I have multiple of datasets and I need to write sampler for this case. thanks.

Comments:
Hi Apparently I need to shard the data and give one host a chunk, could you provide me please with examples on how to do it? I want to use it jointly with finetune_trainer.py in huggingface repo seq2seq examples. thanks. 

Hey @rabeehkarimimahabadi I'm actually looking for the same feature. Did you manage to get somewhere?

@rabeehkarimimahabadi need the same feature

Hi! I think you can use the `accelerate` library for that, which implements distributed sampling.
Title: nlp viewer error
Author: jc-hou (NONE)
State: closed
Created: 2020-11-07T17:08:58Z
Labels: nlp-viewer

Hello, 
when I select amazon_us_reviews in nlp viewer, it shows error.
https://huggingface.co/nlp/viewer/?dataset=amazon_us_reviews
![image](https://user-images.githubusercontent.com/30210529/98447334-4aa81200-2124-11eb-9dca-82c3ab34ccc2.png)

Comments:
and also for 'blog_authorship_corpus'
https://huggingface.co/nlp/viewer/?dataset=blog_authorship_corpus
![image](https://user-images.githubusercontent.com/30210529/98557329-5c182800-22a4-11eb-9b01-5b910fb8fcd4.png)


Is this the problem of my local computer or ??

Related to:
- #673
Title: Fix seqeval metric
Author: sgugger (CONTRIBUTOR)
State: closed
Created: 2020-11-06T16:11:43Z

The current seqeval metric returns the following error when computed:
```
~/.cache/huggingface/modules/datasets_modules/metrics/seqeval/78a944d83252b5a16c9a2e49f057f4c6e02f18cc03349257025a8c9aea6524d8/seqeval.py in _compute(self, predictions, references, suffix)
    102         scores = {}
    103         for type_name, score in report.items():
--> 104             scores[type_name]["precision"] = score["precision"]
    105             scores[type_name]["recall"] = score["recall"]
    106             scores[type_name]["f1"] = score["f1-score"]

KeyError: 'LOC'
```
This is because the current code basically tries to do:
```
scores = {}
scores["LOC"]["precision"] = some_value
```
which does not work in python. This PR fixes that while keeping the previous nested structure of results, with the same keys.

Comments:
Title: Add Google Taskmaster dataset
Author: yjernite (MEMBER)
State: closed
Created: 2020-11-06T15:10:41Z
Labels: dataset request

## Adding a Dataset
- **Name:** Taskmaster
- **Description:** A large dataset of task-oriented dialogue with annotated goals (55K dialogues covering entertainment and travel reservations)
- **Paper:** https://arxiv.org/abs/1909.05358
- **Data:** https://github.com/google-research-datasets/Taskmaster
- **Motivation:** One of few annotated datasets of this size for goal-oriented dialogue

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).

Comments:
Hey @yjernite. Was going to start working on this but found taskmaster 1,2 & 3 in the datasets library already so think this can be closed now?

You are absolutely right :) 

Closed by https://github.com/huggingface/datasets/pull/1193 https://github.com/huggingface/datasets/pull/1197 https://github.com/huggingface/datasets/pull/1213
Title: dataset(dgs): initial dataset loading script
Author: AmitMY (CONTRIBUTOR)
State: closed
Created: 2020-11-06T10:14:43Z

When trying to create dummy data I get:

> Dataset datasets with config None seems to already open files in the method `_split_generators(...)`. You might consider to instead only open files in the method `_generate_examples(...)` instead. If this is not possible the dummy data has t o be created with less guidance. Make sure you create the file dummy_data.

I am not sure how to manually create the dummy_data (what exactly it should contain)

Also note, this library says:
> ImportError: To be able to use this dataset, you need to install the following dependencies['pympi'] using 'pip install pympi' for instance'

When you actually need to `pip install pympi-ling`

Comments:
Hi @AmitMY, 

Were you able to figure this out?

I did not.
With all the limitations this repo currently has, I had to create a repo of my own using tfds to mitigate them. 
https://github.com/sign-language-processing/datasets/tree/master/sign_language_datasets/datasets/dgs_corpus

Closing as I don't know how to support this PR further
Title: Quail dataset urls are out of date
Author: ngdodd (CONTRIBUTOR)
State: closed
Created: 2020-11-05T19:40:19Z

<h3>Code</h3>

```
from datasets import load_dataset
quail = load_dataset('quail')
```

<h3>Error</h3>

```
FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/text-machine-lab/quail/master/quail_v1.2/xml/ordered/quail_1.2_train.xml
```


As per [quail v1.3 commit](https://github.com/text-machine-lab/quail/commit/506501cfa34d9ec6c042d31026ba6fea6bcec8ff) it looks like the location and suggested ordering has changed. In [https://github.com/huggingface/datasets/blob/master/datasets/quail/quail.py#L52-L58](https://github.com/huggingface/datasets/blob/master/datasets/quail/quail.py#L52-L58) the quail v1.2 datasets are being pointed to, which don't exist anymore.

Comments:
Hi ! Thanks for reporting.
We should fix the urls and use quail 1.3.
If you want to contribute feel free to fix the urls and open a PR :) 

Done! PR [https://github.com/huggingface/datasets/pull/820](https://github.com/huggingface/datasets/pull/820)

Updated links and also regenerated the metadata and dummy data for v1.3 in order to pass verifications as described here: [https://huggingface.co/docs/datasets/share_dataset.html#adding-tests-and-metadata-to-the-dataset](https://huggingface.co/docs/datasets/share_dataset.html#adding-tests-and-metadata-to-the-dataset). 

Closing since #820 is merged.
Thanks again for fixing the urls :)
Title: On loading a metric from datasets, I get the following error
Author: laibamehnaz (NONE)
State: closed
Created: 2020-11-05T15:14:38Z

`from datasets import load_metric`

`metric = load_metric('bleurt')`

Traceback:
210 class _ArrayXDExtensionType(pa.PyExtensionType):
    211 
    212     ndims: int = None

AttributeError: module 'pyarrow' has no attribute 'PyExtensionType'

Any help will be appreciated. Thank you.

Comments:
Hi ! We support only pyarrow > 0.17.1 so that we have access to the `PyExtensionType` object.
Could you update pyarrow and try again ?
```
pip install --upgrade pyarrow
```