kardwalker/my_dataset huggingface dataset:
GitHub issue #816: [Caching] Dill globalvars() output order is not deterministic and can cause cache issues.
State: closed | Created: 2020-11-09T15:01:20Z

Dill uses `dill.detect.globalvars` to get the globals used by a function in a recursive dump. `globalvars` returns a dictionary of all the globals that a dumped function needs. However the order of the keys in this dict is not deterministic and can cause caching issues.

To fix that one could register an implementation of dill's `save_function` in the `datasets` pickler that sorts the globals keys before dumping a function.
GitHub issue #813: How to implement DistributedSampler with datasets 
State: closed | Created: 2020-11-08T15:27:11Z

Hi,
I am using your datasets to define my dataloaders, and I am training finetune_trainer.py in huggingface repo on them.
I need a distributedSampler to be able to train the models on TPUs being able to distribute the load across the TPU cores. Could you tell me how I can implement the distribued sampler when using datasets in which datasets are iterative? To give you more context, I have multiple of datasets and I need to write sampler for this case. thanks.
GitHub issue #812: Too much logging 
State: closed | Created: 2020-11-07T23:56:30Z

I'm doing this in the beginning of my script:

from datasets.utils import logging as datasets_logging
datasets_logging.set_verbosity_warning()

but I'm still getting these logs:

[2020-11-07 15:45:41,908][filelock][INFO] - Lock 139958278886176 acquired on /home/username/.cache/huggingface/datasets/cfe20ffaa80ef1c145a0a210d5b9cdce2b60002831e6ed0edc7ab9275d6f0d48.1bd4ccbce9de3dad0698d84674a19d6cc66a84db736a6398110bd196795dde7e.py.lock

[2020-11-07 15:45:41,909][filelock][INFO] - Lock 139958278886176 released on /home/username/.cache/huggingface/datasets/cfe20ffaa80ef1c145a0a210d5b9cdce2b60002831e6ed0edc7ab9275d6f0d48.1bd4ccbce9de3dad0698d84674a19d6cc66a84db736a6398110bd196795dde7e.py.lock

using datasets version = 1.1.2
GitHub issue #811: nlp viewer error
State: closed | Created: 2020-11-07T17:08:58Z

Hello, 
when I select amazon_us_reviews in nlp viewer, it shows error.
https://huggingface.co/nlp/viewer/?dataset=amazon_us_reviews
![image](https://user-images.githubusercontent.com/30210529/98447334-4aa81200-2124-11eb-9dca-82c3ab34ccc2.png)
GitHub issue #810: Fix seqeval metric
State: closed | Created: 2020-11-06T16:11:43Z

The current seqeval metric returns the following error when computed:
```
~/.cache/huggingface/modules/datasets_modules/metrics/seqeval/78a944d83252b5a16c9a2e49f057f4c6e02f18cc03349257025a8c9aea6524d8/seqeval.py in _compute(self, predictions, references, suffix)
    102         scores = {}
    103         for type_name, score in report.items():
--> 104             scores[type_name]["precision"] = score["precision"]
    105             scores[type_name]["recall"] = score["recall"]
    106             scores[type_name]["f1"] = score["f1-score"]

KeyError: 'LOC'
```
This is because the current code basically tries to do:
```
scores = {}
scores["LOC"]["precision"] = some_value
```
which does not work in python. This PR fixes that while keeping the previous nested structure of results, with the same keys.
GitHub issue #809: Add Google Taskmaster dataset
State: closed | Created: 2020-11-06T15:10:41Z

## Adding a Dataset
- **Name:** Taskmaster
- **Description:** A large dataset of task-oriented dialogue with annotated goals (55K dialogues covering entertainment and travel reservations)
- **Paper:** https://arxiv.org/abs/1909.05358
- **Data:** https://github.com/google-research-datasets/Taskmaster
- **Motivation:** One of few annotated datasets of this size for goal-oriented dialogue

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).
GitHub issue #808: dataset(dgs): initial dataset loading script
State: closed | Created: 2020-11-06T10:14:43Z

When trying to create dummy data I get:

> Dataset datasets with config None seems to already open files in the method `_split_generators(...)`. You might consider to instead only open files in the method `_generate_examples(...)` instead. If this is not possible the dummy data has t o be created with less guidance. Make sure you create the file dummy_data.

I am not sure how to manually create the dummy_data (what exactly it should contain)

Also note, this library says:
> ImportError: To be able to use this dataset, you need to install the following dependencies['pympi'] using 'pip install pympi' for instance'

When you actually need to `pip install pympi-ling`
GitHub issue #806: Quail dataset urls are out of date
State: closed | Created: 2020-11-05T19:40:19Z

<h3>Code</h3>

```
from datasets import load_dataset
quail = load_dataset('quail')
```

<h3>Error</h3>

```
FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/text-machine-lab/quail/master/quail_v1.2/xml/ordered/quail_1.2_train.xml
```


As per [quail v1.3 commit](https://github.com/text-machine-lab/quail/commit/506501cfa34d9ec6c042d31026ba6fea6bcec8ff) it looks like the location and suggested ordering has changed. In [https://github.com/huggingface/datasets/blob/master/datasets/quail/quail.py#L52-L58](https://github.com/huggingface/datasets/blob/master/datasets/quail/quail.py#L52-L58) the quail v1.2 datasets are being pointed to, which don't exist anymore.
GitHub issue #805: On loading a metric from datasets, I get the following error
State: closed | Created: 2020-11-05T15:14:38Z

`from datasets import load_metric`

`metric = load_metric('bleurt')`

Traceback:
210 class _ArrayXDExtensionType(pa.PyExtensionType):
    211 
    212     ndims: int = None

AttributeError: module 'pyarrow' has no attribute 'PyExtensionType'

Any help will be appreciated. Thank you.
GitHub issue #804: Empty output/answer in TriviaQA test set (both in 'kilt_tasks' and 'trivia_qa')
State: closed | Created: 2020-11-05T11:38:01Z

# The issue

It's all in the title, it appears to be fine on the train and validation sets.

Is there some kind of mapping to do like for the questions (see https://github.com/huggingface/datasets/blob/master/datasets/kilt_tasks/README.md) ? 

# How to reproduce
```py
from datasets import load_dataset
kilt_tasks = load_dataset("kilt_tasks")
trivia_qa = load_dataset('trivia_qa', 'unfiltered.nocontext')
# both in "kilt_tasks"
In [18]: any([output['answer'] for output in kilt_tasks['test_triviaqa']['output']])                                                                                                                        
Out[18]: False
# and "trivia_qa"
In [13]: all([answer['value'] == '<unk>' for answer in trivia_qa['test']['answer']])                                                                                                                        
Out[13]: True
# appears to be fine on the train and validation sets.
In [14]: all([answer['value'] == '<unk>' for answer in trivia_qa['train']['answer']])                                                                                                                       
Out[14]: False

In [15]: all([answer['value'] == '<unk>' for answer in trivia_qa['validation']['answer']])                                                                                                                  
Out[15]: False

In [16]: any([output['answer'] for output in kilt_tasks['train_triviaqa']['output']])                                                                                                                       
Out[16]: True

In [17]: any([output['answer'] for output in kilt_tasks['validation_triviaqa']['output']])                                                                                                                  
Out[17]: True

```
GitHub issue #802: Add XGlue
State: closed | Created: 2020-11-04T17:29:54Z

Dataset is ready to merge. An important feature of this dataset is that for each config the train data is in English, while dev and test data are in multiple languages. Therefore, @lhoestq and I decided offline that we will give the dataset the following API, *e.g.* for 

```python
load_dataset("xglue", "ner")  # would give the splits 'train', 'validation.en', 'test.en', 'validation.es', 'test.es', ... 
```
=> therefore one can load a single language test via

```python
load_dataset("xglue", "ner", split="test.es")
```

Close #749.
GitHub issue #801: How to join two datasets?
State: closed | Created: 2020-11-04T03:53:11Z

Hi,

I'm wondering if it's possible to join two (preprocessed) datasets with the same number of rows but different labels? 

I'm currently trying to create paired sentences for BERT from `wikipedia/'20200501.en`, and I couldn't figure out a way to create a paired sentence using `.map()` where the second sentence is **not** the next sentence (i.e., from a different article) of the first sentence.

Thanks!
GitHub issue #797: Token classification labels are strings and we don't have the list of labels
State: closed | Created: 2020-11-03T15:33:30Z

Not sure if this is an issue we want to fix or not, putting it here so it's not forgotten. Right now, in token classification datasets, the labels for NER, POS and the likes are typed as `Sequence` of `strings`, which is wrong in my opinion. These should be `Sequence` of `ClassLabel` or some types that gives easy access to the underlying labels.

The main problem for preprocessing those datasets is that the list of possible labels is not stored inside the `Dataset` object which makes converting the labels to IDs quite difficult (you either have to know the list of labels in advance or run a full pass through the dataset to get the list of labels, the `unique` method being useless with the type `Sequence[str]`).
GitHub issue #795: Descriptions of raw and processed versions of wikitext are inverted
State: closed | Created: 2020-11-03T10:24:51Z

Nothing of importance, but it looks like the descriptions of wikitext-n-v1 and wikitext-n-raw-v1 are inverted for both n=2 and n=103. I just verified by loading them and the `<unk>` tokens are present in the non-raw versions, which confirms that it's a mere inversion of the descriptions and not of the datasets themselves.

Also it would be nice if those descriptions appeared in the dataset explorer.

https://github.com/huggingface/datasets/blob/87bd0864845ea0a1dd7167918dc5f341bf807bd3/datasets/wikitext/wikitext.py#L52
GitHub issue #793: [Datasets] fix discofuse links
State: closed | Created: 2020-11-03T08:03:45Z

The discofuse links were changed: https://github.com/google-research-datasets/discofuse/commit/d27641016eb5b3eb2af03c7415cfbb2cbebe8558. 
The old links are broken

I changed the links and created the new dataset_infos.json.

Pinging @thomwolf @lhoestq for notification.
GitHub issue #790: Error running pip install -e ".[dev]" on MacOS 10.13.6: faiss/python does not exist
State: closed | Created: 2020-11-02T12:36:35Z

I was following along with https://huggingface.co/docs/datasets/share_dataset.html#adding-tests-and-metadata-to-the-dataset when I ran into this error.

```sh
git clone https://github.com/huggingface/datasets
cd datasets
virtualenv venv -p python3 --system-site-packages
source venv/bin/activate
pip install -e ".[dev]"
```


![image](https://user-images.githubusercontent.com/59632/97868518-72871800-1cd5-11eb-9cd2-37d4e9d20b39.png)

![image](https://user-images.githubusercontent.com/59632/97868592-977b8b00-1cd5-11eb-8f3c-0c409616149c.png)

Python 3.7.7
GitHub issue #789: dataset(ncslgr): add initial loading script
State: closed | Created: 2020-11-02T06:50:10Z

Its a small dataset, but its heavily annotated
https://www.bu.edu/asllrp/ncslgr.html

![image](https://user-images.githubusercontent.com/5757359/97838609-3c539380-1ce9-11eb-885b-a15d4c91ea49.png)
GitHub issue #788: failed to reuse cache
State: closed | Created: 2020-11-02T02:42:36Z

I packed the `load_dataset ` in a function of class, and cached data in a directory. But when I import the class and use the function, the data still have to be downloaded again. The information (Downloading and preparing dataset cnn_dailymail/3.0.0 (download: 558.32 MiB, generated: 1.28 GiB, post-processed: Unknown size, total: 1.82 GiB) to ******) which logged to terminal shows the path is right to the cache directory, but the files still have to be downloaded again.
GitHub issue #787: Adding nli_tr dataset
State: closed | Created: 2020-11-01T21:49:44Z

Hello,

In this pull request, we have implemented the necessary interface to add our recent dataset [NLI-TR](https://github.com/boun-tabi/NLI-TR).  The datasets will be presented on a full paper at EMNLP 2020 this month. [[arXiv link] ](https://arxiv.org/pdf/2004.14963.pdf)

The dataset is the neural machine translation of SNLI and MultiNLI datasets into Turkish.  So, we followed a similar format with the original datasets hosted in the HuggingFace datasets hub.  

Our dataset is designed to be accessed as follows by following the interface of the GLUE dataset that provides multiple datasets in a single interface over the HuggingFace datasets hub.  

```
from datasets import load_dataset
multinli_tr = load_dataset("nli_tr", "multinli_tr")
snli_tr = load_dataset("nli_tr", "snli_tr")
```

Thanks for your help in reviewing our pull request.
GitHub issue #786: feat(dataset): multiprocessing _generate_examples
State: closed | Created: 2020-10-31T16:52:16Z

forking this out of #741, this issue is only regarding multiprocessing

I'd love if there was a dataset configuration parameter `workers`, where when it is `1` it behaves as it does right now, and when its `>1` maybe `_generate_examples` can also get the `pool` and return an iterable using the pool.

In my use case, I would instead of:
```python
for datum in data:
     yield self.load_datum(datum)
```
do:
```python
return pool.map(self.load_datum, data)
```

As the dataset in question, as an example, has **only** 7000 rows, and takes 10 seconds to load each row on average, it takes almost 20 hours to load the entire dataset.
If this was a larger dataset (and many such datasets exist), it would take multiple days to complete.

Using multiprocessing, for example, 40 cores, could speed it up dramatically. For this dataset, hopefully to fully load in under an hour.
GitHub issue #785: feat(aslg_pc12): add dev and test data splits
State: closed | Created: 2020-10-31T13:25:38Z

For reproducibility sake, it's best if there are defined dev and test splits.

The original paper author did not define splits for the entire dataset, not for the sample loaded via this library, so I decided to define:
- 5/7th for train
- 1/7th for dev
- 1/7th for test
GitHub issue #784: Issue with downloading Wikipedia data for low resource language
State: closed | Created: 2020-10-31T11:40:00Z

Hi, I tried to download Sundanese and Javanese wikipedia data with the following snippet
```
jv_wiki = datasets.load_dataset('wikipedia', '20200501.jv', beam_runner='DirectRunner')
su_wiki = datasets.load_dataset('wikipedia', '20200501.su', beam_runner='DirectRunner')
```
And I get the following error for these two languages:
Javanese
```
FileNotFoundError: Couldn't find file at https://dumps.wikimedia.org/jvwiki/20200501/dumpstatus.json
```

Sundanese
```
FileNotFoundError: Couldn't find file at https://dumps.wikimedia.org/suwiki/20200501/dumpstatus.json
```

I found from https://github.com/huggingface/datasets/issues/577#issuecomment-688435085 that for small languages, they are directly downloaded and parsed from the Wikipedia dump site, but both of `https://dumps.wikimedia.org/jvwiki/20200501/dumpstatus.json` and `https://dumps.wikimedia.org/suwiki/20200501/dumpstatus.json` are no longer valid.

 Any suggestions on how to handle this issue? Thanks!
GitHub issue #781: Add XNLI train set
State: closed | Created: 2020-10-30T13:21:53Z

I added the train set that was built using the translated MNLI.
Now you can load the dataset specifying one language:
```python
from datasets import load_dataset

xnli_en = load_dataset("xnli", "en")
print(xnli_en["train"][0])
# {'hypothesis': 'Product and geography are what make cream skimming work .', 'label': 1, 'premise': 'Conceptually cream skimming has two basic dimensions - product and geography .'}
print(xnli_en["test"][0])                                                                                                                   
# {'hypothesis': 'I havent spoken to him again.', 'label': 2, 'premise': "Well, I wasn't even thinking about that, but I was so frustrated, and, I ended up talking to him again."}
```

Cc @sgugger
GitHub issue #780: Add ASNQ dataset
State: closed | Created: 2020-10-29T23:31:56Z

This pull request adds the ASNQ dataset. It is a dataset for answer sentence selection derived from Google Natural Questions (NQ) dataset (Kwiatkowski et al. 2019). The dataset details can be found in the paper at https://arxiv.org/abs/1911.04118

The dataset is authored by Siddhant Garg, Thuy Vu and Alessandro Moschitti. 

_Please note that I have no affiliation with the authors._

Repo: https://github.com/alexa/wqa_tanda
GitHub issue #778: Unexpected behavior when loading cached csv file?
State: closed | Created: 2020-10-29T16:06:10Z

I read a csv file from disk and forgot so specify the right delimiter. When i read the csv file again specifying the right delimiter it had no effect since it was using the cached dataset. I am not sure if this is unwanted behavior since i can always specify `download_mode="force_redownload"`. But i think it would be nice if the information what `delimiter` or what `column_names` were used would influence the identifier of the cached dataset.

Small snippet to reproduce the behavior:
```python
import datasets

with open("dummy_data.csv", "w") as file:
    file.write("test,this;text\n")

print(datasets.load_dataset("csv", data_files="dummy_data.csv", split="train").column_names)
# ["test", "this;text"]

print(datasets.load_dataset("csv", data_files="dummy_data.csv", split="train", delimiter=";").column_names)
# still ["test", "this;text"]
```

By the way, thanks a lot for this amazing library! :)
GitHub issue #775: Properly delete metrics when a process is killed
State: closed | Created: 2020-10-29T12:52:07Z

Tests are flaky when using metrics in distributed setup.
There is because of one test that make sure that using two possibly incompatible metric computation (same exp id) either works or raises the right error.
However if the error is raised, all the processes of the metric are killed, and the open files (arrow + lock files) are not closed correctly. This causes PermissionError on Windows when deleting the temporary directory.

To fix that I added a `finally` clause in the function passed to multiprocess to properly close the files when the process exits.
GitHub issue #773: Adding CC-100: Monolingual Datasets from Web Crawl Data
State: closed | Created: 2020-10-28T18:20:41Z

## Adding a Dataset
- **Name:** CC-100: Monolingual Datasets from Web Crawl Data
- **Description:** https://twitter.com/alex_conneau/status/1321507120848625665
- **Paper:** https://arxiv.org/abs/1911.02116
- **Data:** http://data.statmt.org/cc-100/
- **Motivation:** A large scale multi-lingual language modeling dataset. Text is de-duplicated and filtered by how "Wikipedia-like" it is, hopefully helping avoid some of the worst parts of the common crawl.

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).
GitHub issue #770: Fix custom builder caching
State: closed | Created: 2020-10-28T13:32:24Z

The cache directory of a dataset didn't take into account additional parameters that the user could specify such as `features` or any parameter of the builder configuration kwargs (ex: `encoding` for the `text` dataset).

To fix that, the cache directory name now has a suffix that depends on all of them.

Fix #730
Fix #750
GitHub issue #769: How to choose proper download_mode in function load_dataset?
State: closed | Created: 2020-10-28T09:16:19Z

Hi, I am a beginner to datasets and I try to use datasets to load my csv file.
my csv file looks like this

``` 
text,label
"Effective but too-tepid biopic",3
"If you sometimes like to go to the movies to have fun , Wasabi is a good place to start .",4
"Emerges as something rare , an issue movie that 's so honest and keenly observed that it does n't feel like one .",5
```

First I try to use this command to load my csv file .  

``` python
dataset=load_dataset('csv', data_files=['sst_test.csv'])
```

It seems good, but when i try to overwrite the convert_options to convert  'label' columns from int64 to float32 like this.

``` python
import pyarrow as pa
from pyarrow import csv
read_options = csv.ReadOptions(block_size=1024*1024)
parse_options = csv.ParseOptions()
convert_options = csv.ConvertOptions(column_types={'text': pa.string(), 'label': pa.float32()})
dataset = load_dataset('csv', data_files=['sst_test.csv'], read_options=read_options,
                       parse_options=parse_options, convert_options=convert_options)
```

It keeps the same:

```shell
Dataset(features: {'text': Value(dtype='string', id=None), 'label': Value(dtype='int64', id=None)}, num_rows: 2210)
```

I think this issue is caused by the parameter "download_mode" Default to REUSE_DATASET_IF_EXISTS because after I delete the cache_dir, it seems right.

Is it a bug? How to choose proper download_mode to avoid this issue?
GitHub issue #768: Add a `lazy_map` method to `Dataset` and `DatasetDict`
State: open | Created: 2020-10-27T22:33:03Z

The library is great, but it would be even more awesome with a `lazy_map` method implemented on `Dataset` and `DatasetDict`. This would apply a function on a give item but when the item is requested. Two use cases:

1. load image on the fly
2. apply a random function and get different outputs at each epoch (like data augmentation or randomly masking a part of a sentence for BERT-like objectives).
GitHub issue #767: Add option for named splits when using ds.train_test_split
State: open | Created: 2020-10-27T19:59:44Z

### Feature Request 🚀 

Can we add a way to name your splits when using the `.train_test_split` function?

In almost every use case I've come across, I have a `train` and a `test` split in my `DatasetDict`, and I want to create a `validation` split. Therefore, its kinda useless to get a `test` split back from `train_test_split`, as it'll just overwrite my real `test` split that I intended to keep.

### Workaround

this is my hack for dealin with this, for now :slightly_smiling_face:

```python
from datasets import load_dataset
​
​
ds = load_dataset('imdb')
ds['train'], ds['validation'] = ds['train'].train_test_split(.1).values()
```
GitHub issue #766: [GEM] add DART data-to-text generation dataset
State: closed | Created: 2020-10-27T17:34:04Z

## Adding a Dataset
- **Name:** DART
- **Description:** DART consists of 82,191 examples across different domains with each input being a semantic RDF triple set derived from data records in tables and the tree ontology of the schema, annotated with sentence descriptions that cover all facts in the triple set.
- **Paper:** https://arxiv.org/abs/2007.02871v1
- **Data:** https://github.com/Yale-LILY/dart
- **Motivation:** the dataset will likely be included in the GEM benchmark

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).
GitHub issue #765: [GEM] Add DART data-to-text generation dataset
State: closed | Created: 2020-10-27T17:32:23Z

## Adding a Dataset
- **Name:** DART
- **Description:** DART consists of 82,191 examples across different domains with each input being a semantic RDF triple set derived from data records in tables and the tree ontology of the schema, annotated with sentence descriptions that cover all facts in the triple set.
- **Paper:** https://arxiv.org/abs/2007.02871v1
- **Data:** https://github.com/Yale-LILY/dart
- **Motivation:** It will likely be included in the GEM generation evaluation benchmark

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).
GitHub issue #763: Fixed errors in bertscore related to custom baseline
State: closed | Created: 2020-10-27T16:08:35Z

[bertscore version 0.3.6 ](https://github.com/Tiiiger/bert_score) added support for custom baseline files. This update added extra argument `baseline_path` to BERTScorer class as well as some extra boolean parameters `use_custom_baseline` in functions like `get_hash(model, num_layers, idf, rescale_with_baseline, use_custom_baseline)`.

This PR fix those matching errors in bertscore metric implementation.
GitHub issue #761: Downloaded datasets are not usable offline
State: closed | Created: 2020-10-26T20:54:46Z

I've been trying to use the IMDB dataset offline, but after downloading it and turning off the internet it still raises  an error from the ```requests``` library trying to reach for the online dataset.
Is this the intended behavior ?
(Sorry, I wrote the the first version of this issue while still on nlp 0.3.0).
GitHub issue #759: (Load dataset failure) ConnectionError: Couldn’t reach https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/cnn_dailymail/cnn_dailymail.py
State: closed | Created: 2020-10-25T15:34:57Z

Hey, I want to load the cnn-dailymail dataset for fine-tune.
I write the code like this
from datasets import load_dataset

test_dataset = load_dataset(“cnn_dailymail”, “3.0.0”, split=“train”)

And I got the following errors.

Traceback (most recent call last):
File “test.py”, line 7, in
test_dataset = load_dataset(“cnn_dailymail”, “3.0.0”, split=“test”)
File “C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\load.py”, line 589, in load_dataset
module_path, hash = prepare_module(
File “C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\load.py”, line 268, in prepare_module
local_path = cached_path(file_path, download_config=download_config)
File “C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\utils\file_utils.py”, line 300, in cached_path
output_path = get_from_cache(
File “C:\Users\666666\AppData\Local\Programs\Python\Python38\lib\site-packages\datasets\utils\file_utils.py”, line 475, in get_from_cache
raise ConnectionError(“Couldn’t reach {}”.format(url))
ConnectionError: Couldn’t reach https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/cnn_dailymail/cnn_dailymail.py

How can I fix this ?
GitHub issue #758: Process 0 very slow when using num_procs with map to tokenizer
State: closed | Created: 2020-10-24T02:40:20Z

<img width="721" alt="image" src="https://user-images.githubusercontent.com/17930170/97066109-776d0d00-15ed-11eb-8bba-bb4d2e0fcc33.png">
The code I am using is
```

        dataset = load_dataset("text", data_files=[file_path], split='train')
        dataset = dataset.map(lambda ex: tokenizer(ex["text"], add_special_tokens=True,
                                                truncation=True, max_length=args.block_size), num_proc=8)
        dataset.set_format(type='torch', columns=['input_ids'])
        dataset.save_to_disk(file_path+'.arrow')
```
GitHub issue #756: Start community-provided dataset docs 
State: closed | Created: 2020-10-23T13:17:41Z

Continuation of #736 with clean fork.

#### Old description
This is what I did to get the pseudo-labels updated. Not sure if it generalizes, but I figured I would write it down. It was pretty easy because all I had to do was make properly formatted directories and change URLs.

In slack @thomwolf called it a user-namespace dataset, but the docs call it community dataset.
I think the first naming is clearer, but I didn't address that here.

I didn't add metadata, will try that.
GitHub issue #752: Clicking on a metric in the search page points to datasets page giving "Missing dataset" warning
State: closed | Created: 2020-10-21T22:56:23Z

Hi! Sorry if this isn't the right place to talk about the website, I just didn't exactly where to write this.

Searching a metric in https://huggingface.co/metrics gives the right results but clicking on a metric (E.g ROUGE) points to https://huggingface.co/datasets/rouge. Clicking on a metric without searching points to the right page.

Thanks for all the great work!
GitHub issue #751: Error loading ms_marco v2.1 using load_dataset()
State: closed | Created: 2020-10-21T19:54:43Z

Code:
`dataset = load_dataset('ms_marco', 'v2.1')`

Error:
```
`---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
<ipython-input-16-34378c057212> in <module>()
      9 
     10 # Downloading and loading a dataset
---> 11 dataset = load_dataset('ms_marco', 'v2.1')

10 frames
/usr/lib/python3.6/json/decoder.py in raw_decode(self, s, idx)
    353         """
    354         try:
--> 355             obj, end = self.scan_once(s, idx)
    356         except StopIteration as err:
    357             raise JSONDecodeError("Expecting value", s, err.value) from None

JSONDecodeError: Unterminated string starting at: line 1 column 388988661 (char 388988660)
`
```
GitHub issue #750: load_dataset doesn't include `features` in its hash
State: closed | Created: 2020-10-21T15:16:41Z

It looks like the function `load_dataset` does not include what's passed in the `features` argument when creating a hash for a given  dataset. As a result, if a user includes new features from an already downloaded dataset, those are ignored.

Example: some models on the hub have a different ordering for the labels than what `datasets` uses for MNLI so I'd like to do something along the lines of:
```
dataset = load_dataset("glue", "mnli")
features = dataset["train"].features
features["label"] = ClassLabel(names = ['entailment', 'contradiction', 'neutral'])  # new label order
dataset = load_dataset("glue", "mnli", features=features)
```
GitHub issue #749: [XGLUE] Adding new dataset
State: closed | Created: 2020-10-21T10:51:36Z

XGLUE is a multilingual GLUE like dataset propesed in this [paper](https://arxiv.org/pdf/2004.01401.pdf).

I'm planning on adding the dataset to the library myself in a couple of weeks.
Also tagging @JetRunner @qiweizhen in case I need some guidance
GitHub issue #747: Add Quail question answering dataset
State: closed | Created: 2020-10-20T19:33:14Z

QuAIL is a multi-domain RC dataset featuring news, blogs, fiction and user stories. Each domain is represented by 200 texts,    which gives us a 4-way data split. The texts are 300-350 word excerpts from CC-licensed texts that were hand-picked so as to make sense to human readers without larger context. Domain diversity mitigates the issue of possible overlap between training and test data of large pre-trained models, which the current SOTA systems are based on. For instance, BERT is trained on Wikipedia + BookCorpus, and was tested on Wikipedia-based SQuAD (Devlin, Chang, Lee, & Toutanova, 2019).

https://text-machine-lab.github.io/blog/2020/quail/ @annargrs
GitHub issue #746: dataset(ngt): add ngt dataset initial loading script
State: closed | Created: 2020-10-20T14:04:58Z

Currently only making the paths to the annotation ELAN (eaf) file and videos available.
This is the first accessible way to download this dataset, which is not manual file-by-file.

Only downloading the necessary files, the annotation files are very small, 20MB for all of them, but the video files are large, 100GB in total, saved in `mpg` format. 
I do not intend to actually store these as an uncompressed array of frames, because it will be huge.

Future updates may add pose estimation files for all videos, making it easier to work with this data
GitHub issue #743: load_dataset for CSV files not working
State: open | Created: 2020-10-19T14:53:51Z

Similar to #622, I've noticed there is a problem when trying to load a CSV file with datasets.

`
from datasets import load_dataset
`
`
dataset = load_dataset("csv", data_files=["./sample_data.csv"], delimiter="\t", column_names=["title", "text"], script_version="master")
`

Displayed error:
`
...
ArrowInvalid: CSV parse error: Expected 2 columns, got 1
`

I should mention that when I've tried to read data from `https://github.com/lhoestq/transformers/tree/custom-dataset-in-rag-retriever/examples/rag/test_data/my_knowledge_dataset.csv` it worked without a problem. I've read that there might be some problems with /r character, so I've removed them from the custom dataset, but the problem still remains.

I've added a colab reproducing the bug, but unfortunately I cannot provide the dataset.
https://colab.research.google.com/drive/1Qzu7sC-frZVeniiWOwzoCe_UHZsrlxu8?usp=sharing

Are there any work around for it ?
Thank you
GitHub issue #742: Add OCNLI, a new CLUE dataset
State: closed | Created: 2020-10-19T11:06:33Z

OCNLI stands for Original Chinese Natural Language Inference. It is a corpus for
            Chinese Natural Language Inference, collected following closely the procedures of MNLI,
            but with enhanced strategies aiming for more challenging inference pairs. We want to
            emphasize we did not use human/machine translation in creating the dataset, and thus
            our Chinese texts are original and not translated.
GitHub issue #741: Creating dataset consumes too much memory
State: closed | Created: 2020-10-18T06:07:06Z

Moving this issue from https://github.com/huggingface/datasets/pull/722 here, because it seems like a general issue.

Given the following dataset example, where each example saves a sequence of 260x210x3 images (max length 400):
```python
    def _generate_examples(self, base_path, split):
        """ Yields examples. """

        filepath = os.path.join(base_path, "annotations", "manual", "PHOENIX-2014-T." + split + ".corpus.csv")
        images_path = os.path.join(base_path, "features", "fullFrame-210x260px", split)

        with open(filepath, "r", encoding="utf-8") as f:
            data = csv.DictReader(f, delimiter="|", quoting=csv.QUOTE_NONE)
            for row in data:
                frames_path = os.path.join(images_path, row["video"])[:-7]
                np_frames = []
                for frame_name in os.listdir(frames_path):
                    frame_path = os.path.join(frames_path, frame_name)
                    im = Image.open(frame_path)
                    np_frames.append(np.asarray(im))
                    im.close()

                yield row["name"], {"video": np_frames}
```

The dataset creation process goes out of memory on a machine with 500GB RAM.
I was under the impression that the "generator" here is exactly for that, to avoid memory constraints.


However, even if you want the entire dataset in memory, it would be in the worst case
`260x210x3 x 400 max length x 7000 samples` in bytes (uint8) = 458.64 gigabytes
So I'm not sure why it's taking more than 500GB.

And the dataset creation fails after 170 examples on a machine with 120gb RAM, and after 672 examples on a machine with 500GB RAM.


---

## Info that might help:
Iterating over examples is extremely slow.
![image](https://user-images.githubusercontent.com/5757359/96359590-3c666780-111d-11eb-9347-1f833ad982a9.png)
If I perform this iteration in my own, custom loop (Without saving to file), it runs at 8-9 examples/sec

And you can see at this state it is using 94% of the memory:
![image](https://user-images.githubusercontent.com/5757359/96359606-7afc2200-111d-11eb-8c11-0afbdba1a6a3.png)

And it is only using one CPU core, which is probably why it's so slow:
![image](https://user-images.githubusercontent.com/5757359/96359630-a3841c00-111d-11eb-9ba0-7fd3cdf51d26.png)
GitHub issue #739: Add wiki dpr multiset embeddings
State: closed | Created: 2020-10-16T09:05:49Z

There are two DPR encoders, one trained on Natural Questions and one trained on a multiset/hybrid dataset.
Previously only the embeddings from the encoder trained on NQ were available. I'm adding the ones from the encoder trained on the multiset/hybrid dataset.
In the configuration you can now specify `embeddings_name="nq"` or `embeddings_name="multiset"`
GitHub issue #738: Replace seqeval code with original classification_report for simplicity
State: closed | Created: 2020-10-16T08:51:45Z

Recently, the original seqeval has enabled us to get per type scores and overall scores as a dictionary.

This PR replaces the current code with the original function(`classification_report`) to simplify it.

Also, the original code has been updated to fix #352.
- Related issue: https://github.com/chakki-works/seqeval/pull/38


```python
from datasets import load_metric
metric = load_metric("seqeval")
y_true = [['O', 'O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
y_pred = [['O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
metric.compute(predictions=y_pred, references=y_true)
# Output: {'MISC': {'precision': 0.0, 'recall': 0.0, 'f1': 0, 'number': 1}, 'PER': {'precision': 1.0, 'recall': 1.0, 'f1': 1.0, 'number': 1}, 'overall_precision': 0.5, 'overall_recall': 0.5, 'overall_f1': 0.5, 'overall_accuracy': 0.8}
```
GitHub issue #737: Trec Dataset Connection Error
State: closed | Created: 2020-10-15T15:57:53Z

**Datasets Version:**
1.1.2

**Python Version:**
3.6/3.7


**Code:**
```python
from datasets import load_dataset
load_dataset("trec")
```

**Expected behavior:**
Download Trec dataset and load Dataset object

**Current Behavior:**
Get a connection error saying it couldn't reach http://cogcomp.org/Data/QA/QC/train_5500.label (but the link doesn't seem broken)

<details>
  <summary>Error Logs</summary>
 

Using custom data configuration default
Downloading and preparing dataset trec/default (download: 350.79 KiB, generated: 403.39 KiB, post-processed: Unknown size, total: 754.18 KiB) to /root/.cache/huggingface/datasets/trec/default/1.1.0/ca4248481ad244f235f4cf277186cad2ee8769f975119a2bbfc41b8932b88bd7...
---------------------------------------------------------------------------
ConnectionError                           Traceback (most recent call last)
<ipython-input-8-66bf1242096e> in <module>()
----> 1 load_dataset("trec")

10 frames
/usr/local/lib/python3.6/dist-packages/datasets/utils/file_utils.py in get_from_cache(url, cache_dir, force_download, proxies, etag_timeout, resume_download, user_agent, local_files_only, use_etag)
    473         elif response is not None and response.status_code == 404:
    474             raise FileNotFoundError("Couldn't find file at {}".format(url))
--> 475         raise ConnectionError("Couldn't reach {}".format(url))
    476 
    477     # Try a second time

ConnectionError: Couldn't reach http://cogcomp.org/Data/QA/QC/train_5500.label

</details>
GitHub issue #736: Start community-provided dataset docs
State: closed | Created: 2020-10-15T13:41:39Z

This is one I did to get the pseudo-labels updated. Not sure if it generalizes, but I figured I would write it down. It was pretty easy because all I had to do was make properly formatted directories and change URLs.

+ In slack @thomwolf  called it a `user-namespace` dataset, but the docs call it `community dataset`.
I think the first naming is clearer, but I didn't address that here.


+ I didn't add metadata, will try that.
GitHub issue #735: Throw error when an unexpected key is used in data_files
State: closed | Created: 2020-10-15T10:55:27Z

I have found that only "train", "validation" and "test" are valid keys in the `data_files` argument. When you use any other ones, those attached files are silently ignored - leading to unexpected behaviour for the users.

So the following, unintuitively, returns only one key (namely `train`).

```python
datasets = load_dataset("text", data_files={"train": train_f, "valid": valid_f})
print(datasets.keys())
# dict_keys(['train'])
```

whereas using `validation` instead, does return the expected result:

```python
datasets = load_dataset("text", data_files={"train": train_f, "validation": valid_f})
print(datasets.keys())
# dict_keys(['train', 'validation'])
```

I would like to see more freedom in which keys one can use, but if that is not possible at least an error should be thrown when using an unexpected key.
GitHub issue #730: Possible caching bug
State: closed | Created: 2020-10-14T02:02:34Z

The following code with `test1.txt` containing just "🤗🤗🤗":
```
dataset = datasets.load_dataset('text', data_files=['test1.txt'], split="train", encoding="latin_1")
print(dataset[0])
dataset = datasets.load_dataset('text', data_files=['test1.txt'], split="train", encoding="utf-8")
print(dataset[0])
``` 
produces this output:
```
Downloading and preparing dataset text/default-15600e4d83254059 (download: Unknown size, generated: Unknown size, post-processed: Unknown size, total: Unknown size) to /home/arne/.cache/huggingface/datasets/text/default-15600e4d83254059/0.0.0/52cefbb2b82b015d4253f1aeb1e6ee5591124a6491e834acfe1751f765925155...
Dataset text downloaded and prepared to /home/arne/.cache/huggingface/datasets/text/default-15600e4d83254059/0.0.0/52cefbb2b82b015d4253f1aeb1e6ee5591124a6491e834acfe1751f765925155. Subsequent calls will reuse this data.
{'text': 'ð\x9f¤\x97ð\x9f¤\x97ð\x9f¤\x97'}
Using custom data configuration default
Reusing dataset text (/home/arne/.cache/huggingface/datasets/text/default-15600e4d83254059/0.0.0/52cefbb2b82b015d4253f1aeb1e6ee5591124a6491e834acfe1751f765925155)
{'text': 'ð\x9f¤\x97ð\x9f¤\x97ð\x9f¤\x97'}
```
Just changing the order (and deleting the temp files):
```
dataset = datasets.load_dataset('text', data_files=['test1.txt'], split="train", encoding="utf-8")
print(dataset[0])
dataset = datasets.load_dataset('text', data_files=['test1.txt'], split="train", encoding="latin_1")
print(dataset[0])
```
produces this:
```
Using custom data configuration default
Downloading and preparing dataset text/default-15600e4d83254059 (download: Unknown size, generated: Unknown size, post-processed: Unknown size, total: Unknown size) to /home/arne/.cache/huggingface/datasets/text/default-15600e4d83254059/0.0.0/52cefbb2b82b015d4253f1aeb1e6ee5591124a6491e834acfe1751f765925155...
Dataset text downloaded and prepared to /home/arne/.cache/huggingface/datasets/text/default-15600e4d83254059/0.0.0/52cefbb2b82b015d4253f1aeb1e6ee5591124a6491e834acfe1751f765925155. Subsequent calls will reuse this data.
{'text': '🤗🤗🤗'}
Using custom data configuration default
Reusing dataset text (/home/arne/.cache/huggingface/datasets/text/default-15600e4d83254059/0.0.0/52cefbb2b82b015d4253f1aeb1e6ee5591124a6491e834acfe1751f765925155)
{'text': '🤗🤗🤗'}
```

Is it intended that the cache path does not depend on the config entries?

tested with datasets==1.1.2 and python==3.8.5
GitHub issue #729: Better error message when one forgets to call `add_batch` before `compute`
State: closed | Created: 2020-10-12T17:59:22Z

When using metrics, if for some reason a user forgets to call `add_batch` to a metric before `compute` (with no arguments), the error message is a bit cryptic and could probably be made clearer.

## Reproducer

```python
import datasets
import torch
from datasets import Metric

class GatherMetric(Metric):
    def _info(self):
        return datasets.MetricInfo(
            description="description",
            citation="citation",
            inputs_description="kwargs",
            features=datasets.Features({
                'predictions': datasets.Value('int64'),
                'references': datasets.Value('int64'),
            }),
            codebase_urls=[],
            reference_urls=[],
            format='numpy'
        )

    def _compute(self, predictions, references):
        return {"predictions": predictions, "labels": references}

metric = GatherMetric(cache_dir="test-metric")
inputs = torch.randint(0, 2, (1024,))
targets = torch.randint(0, 2, (1024,))

batch_size = 8
for i in range(0, 1024, batch_size):
     pass # User forgets to call `add_batch`
result = metric.compute()
```

## Stack trace:

```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-267729d187fa> in <module>
      3      pass
      4     # metric.add_batch(predictions=inputs[i:i+batch_size], references=targets[i:i+batch_size])
----> 5 result = metric.compute()

~/git/datasets/src/datasets/metric.py in compute(self, *args, **kwargs)
    380         if predictions is not None:
    381             self.add_batch(predictions=predictions, references=references)
--> 382         self._finalize()
    383 
    384         self.cache_file_name = None

~/git/datasets/src/datasets/metric.py in _finalize(self)
    343         elif self.process_id == 0:
    344             # Let's acquire a lock on each node files to be sure they are finished writing
--> 345             file_paths, filelocks = self._get_all_cache_files()
    346 
    347             # Read the predictions and references

~/git/datasets/src/datasets/metric.py in _get_all_cache_files(self)
    280         filelocks = []
    281         for process_id, file_path in enumerate(file_paths):
--> 282             filelock = FileLock(file_path + ".lock")
    283             try:
    284                 filelock.acquire(timeout=self.timeout)

TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
```
GitHub issue #727: Parallel downloads progress bar flickers
State: open | Created: 2020-10-12T13:36:05Z

When there are parallel downloads using the download manager, the tqdm progress bar flickers since all the progress bars are on the same line.

To fix that we could simply specify `position=i` for i=0 to n the number of files to download when instantiating the tqdm progress bar. 

Another way would be to have one "master" progress bar that tracks the number of finished downloads, and then one progress bar per process that show the current downloads.
GitHub issue #726: "Checksums didn't match for dataset source files" error while loading openwebtext dataset
State: closed | Created: 2020-10-12T11:45:10Z

Hi,
I have encountered this problem during loading the openwebtext dataset:
```
>>> dataset = load_dataset('openwebtext')
Downloading and preparing dataset openwebtext/plain_text (download: 12.00 GiB, generated: 37.04 GiB, post-processed: Unknown size, total: 49.03 GiB) to /home/admin/.cache/huggingface/datasets/openwebtext/plain_text/1.0.0/5c636399c7155da97c982d0d70ecdce30fbca66a4eb4fc768ad91f8331edac02...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/admin/workspace/anaconda3/envs/torch1.6-py3.7/lib/python3.7/site-packages/datasets/load.py", line 611, in load_dataset
    ignore_verifications=ignore_verifications,
  File "/home/admin/workspace/anaconda3/envs/torch1.6-py3.7/lib/python3.7/site-packages/datasets/builder.py", line 476, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/home/admin/workspace/anaconda3/envs/torch1.6-py3.7/lib/python3.7/site-packages/datasets/builder.py", line 536, in _download_and_prepare
    self.info.download_checksums, dl_manager.get_recorded_sizes_checksums(), "dataset source files"
  File "/home/admin/workspace/anaconda3/envs/torch1.6-py3.7/lib/python3.7/site-packages/datasets/utils/info_utils.py", line 39, in verify_checksums
    raise NonMatchingChecksumError(error_msg + str(bad_urls))
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://zenodo.org/record/3834942/files/openwebtext.tar.xz']
```
I think this problem is caused because the released dataset has changed. Or I should download the dataset manually?

Sorry for release the unfinised issue by mistake.