konrud/llm_course_chapter5_own_dataset_github_issues huggingface dataset:
Added PragmEval benchmark


> Really cool ! Thanks for adding this one :)
> Good job at adding all those citations for each task
> 
> Looks like the dummy data test doesn't pass. Maybe some files are missing in the dummy_data.zip files ?
> The error reports `pragmeval/verifiability/train.tsv` to be missing
> 
> Also could you add the tags part of the dataset card (the rest is optional) ?
> See more info here : https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#tag-the-dataset-and-write-the-dataset-card

In the prior commits I generated dataset_infos and the dummy files myself
Now they are generated with the cli, and the tests now seem to be passing better
I will look into the tag


Looks like you did a good job with dummy data in the first place !
The downside of automatically generated dummy data is that the files are heavier (here 40KB per file).
If you could replace the generated dummy files with the one you created yourself it would be awesome, since the one you did yourself are way lighter (around 1KB per file). Using small files make `git clone` run faster so we encourage to use small dummy_data files.

could you rebase from master ? it should fix the CI

> could you rebase from master ? it should fix the CI

I think it is due to the file structure of the dummy data that causes test failure. The automatically generated dummy data pass the tests

Indeed the error reports that `pragmeval/verifiability/train.tsv` is missing for the verifiability dummy_data.zip file.
To fix that you should add the missing data files in each dummy_data.zip file.
To test that your dummy data work you can run
```
RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_
```
if some file is missing it should tell you which one

Also it looks like you haven't rebased from master yet, even though you did a `rebase` commit. 

rebasing should fix the other CI fails

It's ok if we have `RemoteDatasetTest ` errors, they're fixed on master

merging since the CI is fixed on master

Hey @sileod! Super nice to see you participating ;)

Did you officially joined the sprint by posting on [the forum thread](https://discuss.huggingface.co/t/open-to-the-community-one-week-team-effort-to-reach-v2-0-of-hf-datasets-library/2176) and joining our slack?

I can't seem to find you there! Should I add you directly with your gmail address?

Hi @sileod 👋
add prachathai67k

`prachathai-67k`: News Article Corpus and Multi-label Text Classificdation from Prachathai.com
The prachathai-67k dataset was scraped from the news site Prachathai.
We filtered out those articles with less than 500 characters of body text, mostly images and cartoons.
It contains 67,889 articles wtih 12 curated tags from August 24, 2004 to November 15, 2018.
The dataset was originally scraped by @lukkiddd and cleaned by @cstorm125.
You can also see preliminary exploration at https://github.com/PyThaiNLP/prachathai-67k/blob/master/exploration.ipynb

Test failing for same issues as https://github.com/huggingface/datasets/pull/939
Please advise.

```
=========================== short test summary info ============================
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_class_flue
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_class_norwegian_ner
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_configs_flue
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_configs_norwegian_ner
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_flue
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_norwegian_ner
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_xglue
===== 7 failed, 1309 passed, 932 skipped, 11 warnings in 166.71s (0:02:46) =====
```

Closing and opening a new pull request to solve rebase issues

To be continued on https://github.com/huggingface/datasets/pull/982
added health_fact dataset 

Added dataset Explainable Fact-Checking for Public Health Claims (dataset_id: health_fact)

Hi @lhoestq,
Initially I tried int(-1) only in place of nan labels and missing values but I kept on getting this error ```pyarrow.lib.ArrowTypeError: Expected bytes, got a 'int' object``` maybe because I'm sending int values (-1) to objects which are string type
Prachathai67k

Add `prachathai-67k`: News Article Corpus and Multi-label Text Classificdation from Prachathai.com

The `prachathai-67k` dataset was scraped from the news site [Prachathai](prachathai.com). We filtered out those articles with less than 500 characters of body text, mostly images and cartoons. It contains 67,889 articles wtih 12 curated tags from August 24, 2004 to November 15, 2018. The dataset was originally scraped by [@lukkiddd](https://github.com/lukkiddd) and cleaned by [@cstorm125](https://github.com/cstorm125). Download the dataset [here](https://www.dropbox.com/s/fsxepdka4l2pr45/prachathai-67k.zip?dl=1). You can also see preliminary exploration in [exploration.ipynb](https://github.com/PyThaiNLP/prachathai-67k/blob/master/exploration.ipynb).

This dataset is a part of [pyThaiNLP](https://github.com/PyThaiNLP/) Thai text [classification-benchmarks](https://github.com/PyThaiNLP/classification-benchmarks). For the benchmark, we selected the following tags with substantial volume that resemble **classifying types of articles**:

* `การเมือง` - politics
* `สิทธิมนุษยชน` - human_rights
* `คุณภาพชีวิต` - quality_of_life
* `ต่างประเทศ` - international
* `สังคม` - social
* `สิ่งแวดล้อม` - environment
* `เศรษฐกิจ` - economics
* `วัฒนธรรม` - culture
* `แรงงาน` - labor
* `ความมั่นคง` - national_security
* `ไอซีที` - ict
* `การศึกษา` - education

Wrongly branching from existing branch of wisesight_sentiment. Closing and opening another one specifically for prachathai67k
Add People's Daily NER dataset


> LGTM thanks :)
> 
> 
> 
> Before we merge, could you add a dataset card ? see here for more info: https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#tag-the-dataset-and-write-the-dataset-card
> 
> 
> 
> Note that only the tags at the top of the dataset card are mandatory, if you feel like it's going to take too much time writing the rest to fill it all you can just skip the paragraphs

Nope. I don't think there is a citation. Also, can I do the dataset card later (maybe in bulk)?

We're doing one PR = one dataset to keep track of things. Feel free to add the tags later in this PR if you want to.
Also only the tags are required now, because we don't want people spending too much time on the cards

added @lhoestq 

Merging since the CI is fixed on master
Local machine/cluster Beam Datasets example/tutorial

Hi,

I'm wondering if https://huggingface.co/docs/datasets/beam_dataset.html has an non-GCP or non-Dataflow version example/tutorial? I tried to migrate it to run on DirectRunner and SparkRunner, however, there were way too many runtime errors that I had to fix during the process, and even so I wasn't able to get either runner correctly producing the desired output.

Thanks!
Shang

I tried to make it run once on the SparkRunner but it seems that this runner has some issues when it is run locally.
From my experience the DirectRunner is fine though, even if it's clearly not memory efficient.

It would be awesome though to make it work locally on a SparkRunner !
Did you manage to make your processing work ?

We've deprecated the Beam API in `datasets`. As part of it, the Beam datasets have also been converted to non-Beam-based to make using them straightforward.
add PIB dataset

This pull request will add PIB dataset.

Hi, 

I am unable to get success in these tests. Can someone help me by pointing out possible errors?

Thanks

Hi ! you can read the tests by logging in to circleci.

Anyway for information here are the errors : 
```
datasets/pib/pib.py:19:1: F401 'csv' imported but unused
datasets/pib/pib.py:20:1: F401 'json' imported but unused
datasets/pib/pib.py:36:84: W291 trailing whitespace
```
and 
```
FAILED tests/test_file_encoding.py::TestFileEncoding::test_no_encoding_on_file_open
```

To fix the `test_no_encoding_on_file_open` you just have to specify an encoding while opening a text file. For example `encoding="utf-8"`


All suggested changes are done.

Nice ! can you re-generate the dataset_infos.json file to take into account the feature type change ?
```
datasets-cli test ./datasets/pib --save_infos --all_configs --ignore_verifications
```
And also format your code ?
```
make style
```
[WIP] complex_webqa - Error zipfile.BadZipFile: Bad CRC-32

Have a string `zipfile.BadZipFile: Bad CRC-32 for file 'web_snippets_train.json'` error when downloading the largest file from dropbox: `https://www.dropbox.com/sh/7pkwkrfnwqhsnpo/AABVENv_Q9rFtnM61liyzO0La/web_snippets_train.json.zip?dl=1`

Didn't managed to see how to solve that.

Putting aside for now.


Thanks for your contribution, @thomwolf. Are you still interested in adding this dataset?

We are removing the dataset scripts from this GitHub repo and moving them to the Hugging Face Hub: https://huggingface.co/datasets

We would suggest that you create this dataset there. Please, feel free to tell us if you need some help.
add inquisitive

Adding inquisitive qg dataset

More info: https://github.com/wjko2/INQUISITIVE

`dummy_data` right now contains all article files, keeping only the required articles for dummy data fails the dummy data test.
Any idea ?

> `dummy_data` right now contains all article files, keeping only the required articles for dummy data fails the dummy data test.
> Any idea ?

We should definitely find a way to make it work with only a few articles.

If it doesn't work right now for dummy data, I guess it's because it tries to load every single article file ?

If so, then maybe you can use `os.listdir` method to first check all the data files available in the path where the `articles.tgz` file is extracted. Then you can simply iter through the data files and depending on their ID, include them in the train or test set. With this method you should be able to have only a few articles files per split in the dummy data. Does that make sense ?

fixed! so the issue was, `articles_ids` were prepared based on the number of files in articles dir, so for dummy data questions it was not able to load some articles due to incorrect ids and the test was failing
Add DART

- **Name:** *DART*
- **Description:** *DART is a large dataset for open-domain structured data record to text generation.*
- **Paper:** *https://arxiv.org/abs/2007.02871*
- **Data:** *https://github.com/Yale-LILY/dart#leaderboard*

### Checkbox

- [x] Create the dataset script `/datasets/my_dataset/my_dataset.py` using the template
- [x] Fill the `_DESCRIPTION` and `_CITATION` variables
- [x] Implement `_infos()`, `_split_generators()` and `_generate_examples()`
- [x] Make sure that the `BUILDER_CONFIGS` class attribute is filled with the different configurations of the dataset and that the `BUILDER_CONFIG_CLASS` is specified if there is a custom config class.
- [x] Generate the metadata file `dataset_infos.json` for all configurations
- [x] Generate the dummy data `dummy_data.zip` files to have the dataset script tested and that they don't weigh too much (<50KB)
- [x] Add the dataset card `README.md` using the template : fill the tags and the various paragraphs
- [x] Both tests for the real data and the dummy data pass.

LGTM!
Add CC-100 dataset

Add CC-100.

Close #773 

Hello @lhoestq, I would like just to ask you if it is OK that I include this feature 9f32ba1 in this PR or you would prefer to have it in a separate one.

I was wondering whether include also a test, but I did not find any test for the other file formats...

Hi ! Sure that would be valuable to support .xz files. Feel free to open a separate PR for this.
And feel free to create the first test case for extracting compressed files if you have some inspiration (maybe create test_file_utils.py ?). We can still spend more time on tests next week when the sprint is over though so don't spend too much time on it.

@lhoestq, DONE! ;) See PR #950.

Thanks for adding support for `.xz` files :)

Feel free to rebase from master to include it in your PR

@lhoestq DONE; I have merged instead, to avoid changing the history of my public PR ;)

Hi @lhoestq, I would need that you generate the dataset_infos.json and the dummy data for this dataset with a bigger computer. Sorry, but my laptop did not succeed...

Thanks for your work @albertvillanova 
We'll definitely look into it after this sprint :)

Looks like #1456 added CC100 already.
The difference with your approach is that this implementation uses the `BuilderConfig` parameters to allow the creation of custom configs for all the languages, without having to specify them in the `BUILDER_CONFIGS` class attribute.
For example even if the dataset doesn't have a config for english already, you can still load the english CC100 with
```python
from datasets import load_dataset

load_dataset("cc100", lang="en")
```

@lhoestq, oops!! I remember having assigned this dataset to me in the Google sheet, besides having mentioned the corresponding issue in the Pull Request... Nevermind! :)

Yes indeed I can see that...
Sorry for noticing that only now 

The code of the other PR ended up being pretty close to yours though
If you want to add more details to the cc100 dataset card or in the script feel to do so, any addition is welcome
Add XOR QA Dataset

Added XOR Question Answering Dataset. The link to the dataset can be found [here](https://nlp.cs.washington.edu/xorqa/)

- [x] Followed the instructions in CONTRIBUTING.md
- [x] Ran the tests successfully
- [x] Created the dummy data

Hi @sumanthd17 

Looks like a good start! You will also need to add a Dataset card, following the instructions given [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#manually-tag-the-dataset-and-write-the-dataset-card)

I followed the instructions mentioned there but my dataset isn't showing up in the dropdown list. Am I missing something here? @yjernite 

> I followed the instructions mentioned there but my dataset isn't showing up in the dropdown list. Am I missing something here? @yjernite

The best way is to run the tagging app locally and provide it the location to the `dataset_infos.json` after you've run the CLI:
https://github.com/huggingface/datasets-tagging


This is a really good data card!!

Small changes to make it even better:
- Tags: the dataset has both "original" data and data that is "extended" from a source dataset: TydiQA - you should choose both options in the tagging apps
- The language and annotation creator tags are off: the language here is the questions: I understand it's a mix of crowd-sourced and expert-generated? Is there any machine translation involved? The annotations are the span selections: is that crowd-sourced?
- Personal and sensitive information: there should be a statement there, even if only to say that none could be found or that it only mentions public figures
add dream dataset

Adding Dream: a Dataset and for Dialogue-Based Reading Comprehension

More details:
https://dataset.org/dream/
https://github.com/nlpdata/dream

> Awesome good job !
> 
> Could you also add a dataset card using the template guide here : https://github.com/huggingface/datasets/blob/master/templates/README_guide.md
> If you can't fill some fields then just leave `[N/A]`

Quick amendment: `[N/A]` is for fields that are not relevant: if you can't find the information just leave `[More Information Needed]`

@lhoestq since datset cards are optional for this sprint I'll add those later. Good for merge.

Indeed we only require the tags to be added now (the yaml part at the top of the dataset card).
Could you add them please ?
You can find more infos here : https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#tag-the-dataset-and-write-the-dataset-card

@lhoestq added tags, I'll fill rest of the info after current sprint :)

The tests are failing tests for other datasets, not this one.

@lhoestq  could you tell me why these tests are failing, they don't seem related to this PR.
wrong length with datasets 

Hi
I have a MRPC dataset which I convert it to seq2seq format, then this is of this format:

`Dataset(features: {'src_texts': Value(dtype='string', id=None), 'tgt_texts': Value(dtype='string', id=None)}, num_rows: 10)
`

I feed it to a dataloader:
```
dataloader = DataLoader(
            train_dataset,
            batch_size=self.args.train_batch_size,
            sampler=train_sampler,
            collate_fn=self.data_collator,
            drop_last=self.args.dataloader_drop_last,
            num_workers=self.args.dataloader_num_workers,
        )
```

now if I type len(dataloader) this is 1, which is wrong, and this needs to be 10. could you assist me please? thanks 


Also, I cannot first convert it to torch format, since huggingface seq2seq_trainer codes process the datasets afterwards during datacollector function to make it optimize for TPUs. 

sorry I misunderstood length of dataset with dataloader, closed. thanks
Shall we change the hashing to encoding to reduce potential replicated cache files?

Hi there. For now, we are using `xxhash` to hash the transformations to fingerprint and we will save a copy of the processed dataset to disk if there is a new hash value. However, there are some transformations that are idempotent or commutative to each other. I think that encoding the transformation chain as the fingerprint may help in those cases, for example, use `base64.urlsafe_b64encode`. In this way, before we want to save a new copy, we can decode the transformation chain and normalize it to prevent omit potential reuse. As the main targets of this project are the really large datasets that cannot be loaded entirely in memory, I believe it would save a lot of time if we can avoid some write.

If you have interest in this, I'd love to help :).

This is an interesting idea !
Do you have ideas about how to approach the decoding and the normalization ?

@lhoestq
I think we first need to save the transformation chain to a list in `self._fingerprint`. Then we can
- decode all the current saved datasets to see if there is already one that is equivalent to the transformation we need now.
- or, calculate all the possible hash value of the current chain for comparison so that we could continue to use hashing.
If we find one, we can adjust the list in `self._fingerprint` to it.

As for the transformation reordering rules, we can just start with some manual rules, like two sort on the same column should merge to one, filter and select can change orders.

And for encoding and decoding, we can just manually specify `sort` is 0, `shuffling` is 2 and create a base-n number or use some general algorithm like `base64.urlsafe_b64encode`.

Because we are not doing lazy evaluation now, we may not be able to normalize the transformation to its minimal form. If we want to support that, we can provde a `Sequential` api and let user input a list or transformation, so that user would not use the intermediate datasets. This would look like tf.data.Dataset.
My new dataset PEC

A new dataset PEC published in EMNLP 2020.

How to resolve these failed checks?

Thanks for adding this one :) 

To fix the check_code_quality, please run `make style` with the latest version of black, isort, flake8
To fix the test_no_encoding_on_file_open, make sure to specify the encoding each time you call `open()` on a text file.
For example : `encoding="utf-8"`
To fix the test_load_dataset_pec , you must add the dummy_data.zip file. It is used to test the dataset script and make sure it runs fine. To add it, please refer to the steps in https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-add-a-dataset



Could you also add a dataset card ? you can find a template here : https://github.com/huggingface/datasets/blob/master/templates/README.md

That would be awesome

> Thanks for adding this one :)
> 
> To fix the check_code_quality, please run `make style` with the latest version of black, isort, flake8
> To fix the test_no_encoding_on_file_open, make sure to specify the encoding each time you call `open()` on a text file.
> For example : `encoding="utf-8"`
> To fix the test_load_dataset_pec , you must add the dummy_data.zip file. It is used to test the dataset script and make sure it runs fine. To add it, please refer to the steps in https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-add-a-dataset

Thank you for the detailed suggestion.

I have added dummy_data but it still failed the DistributedDatasetTest check. My dataset has a central file (containing a python dict) that needs to be accessed by each data example. Is it because the central file cannot be distributed (which would lead to a partial dictionary)?

Specifically, the central file contains a dictionary of speakers with their attributes. Each data example is also associated with a speaker. As of now, I keep the central file and data files separately. If I remove the central file by appending the speaker attributes to each data example, then there would be lots of redundancy because there are lots of duplicate speakers in the data files.

The `DistributedDatasetTest` fail and the changes of this PR are not related, there was just a bug in the CI. You can ignore it

> Really cool thanks !
> 
> Could you make the dummy files smaller ? For example by reducing the size of persona.txt ?
> I also left a comment about the files concatenation. It would be cool to replace that with simple iterations through the different files.
> 
> Then once this is done, you can add a dataset card using the template guide here : https://github.com/huggingface/datasets/blob/master/templates/README_guide.md
> If some fields can't be filled, just leave `[N/A]`

Small change: if you don't have the information for a field, please leave `[More Information Needed]` rather than `[N/A]`

The full information can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#manually-tag-the-dataset-and-write-the-dataset-card)
Add FiNER dataset

Hi,

this PR adds "A Finnish News Corpus for Named Entity Recognition" as new `finer` dataset.

The dataset is described in [this paper](https://arxiv.org/abs/1908.04212). The data is publicly available in [this GitHub](https://github.com/mpsilfve/finer-data).

Notice: they provide two testsets. The additional test dataset taken from Wikipedia is named as "test_wikipedia" split.

> That's really cool thank you !
> 
> Could you also add a dataset card ?
> You can find a template here : https://github.com/huggingface/datasets/blob/master/templates/README.md

The full information for adding a dataset card can be found here :) 
https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#manually-tag-the-dataset-and-write-the-dataset-card


Thanks your suggestions! I've fixed them, and currently working on the dataset card!

@yjernite and @lhoestq I will add the dataset card a bit later in a separate PR if that's ok for you!

Yes I want to re-emphasize if it was not clear that dataset cards are optional for the sprint. 

Only the tags are required for merging a datasets.

Please try to enforce this rule as well @lhoestq and @yjernite 

Yes @stefan-it if you could just add the tags (the yaml part at the top of the dataset card) that'd be perfect :) 

Oh, sorry, will add them now!


Initial README file is now added :) 

the `RemoteDatasetTest ` errors in the CI are fixed on master so it's fine

merging since the CI is fixed on master
Disallow backslash in urls

Following #903 @albertvillanova noticed that there are sometimes bad usage of `os.path.join` in datasets scripts to create URLS. However this should be avoided since it doesn't work on windows.

I'm suggesting a test to make sure we that all the urls don't have backslashes in them in the datasets scripts.
The tests works by adding a callback feature to the MockDownloadManager used to test the dataset scripts. In a download callback I just make sure that the url is valid.

Looks like the test doesn't detect all the problems fixed by #907 , I'll fix that

Ok found why it doesn't detect the problems fixed by #907 . That's because for all those datasets the urls are actually fine (no backslash) on windows, even if it uses `os.path.join`.

This is because of the behavior of `os.path.join` on windows when the first path ends with a slash : 

```python
import os
os.path.join("https://test.com/foo", "bar.txt")
# 'https://test.com/foo\\bar.txt'
os.path.join("https://test.com/foo/", "bar.txt")
# 'https://test.com/foo/bar.txt'
```

However even though the urls are correct, this is definitely bad practice and we should never use `os.path.join` for urls
Fix URL with backslash in Windows

In Windows, `os.path.join` generates URLs containing backslashes, when the first "path" does not end with a slash. 

In general, `os.path.join` should be avoided to generate URLs.

@lhoestq I was indeed working on that... to make another commit on this feature branch...

But as you prefer... nevermind! :)

Ah what do you have in mind for the tests ? I was thinking of adding a check in the MockDownloadManager used for tests based on dummy data. I'm creating a PR right now, I'd be happy to have your opinion

Indeed I was thinking of something similar: monckeypatching the HTTP request...

Therefore, if you agree, I am removing all the rest of `os.path.join`, both from the code and the docs...

If you spot other `os.path.join` for urls in dataset scripts or metrics scripts feel free to fix them.
In the library itself (/src/datasets) it should be fine since there are tests and a windows CI, but if you have doubts of some usage of `os.path.join` somewhere, let me know.

Alright create the test in #905 .
The windows CI is failing for all the datasets that have bad usage of `os.path.join` for urls.
There are of course the ones you fixed in this PR (thanks again !) but I found others as well such as pg19 and blimp.
You can check the full list by looking at the CI failures of the commit 1ce3354

I am merging this one as well as #906 that should fix all of the datasets.
Then I'll rebase #905 which adds the test that checks for bad urls and make sure it' all green now
Addition of Nl2Bash Dataset

## Overview
The NL2Bash data contains over 10,000 instances of linux shell commands and their corresponding natural language descriptions provided by experts, from the Tellina system. The dataset features 100+ commonly used shell utilities.
## Footnotes
The following dataset marks the first ML on source code related Dataset in datasets module. It'll be really useful as a lot of the research direction involves Transformer Based Model.
Thanks.
### Reference Links

> Paper Link = https://arxiv.org/pdf/1802.08979.pdf 
> Github Link = https://github.com/TellinaTool/nl2bash







Hello, thanks. I had a talk with the dataset authors, found out that the data now is obsolete and they'll get a stable version soon. So temporality closing the PR.
 Although I have a question, What should _id_ be in the return statement? Should that be something like a start index (or) the type of split will do? Thanks. 

@reshinthadithyan we should hold off on this for a couple of weeks till NeurIPS concludes. The [NLC2CMD](http://nlc2cmd.us-east.mybluemix.net/) data will be out then; which includes a cleaner version of this NL2Bash data. The older data is sort of obsolete now. 

Ah nvm you already commented 😆
datasets.load_dataset() custom chaching directory bug

Hello,
I'm having issue with loading a dataset with a custom `cache_dir`. Despite specifying the output dir, it is still downloaded to 
 `~/.cache`.

## Environment info
- `datasets` version: 1.1.3
- Platform: Linux-4.19.129-aufs-1-x86_64-with-debian-10.1
- Python version: 3.7.3

## The code I'm running:
```python
import datasets
from pathlib import Path

validation_dataset = datasets.load_dataset("natural_questions", split="validation[:5%]", cache_dir=Path("./data"))  
```

## The output:

* The dataset is downloaded to my home directory's `.cache` 
* A new empty directory named "`natural_questions` is created in the specified directory `.data`
* `tree data` in the shell outputs:
```
data
└── natural_questions
    └── default
        └── 0.0.2
3 directories, 0 files
```

The output:
```
Downloading: 8.61kB [00:00, 5.11MB/s]                                                                                                                                                                              
Downloading: 13.6kB [00:00, 7.89MB/s]                                                                                                                                                                              
Using custom data configuration default                                                                                                                                                                            
Downloading and preparing dataset natural_questions/default (download: 41.97 GiB, generated: 92.95 GiB, post-processed: Unknown size, total: 134.92 GiB) to ./data/natural_questions/default/0.0.2/867dbbaf9137c1b8
3ecb19f5eb80559e1002ea26e702c6b919cfa81a17a8c531...                                                                                                                                                                
Downloading: 100%|██████████████████████████████████████████████████| 13.6k/13.6k [00:00<00:00, 1.51MB/s]                                                                                                          
Downloading:   7%|███▎                                            | 6.70G/97.4G [03:46<1:37:05, 15.6MB/s]
```

## Expected behaviour:
The dataset "Natural Questions" should be downloaded to the directory "./data"


Thanks for reporting ! I'm looking into it.
Adding SQA dataset

As discussed in #880 

Seems like automatic dummy-data generation doesn't work if the builder is a `ArrowBasedBuilder`, do you think you could take a look @lhoestq ?

This dataset seems to have around 1000 configs. Therefore when creating the dummy data we end up with hundreds of MB of dummy data which we don't want to add in the repo.
Let's make this PR on hold for now and find a solution after the sprint of next week

Closing in favor of #1566
Dataset viewer issues

I was looking through the dataset viewer and I like it a lot. Version numbers, citation information, everything's there! I've spotted a few issues/bugs though:

- the URL is still under `nlp`, perhaps an alias for `datasets` can be made
- when I remove a **feature** (and the feature list is empty), I get an error. This is probably expected, but perhaps a better error message can be shown to the user

```bash
IndexError: list index out of range
Traceback:
File "/home/sasha/streamlit/lib/streamlit/ScriptRunner.py", line 322, in _run_script
    exec(code, module.__dict__)
File "/home/sasha/nlp-viewer/run.py", line 316, in <module>
    st.table(style)
File "/home/sasha/streamlit/lib/streamlit/DeltaGenerator.py", line 122, in wrapped_method
    return dg._enqueue_new_element_delta(marshall_element, delta_type, last_index)
File "/home/sasha/streamlit/lib/streamlit/DeltaGenerator.py", line 367, in _enqueue_new_element_delta
    rv = marshall_element(msg.delta.new_element)
File "/home/sasha/streamlit/lib/streamlit/DeltaGenerator.py", line 120, in marshall_element
    return method(dg, element, *args, **kwargs)
File "/home/sasha/streamlit/lib/streamlit/DeltaGenerator.py", line 2944, in table
    data_frame_proto.marshall_data_frame(data, element.table)
File "/home/sasha/streamlit/lib/streamlit/elements/data_frame_proto.py", line 54, in marshall_data_frame
    _marshall_styles(proto_df.style, df, styler)
File "/home/sasha/streamlit/lib/streamlit/elements/data_frame_proto.py", line 73, in _marshall_styles
    translated_style = styler._translate()
File "/home/sasha/.local/share/virtualenvs/lib-ogGKnCK_/lib/python3.7/site-packages/pandas/io/formats/style.py", line 351, in _translate
    * (len(clabels[0]) - len(hidden_columns))
```

- there seems to be **an encoding issue** in the default view, the dataset examples are shown as raw monospace text, without a decent encoding. That makes it hard to read for languages that use a lot of special characters. Take for instance the [cs-en WMT19 set](https://huggingface.co/nlp/viewer/?dataset=wmt19&config=cs-en). This problem goes away when you enable "List view", because then some syntax highlighteris used, and the special characters are coded correctly.


Thanks for reporting !
cc @srush for the empty feature list issue and the encoding issue
cc @julien-c maybe we can update the url and just have a redirection from the old url to the new one ?

Ok, I redirected on our side to a new url. ⚠️ @srush: if you update the Streamlit config too to `/datasets/viewer`, let me know because I'll need to change our nginx config at the same time

9

‏⠀‏‏‏⠀‏‏‏⠀ ‏⠀ 

‏⠀‏‏‏⠀‏‏‏⠀ ‏⠀
Allow several tags sets

Hi !

Currently we have three dataset cards : snli, cnn_dailymail and allocine.
For each one of those datasets a set of tag is defined. The set of tags contains fields like `multilinguality`, `task_ids`, `licenses` etc.

For certain datasets like `glue` for example, there exist several configurations: `sst2`, `mnli` etc. Therefore we should define one set of tags per configuration. However the current format used for tags only supports one set of tags per dataset.

In this PR I propose a simple change in the yaml format used for tags to allow for several sets of tags.

Let me know what you think, especially @julien-c let me know if it's good for you since it's going to be parsed by moon-landing

Closing since we don't need to update the tags of those three datasets (for each one of them there is only one tag set)
add metrec: arabic poetry dataset


@lhoestq removed prints and added the dataset card. 

@lhoestq, I want to add other datasets as well. I am not sure if it is possible to do so with the same branch. 

Hi @zaidalyafeai, really excited to get more Arabic coverage in the lib, thanks for your contribution!

Couple of last comments:
- this PR seems to modify some files that are unrelated to your dataset. Could you rebase from master? It should take care of that.
- The dataset card is a good start! Can you describe the task in a few words and add more information in the Data Structure part, including listing and describing the fields? Also, if you don't know how to fill out a paragraph, or if you have some information but think more would be beneficial, please leave `[More Information Needed]` instead of `[N/A]`

> Hi @zaidalyafeai, really excited to get more Arabic coverage in the lib, thanks for your contribution!
> 
> Couple of last comments:
> 
> * this PR seems to modify some files that are unrelated to your dataset. Could you rebase from master? It should take care of that.
> * The dataset card is a good start! Can you describe the task in a few words and add more information in the Data Structure part, including listing and describing the fields? Also, if you don't know how to fill out a paragraph, or if you have some information but think more would be beneficial, please leave `[More Information Needed]` instead of `[N/A]`

I have no idea how some other files changed. I tried to rebase and push but this created some errors. I had to run the command 
`git push -u --force origin add-metrec-dataset` which might cause some problems. 

Feel free to create another branch/another PR without all the other changes

@yjernite can you explain which other files are changed because of the PR ? https://github.com/huggingface/datasets/pull/893/files only shows files related to the dataset. 

Right ! github is nice with us today :)

Looks like this one is ready to merge, thanks @zaidalyafeai !

@lhoestq thanks for the merge. I am not a GitHub geek. I already have another dataset to add. I'm not sure how to add another given my forked repo. Do I  follow the same steps with a different checkout name ?

If you've followed the instructions in here : https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#start-by-preparing-your-environment

(especially point 2. and the command `git remote add upstream ....`)

Then you can try
```
git checkout master
git fetch upstream
git rebase upstream/master
git checkout -b add-<my-new-dataset-name>
```
Add a few datasets of reference in the documentation

I started making a small list of various datasets of reference in the documentation.
Since many datasets share a lot in common I think it's good to have a list of datasets scripts to get some inspiration from.

Let me know what you think, and if you have ideas of other datasets that we may add to this list, please let me know.

Looks good to me. Do we also support TSV in this helper (explain if it should be text or CSV) and in the dummy-data creator?

snli is basically based on tsv files (but named as .txt) and it is in the list of datasets of reference.
The dummy data creator supports tsv

merging this one.
If you think of other datasets of reference to add we can still add them later
Optional per-dataset default config name

This PR adds a `DEFAULT_CONFIG_NAME` class attribute to `DatasetBuilder`. This allows a dataset to have a specified default config name when a dataset has more than one config but the user does not specify it. For example, after defining `DEFAULT_CONFIG_NAME = "combined"` in PolyglotNER, a user can now do the following:

```python
ds = load_dataset("polyglot_ner")
```
which is equivalent to,
```python
ds = load_dataset("polyglot_ner", "combined")
```
In effect (for this particular dataset configuration), this means that if the user doesn't specify a language, they are given the combined dataset including all languages.

Since it doesn't always make sense to have a default config, this feature is opt-in. If `DEFAULT_CONFIG_NAME` is not defined and a user does not pass a config for a dataset with multiple configs available, a ValueError is raised like usual.

Let me know what you think about this approach @lhoestq @thomwolf and I'll add some documentation and define a default for some of our existing datasets.

I like the idea ! And the approach is right imo

Note that by changing this we will have to add a way for users to get the config lists of a dataset. In the current user workflow, the user could see the list of the config when the missing config error is raised but now it won't be the case because of the default config.

Maybe let's add a test in the test_builder.py test script ?

@lhoestq Okay great, I added a test as well as two new inspect functions: `get_dataset_config_names` and `get_dataset_infos` (the latter is something I've been wanting anyway). As a quick hack, you can also just pass a random config name (e.g. an empty string) to `load_dataset` to get the config names in the error msg as before. Also added a couple paragraphs to the adding new datasets doc.

I'll send a separate PR incorporating this in existing datasets so we can get this merged before our sprint on Monday.

Any ideas on the failing tests? I'm having trouble making sense of it. **Edit**: nvm, it was master.
Nested lists are zipped unexpectedly

I might misunderstand something, but I expect that if I define:
```python
"top": datasets.features.Sequence({
  "middle": datasets.features.Sequence({
    "bottom": datasets.Value("int32")
  })
})
```

And I then create an example:
```python
yield 1, {
  "top": [{
    "middle": [
      {"bottom": 1},
      {"bottom": 2}
    ]
  }]
}
```

I then load my dataset:
```python
train = load_dataset("my dataset")["train"]
```

and expect to be able to access `data[0]["top"][0]["middle"][0]`.

That is not the case. Here is `data[0]` as JSON:

```json
{"top": {"middle": [{"bottom": [1, 2]}]}}
```

Clearly different than the thing I inputted.
```json
{"top": [{"middle": [{"bottom": 1},{"bottom": 2}]}]}
```

Yes following the Tensorflow Datasets convention, objects with type `Sequence of a Dict` are actually stored as a `dictionary of lists`.
See the [documentation](https://huggingface.co/docs/datasets/features.html?highlight=features) for more details

Thanks.
This is a bit (very) confusing, but I guess if its intended, I'll just work with it as if its how my data was originally structured :)
Fix wikipedia custom config

It should be possible to use the wikipedia dataset with any `language` and `date`.
However it was not working as noticed in #784 . Indeed the custom wikipedia configurations were not enabled for some reason.

I fixed that and was able to run 
```python
from datasets import load_dataset
load_dataset("./datasets/wikipedia", language="zh", date="20201120", beam_runner='DirectRunner')
```

cc @stvhuang @SamuelCahyawijaya

Fix #784

I think this issue is still not resolve yet. Please check my comment in the following issue, thanks.
[#577](https://github.com/huggingface/datasets/issues/577#issuecomment-868122769)
Very slow cold-start

Hi,
I expect when importing `datasets` that nothing major happens in the background, and so the import should be insignificant.
When I load a metric, or a dataset, its fine that it takes time.

The following ranges from 3 to 9 seconds:
```
python -m timeit -n 1 -r 1 'from datasets import load_dataset'
```

edit:
sorry for the mis-tag, not sure how I added it.

Good point!

Yes indeed. We can probably improve that by using lazy imports

#1690 added fast start-up of the library
Downloading/caching only a part of a datasets' dataset.

Hi,
I want to use the validation data *only* (of natural question).
I don't want to have the whole dataset cached in my machine, just the dev set.
Is this possible? I can't find a way to do it in the docs.

Thank you,
Sapir

Not at the moment but we could likely support this feature.

?

I think it would be a very helpful feature, because sometimes one only wants to evaluate models on the dev set, and the whole training data may be many times bigger.
This makes the task impossible with limited memory resources.
DataLoader(datasets) become more and more slowly within iterations

Hello, when I for loop my dataloader, the loading speed is becoming more and more slowly!
```
dataset = load_from_disk(dataset_path)  # around 21,000,000 lines

lineloader = tqdm(DataLoader(dataset, batch_size=1))
for idx, line in enumerate(lineloader):
     # do some thing for each line
```
In the begining, the loading speed is around 2000it/s, but after 1 minutes later, the speed is much slower, just around 800it/s.

And when I set `num_workers=4` in DataLoader, the loading speed is much lower, just 130it/s.

Could you please help me with this problem?
Thanks a lot!

Hi ! Thanks for reporting.
Do you have the same slowdown when you iterate through the raw dataset object as well ? (no dataloader)
It would be nice to know whether it comes from the dataloader or not

> Hi ! Thanks for reporting.
> Do you have the same slowdown when you iterate through the raw dataset object as well ? (no dataloader)
> It would be nice to know whether it comes from the dataloader or not

I did not iter data from raw dataset, maybe I will test later. Now I iter all files directly from `open(file)`,  around 20000it/s.

> > Hi ! Thanks for reporting.
> > Do you have the same slowdown when you iterate through the raw dataset object as well ? (no dataloader)
> > It would be nice to know whether it comes from the dataloader or not
> 
> I did not iter data from raw dataset, maybe I will test later. Now I iter all files directly from `open(file)`, around 20000it/s.

Hi ! I meet the same problem, how did you solve it?
imdb dataset cannot be loaded 

Hi
I am trying to load the imdb train dataset

`dataset = datasets.load_dataset("imdb", split="train")`

getting following errors, thanks for your help 
```
Traceback (most recent call last):        
  File "<stdin>", line 1, in <module>
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/load.py", line 611, in load_dataset
    ignore_verifications=ignore_verifications,
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/builder.py", line 476, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/builder.py", line 558, in _download_and_prepare
    verify_splits(self.info.splits, split_dict)
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/utils/info_utils.py", line 73, in verify_splits
    raise NonMatchingSplitsSizesError(str(bad_splits))
datasets.utils.info_utils.NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='test', num_bytes=32660064, num_examples=25000, dataset_name='imdb'), 'recorded': SplitInfo(name='test', num_bytes=26476338, num_examples=20316, dataset_name='imdb')}, {'expected': SplitInfo(name='train', num_bytes=33442202, num_examples=25000, dataset_name='imdb'), 'recorded': SplitInfo(name='train', num_bytes=0, num_examples=0, dataset_name='imdb')}, {'expected': SplitInfo(name='unsupervised', num_bytes=67125548, num_examples=50000, dataset_name='imdb'), 'recorded': SplitInfo(name='unsupervised', num_bytes=0, num_examples=0, dataset_name='imdb')}]
>>> dataset = datasets.load_dataset("imdb", split="train")

```


It looks like there was an issue while building the imdb dataset.
Could you provide more information about your OS and the version of python and `datasets` ?

Also could you try again with 
```python
dataset = datasets.load_dataset("imdb", split="train", download_mode="force_redownload")
```
to make sure it's not a corrupted file issue ?

I was using version 1.1.2 and this resolved with version 1.1.3, thanks. 

Hello,
I have the same pb with 1.8.0

Hi ! I just tried in 1.8.0 and it worked fine. Can you try again ? Maybe the dataset host had some issues that are fixed now

Hello,
It works fine now :) !
Thanks !

Ran into the same issue on a different dataset. I workedaround this by passing 

        verification_mode='no_checks',

to load_dataset method. Ref: https://github.com/huggingface/datasets/blob/871eabc7b23c27d677bc06ae2cc1ec3a2a04b10f/src/datasets/builder.py#L1141

Note that this is a hack before the root cause is solved.