NyssaW/github-issues huggingface dataset:
Switchboard Dialog Act Corpus added under `datasets/swda`

Switchboard Dialog Act Corpus

Intro:
The Switchboard Dialog Act Corpus (SwDA) extends the Switchboard-1 Telephone Speech Corpus, Release 2,
with turn/utterance-level dialog-act tags. The tags summarize syntactic, semantic, and pragmatic information
about the associated turn. The SwDA project was undertaken at UC Boulder in the late 1990s.

Details:
[homepage](http://compprag.christopherpotts.net/swda.html)
[repo](https://github.com/NathanDuran/Switchboard-Corpus/raw/master/swda_data/)

I believe this is an important dataset to have since there is no dataset related to dialogue act added.

I didn't find any formatting for pull request. I hope all this information is enough.

For any support please contact me. 

Comments:
@lhoestq Thank you for your detailed comments! I fixed everything you suggested.

Please let me know if I'm missing anything else.

It looks like the Transcript and Utterance objects are missing, maybe we can mention it in the README ? Or just add them ? @gmihaila @bhavitvyamalik 

Hi @lhoestq,
I'm working on this to add the full dataset

> It looks like the Transcript and Utterance objects are missing, maybe we can mention it in the README ? Or just add them ? @gmihaila @bhavitvyamalik

@lhoestq  Any info on how to add them?

@gmihaila, instead of using the current repo you should look into [this](https://github.com/cgpotts/swda). You can use the `csv` files uploaded in this repo (`swda.zip`) to access other fields and include them in this dataset. It has one dependency too, `swda.py`, you can download that separately and include it in your dataset's folder to be imported while reading the `csv` files.

Almost all the attributes of `Transcript` and `Utterance` objects are of the type str, int, or list. As far as `trees` attribute is concerned in utterance object you can simply parse it as string and user can maybe later convert it to nltk.tree object

@bhavitvyamalik Thank you for the clarification! 

I didn't use [that](https://github.com/cgpotts/swda) because it doesn't have the splits. I think in combination with [what I used](https://github.com/NathanDuran/Switchboard-Corpus) would help.

Let me know if I can help! I can make those changes if you don't have the time.

I'm a bit busy for the next 2 weeks. I'll be able to complete it by end of January only. Maybe you can start with it and I'll help you?
Also, I looked into the official train/val/test splits and not all the files are there in the repo I used so I think either we'll have to skip them or put all of that into just train

Yes, I can start working on it and ask you to do a code review.

Yes, not all files are there. I'll try to find papers that have the correct and full splits, if not, I'll do like you suggested.

Thank you again for your help @bhavitvyamalik !
Switchboard Dialog Act Corpus added under `datasets/swda`

Pleased to announced that I added my first dataset **Switchboard Dialog Act Corpus**.


I think this is an important datasets to be added since it is the only one related to dialogue act classification. 

Hope the pull request is ok. Wasn't able to see any special formatting for the pull request form.


The Switchboard Dialog Act Corpus (SwDA) extends the Switchboard-1 Telephone Speech Corpus, Release 2,
with turn/utterance-level dialog-act tags. The tags summarize syntactic, semantic, and pragmatic information
about the associated turn. The SwDA project was undertaken at UC Boulder in the late 1990s.


[webpage](http://compprag.christopherpotts.net/swda.html)

[repo](https://github.com/NathanDuran/Switchboard-Corpus/raw/master/swda_data/)

Please contact me for any support!

All tests passed and followed all steps in the contribution guide!



Comments:
Need to fix code formatting.
new version of Ted Talks IWSLT (WIT3)

In the previous iteration #1608 I had used language pairs. Which created 21,582 configs (109*108) !!! 

Now, TED talks in _each language_ is a separate config. So it's more cleaner with _just 109 configs_  (one for each language). Dummy files were created manually. 

Locally  I was able to clear the `python datasets-cli test datasets/......` . Which created the `dataset_info.json`  file . The test for the dummy files was also cleared. However couldn't figure out how to specify the local data folder for the real dataset


**Note: that this requires manual download of the dataset.** 
**Note2: The high number of _Files changed (112)_ is because of the large number of dummy files/configs!**

Comments:
> Nice thank you ! Actually as it is a translation dataset we should probably have one configuration = one language pair no ?
> 
> Could you use the same trick for this dataset ?

I was looking for this input, infact I had written a long post on the Slack channel,...(_but unfortunately due to the holidays didn;t get a respones_). Initially I had tried with language pairs and then with specific language configs. 

I'll have a look at the `opus-gnomes` dataset


Oh sorry I must have missed your message then :/
I was off a few days during the holidays

Hopefully this trick can enable the use of any language pair (+ year ?) combination and also simplify a lot the dummy data creation since it will only require a few configs.

Updated it as per the comments. But couldn't figure out why the dummy tests are failing >> 
```
$RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_real_dataset_ted_talks_iwslt
.....
....
tests/test_dataset_common.py:198: in check_load_dataset
    self.parent.assertTrue(len(dataset[split]) > 0)
E   AssertionError: False is not true
```
Unable to Download Hindi Wikipedia Dataset

I used the Dataset Library in Python to load the wikipedia dataset with the Hindi Config 20200501.hi along with something called beam_runner='DirectRunner' and it keeps giving me the error that the file is not found. I have attached the screenshot of the error and the code both. Please help me to understand how to resolve this issue.

![Code](https://user-images.githubusercontent.com/30871963/103437466-1f3a3300-4c4e-11eb-9d54-fc9601abfeec.png)

![Error](https://user-images.githubusercontent.com/30871963/103437407-7ee40e80-4c4d-11eb-8151-a86eb664e6be.png)


Comments:
Currently this dataset is only available when the library is installed from source since it was added after the last release.

We pin the dataset version with the library version so that people can have a reproducible dataset and processing when pinning the library.

We'll see if we can provide access to newer datasets with a warning that they are newer than your library version, that would help in cases like yours.

So for now, should i try and install the library from source and then try out the same piece of code? Will it work then, considering both the versions will match then?

Yes

Hey, so i tried installing the library from source using the commands : **git clone https://github.com/huggingface/datasets**,  **cd datasets** and then **pip3 install -e .**. But i still am facing the same error that file is not found. Please advise.

The Datasets library version now is 1.1.3 by installing from source as compared to the earlier 1.0.3 that i had loaded using pip command but I am still getting same error

![Error](https://user-images.githubusercontent.com/30871963/103479005-69f3b080-4df0-11eb-83ae-58d7bb56a90e.png)


Looks like the wikipedia dump for hindi at the date of 05/05/2020 is not available anymore.
You can try to load a more recent version of wikipedia
```python
from datasets import load_dataset

d = load_dataset("wikipedia", language="hi", date="20210101", split="train", beam_runner="DirectRunner")
```

Okay, thank you so much
wiki_dpr pre-processing performance

I've been working with wiki_dpr and noticed that the dataset processing is seriously impaired in performance [1]. It takes about 12h to process the entire dataset. Most of this time is simply loading and processing the data, but the actual indexing is also quite slow (3h).

I won't repeat the concerns around multiprocessing as they are addressed in other issues (#786), but this is the first obvious thing to do. Using cython to speed up the text manipulation may be also help. Loading and processing a dataset of this size in under 15 minutes does not seem unreasonable on a modern multi-core machine. I have hit such targets myself on similar tasks. Would love to see this improve.

The other issue is that it takes 3h to construct the FAISS index. If only we could use GPUs with HNSW, but we can't. My sharded GPU indexing code can build an IVF + PQ index in 10 minutes on 20 million vectors. Still, 3h seems slow even for the CPU.

It looks like HF is adding only 1000 vectors at a time by default [2], whereas the faiss benchmarks adds 1 million vectors at a time (effectively) [3]. It's possible the runtime could be reduced with a larger batch. Also, it looks like project dependencies ultimately use OpenBLAS, but this is known to have issues when combined with OpenMP, which HNSW does [3]. A workaround is to set the environment variable `OMP_WAIT_POLICY=PASSIVE` via `os.environ` or similar.

References:
[1] https://github.com/huggingface/datasets/blob/master/datasets/wiki_dpr/wiki_dpr.py
[2] https://github.com/huggingface/datasets/blob/master/src/datasets/search.py
[3] https://github.com/facebookresearch/faiss/blob/master/benchs/bench_hnsw.py
[4] https://github.com/facebookresearch/faiss/issues/422

Comments:
Hi ! And thanks for the tips :) 

Indeed currently `wiki_dpr` takes some time to be processed.
Multiprocessing for dataset generation is definitely going to speed up things.

Regarding the index note that for the default configurations, the index is downloaded instead of being built, which avoid spending time on constructing the index. However in other cases it would be awesome to make the construction faster.

Any contribution that can help things faster are welcome. In particular in you have some code that can build a wiki_dpr IVF PQ index in a sharded GPU setup and would like to share it, we can add it to an `examples` folder. In particular since faiss is becoming the library of reference for dataset indexing for tasks like Open Domain Question Answering.



I'd be happy to contribute something when I get the time, probably adding multiprocessing and / or cython support to wiki_dpr. I've written cythonized apache beam code before as well.

For sharded index building, I used the FAISS example code for indexing 1 billion vectors as a start. I'm sure you're aware that the documentation isn't great, but the source code is fairly easy to follow.

Nice thanks ! That would be awesome to make its construction faster :)
wiki_dpr dataset pre-processesing performance

I've been working with wiki_dpr and noticed that the dataset processing is seriously impaired in performance [1]. It takes about 12h to process the entire dataset. Most of this time is simply loading and processing the data, but the actual indexing is also quite slow (3h).

I won't repeat the concerns around multiprocessing as they are addressed in other issues (#786), but this is the first obvious thing to do. Using cython to speed up the text manipulation may be also help. Loading and processing a dataset of this size in under 15 minutes does not seem unreasonable on a modern multi-core machine. I have hit such targets myself on similar tasks. Would love to see this improve.

The other issue is that it takes 3h to construct the FAISS index. If only we could use GPUs with HNSW, but we can't. My sharded GPU indexing code can build an IVF + PQ index in 10 minutes on 20 million vectors. Still, 3h seems slow even for the CPU.

It looks like HF is adding only 1000 vectors at a time by default [2], whereas the faiss benchmarks adds 1 million vectors at a time (effectively) [3]. It's possible the runtime could be reduced with a larger batch. Also, it looks like project dependencies ultimately use OpenBLAS, but this is known to have issues when combined with OpenMP, which HNSW does [3]. A workaround is to set the environment variable `OMP_WAIT_POLICY=PASSIVE` via `os.environ` or similar.

References:
[1] https://github.com/huggingface/datasets/blob/master/datasets/wiki_dpr/wiki_dpr.py
[2] https://github.com/huggingface/datasets/blob/master/src/datasets/search.py
[3] https://github.com/facebookresearch/faiss/blob/master/benchs/bench_hnsw.py
[4] https://github.com/facebookresearch/faiss/issues/422

Comments:
Sorry, double posted.
Fix NER metric example in Overview notebook

Fix errors in `NER metric example` section in `Overview.ipynb`.

```
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-37-ee559b166e25> in <module>()
----> 1 ner_metric = load_metric('seqeval')
      2 references = [['O', 'O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
      3 predictions =  [['O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
      4 ner_metric.compute(predictions, references)


/usr/local/lib/python3.6/dist-packages/datasets/load.py in prepare_module(path, script_version, download_config, download_mode, dataset, force_local_path, **download_kwargs)
    340     if needs_to_be_installed:
    341         raise ImportError(
--> 342             f"To be able to use this {module_type}, you need to install the following dependencies"
    343             f"{[lib_name for lib_name, lib_path in needs_to_be_installed]} using 'pip install "
    344             f"{' '.join([lib_path for lib_name, lib_path in needs_to_be_installed])}' for instance'"

ImportError: To be able to use this metric, you need to install the following dependencies['seqeval'] using 'pip install seqeval' for instance'
```

```
ValueError                                Traceback (most recent call last)
<ipython-input-39-ee559b166e25> in <module>()
      2 references = [['O', 'O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
      3 predictions =  [['O', 'O', 'B-MISC', 'I-MISC', 'I-MISC', 'I-MISC', 'O'], ['B-PER', 'I-PER', 'O']]
----> 4 ner_metric.compute(predictions, references)

/usr/local/lib/python3.6/dist-packages/datasets/metric.py in compute(self, *args, **kwargs)
    378         """
    379         if args:
--> 380             raise ValueError("Please call `compute` using keyword arguments.")
    381 
    382         predictions = kwargs.pop("predictions", None)

ValueError: Please call `compute` using keyword arguments.
```

Comments:
update saving and loading methods for faiss index so to accept path l…

- Update saving and loading methods for faiss index so to accept path like objects from pathlib

The current code only supports using a string type to save and load a faiss index. This change makes it possible to use a string type OR a Path from [pathlib](https://docs.python.org/3/library/pathlib.html). The codes becomes a more intuitive this way I think.

Comments:
Seems ok for me, what do you think @lhoestq ?
Arrow file is too large when saving vector data

I computed the sentence embedding of each sentence of bookcorpus data using bert base and saved them to disk. I used 20M sentences and the obtained arrow file is about 59GB while the original text file is only about 1.3GB. Are there any ways to reduce the size of the arrow file?

Comments:
Hi !
The arrow file size is due to the embeddings. Indeed if they're stored as float32 then the total size of the embeddings is

20 000 000 vectors * 768 dimensions * 4 bytes per dimension ~= 60GB

If you want to reduce the size you can consider using quantization for example, or maybe using dimension reduction techniques.


Thanks for your reply @lhoestq.
I want to save original embedding for these sentences for subsequent calculations. So does arrow have a way to save in a compressed format to reduce the size of the file?

Arrow doesn't have compression since it is designed to have no serialization overhead

I see. Thank you.
NarrativeQA fails to load with `load_dataset`

When loading the NarrativeQA dataset with `load_dataset('narrativeqa')` as given in the documentation [here](https://huggingface.co/datasets/narrativeqa), I receive a cascade of exceptions, ending with

    FileNotFoundError: Couldn't find file locally at narrativeqa/narrativeqa.py, or remotely at 
        https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/narrativeqa/narrativeqa.py or 
        https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/narrativeqa/narrativeqa.py

Workaround: manually copy the `narrativeqa.py` builder into my local directory with 

    curl https://raw.githubusercontent.com/huggingface/datasets/master/datasets/narrativeqa/narrativeqa.py -o narrativeqa.py

and load the dataset as `load_dataset('narrativeqa.py')` everything works fine. I'm on datasets v1.1.3 using Python 3.6.10.

Comments:
Hi @eric-mitchell,
I think the issue might be that this dataset was added during the community sprint and has not been released yet. It will be available with the v2 of `datasets`.
For now, you should be able to load the datasets after installing the latest (master) version of `datasets` using pip:
`pip install git+https://github.com/huggingface/datasets.git@master`

@bhavitvyamalik Great, thanks for this! Confirmed that the problem is resolved on master at [cbbda53](https://github.com/huggingface/datasets/commit/cbbda53ac1520b01f0f67ed6017003936c41ec59).

Update: HuggingFace did an intermediate release yesterday just before the v2.0.

To load it you can just update `datasets`

`pip install --upgrade datasets`
HoVeR dataset fails to load

Hi! I'm getting an error when trying to load **HoVeR** dataset. Another one (**SQuAD**) does work for me. I'm using the latest (1.1.3) version of the library.

Steps to reproduce the error:

```python
>>> from datasets import load_dataset
>>> dataset = load_dataset("hover")
Traceback (most recent call last):
  File "/Users/urikz/anaconda/envs/mentionmemory/lib/python3.7/site-packages/datasets/load.py", line 267, in prepare_module
    local_path = cached_path(file_path, download_config=download_config)
  File "/Users/urikz/anaconda/envs/mentionmemory/lib/python3.7/site-packages/datasets/utils/file_utils.py", line 308, in cached_path
    use_etag=download_config.use_etag,
  File "/Users/urikz/anaconda/envs/mentionmemory/lib/python3.7/site-packages/datasets/utils/file_utils.py", line 486, in get_from_cache
    raise FileNotFoundError("Couldn't find file at {}".format(url))
FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/hover/hover.py

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/urikz/anaconda/envs/mentionmemory/lib/python3.7/site-packages/datasets/load.py", line 278, in prepare_module
    local_path = cached_path(file_path, download_config=download_config)
  File "/Users/urikz/anaconda/envs/mentionmemory/lib/python3.7/site-packages/datasets/utils/file_utils.py", line 308, in cached_path
    use_etag=download_config.use_etag,
  File "/Users/urikz/anaconda/envs/mentionmemory/lib/python3.7/site-packages/datasets/utils/file_utils.py", line 486, in get_from_cache
    raise FileNotFoundError("Couldn't find file at {}".format(url))
FileNotFoundError: Couldn't find file at https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/hover/hover.py

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/urikz/anaconda/envs/mentionmemory/lib/python3.7/site-packages/datasets/load.py", line 589, in load_dataset
    path, script_version=script_version, download_config=download_config, download_mode=download_mode, dataset=True
  File "/Users/urikz/anaconda/envs/mentionmemory/lib/python3.7/site-packages/datasets/load.py", line 282, in prepare_module
    combined_path, github_file_path, file_path
FileNotFoundError: Couldn't find file locally at hover/hover.py, or remotely at https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/hover/hover.py or https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/hover/hover.py
```

Comments:
Hover was added recently, that's why it wasn't available yet.

To load it you can just update `datasets`
```
pip install --upgrade datasets
```

and then you can load `hover` with

```python
from datasets import load_dataset

dataset = load_dataset("hover")
```
Dataset social_bias_frames 404

```
>>> from datasets import load_dataset
>>> dataset = load_dataset("social_bias_frames")
...
Downloading and preparing dataset social_bias_frames/default
...
~/.pyenv/versions/3.7.6/lib/python3.7/site-packages/datasets/utils/file_utils.py in get_from_cache(url, cache_dir, force_download, proxies, etag_timeout, resume_download, user_agent, local_files_only, use_etag)
    484             )
    485         elif response is not None and response.status_code == 404:
--> 486             raise FileNotFoundError("Couldn't find file at {}".format(url))
    487         raise ConnectionError("Couldn't reach {}".format(url))
    488 

FileNotFoundError: Couldn't find file at https://homes.cs.washington.edu/~msap/social-bias-frames/SocialBiasFrames_v2.tgz
```
[Here](https://homes.cs.washington.edu/~msap/social-bias-frames/) we find button `Download data` with the correct URL for the data: https://homes.cs.washington.edu/~msap/social-bias-frames/SBIC.v2.tgz

Comments:
I see, master is already fixed in https://github.com/huggingface/datasets/commit/9e058f098a0919efd03a136b9b9c3dec5076f626
Fix "'BertTokenizerFast' object has no attribute 'max_len'"

Tensorflow 2.3.0 gives:
 FutureWarning: The `max_len` attribute has been deprecated and will be removed in a future version, use `model_max_length` instead.

Tensorflow 2.4.0 gives:
AttributeError 'BertTokenizerFast' object has no attribute 'max_len'

Comments:
bug with sst2 in glue 

Hi
I am getting very low accuracy on SST2 I investigate this and observe that for this dataset sentences are tokenized, while this is correct for the other datasets in GLUE, please see below.
Is there any alternatives I could get untokenized sentences? I am unfortunately under time pressure to report some results on this dataset. thank you for your help. @lhoestq 
 
```
>>> a =  datasets.load_dataset('glue', 'sst2', split="validation", script_version="master")
Reusing dataset glue (/julia/datasets/glue/sst2/1.0.0/7c99657241149a24692c402a5c3f34d4c9f1df5ac2e4c3759fadea38f6cb29c4)
>>> a[:10]
{'idx': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], 'label': [1, 0, 1, 1, 0, 1, 0, 0, 1, 0], 'sentence': ["it 's a charming and often affecting journey . ", 'unflinchingly bleak and desperate ', 'allows us to hope that nolan is poised to embark a major career as a commercial yet inventive filmmaker . ', "the acting , costumes , music , cinematography and sound are all astounding given the production 's austere locales . ", "it 's slow -- very , very slow . ", 'although laced with humor and a few fanciful touches , the film is a refreshingly serious look at young women . ', 'a sometimes tedious film . ', "or doing last year 's taxes with your ex-wife . ", "you do n't have to know about music to appreciate the film 's easygoing blend of comedy and romance . ", "in exactly 89 minutes , most of which passed as slowly as if i 'd been sitting naked on an igloo , formula 51 sank from quirky to jerky to utter turkey . "]}

```

Comments:
Maybe you can use nltk's treebank detokenizer ?
```python
from nltk.tokenize.treebank import TreebankWordDetokenizer

TreebankWordDetokenizer().detokenize("it 's a charming and often affecting journey . ".split())
# "it's a charming and often affecting journey."
```

I am looking for alternative file URL here instead of adding extra processing code: https://github.com/huggingface/datasets/blob/171f2bba9dd8b92006b13cf076a5bf31d67d3e69/datasets/glue/glue.py#L174

I don't know if there exists a detokenized version somewhere. Even the version on kaggle is tokenized
Added `pn_summary` dataset

#1635 

You did a great job with the fluent procedure regarding adding a dataset. I took the chance to add the dataset on my own. Thank you for your awesome job, and I hope this dataset found the researchers happy, specifically those interested in Persian Language (Farsi)!

Comments:
As always, I got stuck in the correct order of imports 😅
@lhoestq, It's finished!

@lhoestq, It's done! Is there anything else that needs changes?
Persian Abstractive/Extractive Text Summarization

Assembling datasets tailored to different tasks and languages is a precious target. This would be great to have this dataset included.

## Adding a Dataset
- **Name:** *pn-summary*
- **Description:** *A well-structured summarization dataset for the Persian language consists of 93,207 records. It is prepared for Abstractive/Extractive tasks (like cnn_dailymail for English). It can also be used in other scopes like Text Generation, Title Generation, and News Category Classification.*
- **Paper:** *https://arxiv.org/abs/2012.11204*
- **Data:** *https://github.com/hooshvare/pn-summary/#download*
- **Motivation:** *It is the first Persian abstractive/extractive Text summarization dataset (like cnn_dailymail for English)!*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Comments:
Inspecting datasets per category

Hi
Is there a way I could get all NLI datasets/all QA datasets to get some understanding of available datasets per category? this is hard for me to inspect the datasets one by one in the webpage, thanks for the suggestions @lhoestq 

Comments:
That's interesting, can you tell me what you think would be useful to access to inspect a dataset?

You can filter them in the hub with the search by the way: https://huggingface.co/datasets have you seen it?

Hi @thomwolf 
thank you, I was not aware of this, I was looking into the data viewer linked into readme page. 

This is exactly what I was looking for, but  this does not work currently, please see the attached 
I am selecting to see all nli datasets in english and it retrieves none. thanks

![5tarDHn9CP6ngeM](https://user-images.githubusercontent.com/53898419/103107612-1509aa80-4638-11eb-85b5-0c995a189969.png)





I see 4 results for NLI in English but indeed some are not tagged yet and missing (GLUE), we will focus on that in January (cc @yjernite): https://huggingface.co/datasets?filter=task_ids:natural-language-inference,languages:en

Hi! You can use `huggingface_hub`'s `list_datasets` for that now:
```python
import huggingface_hub # pip install huggingface_hub
huggingface_hub.list_datasets(filter="task_categories:question-answering")
# or
huggingface_hub.list_datasets(filter=("task_categories:natural-language-inference", "languages:"en"))
```
social_i_qa wrong format of labels

Hi,
there is extra "\n" in labels of social_i_qa datasets, no big deal, but I was wondering if you could remove it to make it consistent.
so label is 'label': '1\n', not '1'
thanks

```
>>> import datasets 
>>> from datasets import load_dataset
>>> dataset = load_dataset(
...    'social_i_qa')
cahce dir  /julia/cache/datasets
Downloading: 4.72kB [00:00, 3.52MB/s]                                                                                                  
cahce dir /julia/cache/datasets
Downloading: 2.19kB [00:00, 1.81MB/s]                                                                                                  
Using custom data configuration default
Reusing dataset social_i_qa (/julia/datasets/social_i_qa/default/0.1.0/4a4190cc2d2482d43416c2167c0c5dccdd769d4482e84893614bd069e5c3ba06)
>>> dataset['train'][0]
{'answerA': 'like attending', 'answerB': 'like staying home', 'answerC': 'a good friend to have', 'context': 'Cameron decided to have a barbecue and gathered her friends together.', 'label': '1\n', 'question': 'How would Others feel as a result?'}

```



Comments:
@lhoestq, should I raise a PR for this? Just a minor change while reading labels text file

Sure feel free to open a PR thanks !
SICK dataset 

Hi, this would be great to have this dataset included. I might be missing something, but I could not find it in the list of already included datasets. Thank you. 

## Adding a Dataset
- **Name:** SICK
- **Description:**  SICK consists of about 10,000 English sentence pairs that include many examples of the lexical, syntactic, and semantic phenomena. 
- **Paper:** https://www.aclweb.org/anthology/L14-1314/
- **Data:** http://marcobaroni.org/composes/sick.html
- **Motivation:** This dataset is well-known in the NLP community used for recognizing entailment between sentences.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Comments:
Adding UKP Argument Aspect Similarity Corpus

Hi, this would be great to have this dataset included.

## Adding a Dataset
- **Name:** UKP Argument Aspect Similarity Corpus
- **Description:** The UKP Argument Aspect Similarity Corpus (UKP ASPECT) includes 3,595 sentence pairs over 28 controversial topics. Each sentence pair was annotated via crowdsourcing as either “high similarity”, “some similarity”, “no similarity” or “not related” with respect to the topic.
- **Paper:** https://www.aclweb.org/anthology/P19-1054/
- **Data:** https://tudatalib.ulb.tu-darmstadt.de/handle/tudatalib/1998
- **Motivation:** this is one of the datasets currently used frequently in recent adapter papers like https://arxiv.org/pdf/2005.00247.pdf 

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

Thank you

Comments:
Adding a link to the guide on adding a dataset if someone want to give it a try: https://github.com/huggingface/datasets#add-a-new-dataset-to-the-hub

we should add this guide to the issue template @lhoestq 

thanks @thomwolf , this is added now. The template is correct, sorry my mistake not to include it. 

Available here: https://huggingface.co/datasets/UKPLab/UKP_ASPECT
`Dataset.map` disable progress bar

I can't find anything to turn off the `tqdm` progress bars while running a preprocessing function using `Dataset.map`. I want to do akin to `disable_tqdm=True` in the case of `transformers`. Is there something like that?

Comments:
Progress bar can be disabled like this:
```python
from datasets.utils.logging import set_verbosity_error
set_verbosity_error()
```

There is this line in `Dataset.map`:
```python
not_verbose = bool(logger.getEffectiveLevel() > WARNING)
```

So any logging level higher than `WARNING` turns off the progress bar.

From the linked issues above, an up-to-date solution is:

```python
from datasets.utils.logging import disable_progress_bar
disable_progress_bar()
```

https://github.com/huggingface/datasets/blob/c6e08fcfc3a04e53430c26fa7c07da4cb18d977d/src/datasets/utils/logging.py#L233-L236

Why not have a parameter in the function such as `progress: bool = True`?

+1. We shouldn't need to play with logging levels for a simple thing like this. For instance, trainers have an option `show_progress` that does exactly this.

Bump on this, such a simple QOL issue why can't we fix this?
Add CLIMATE-FEVER dataset

As suggested by @SBrandeis , fresh PR that adds CLIMATE-FEVER. Replaces PR #1579.

---

A dataset adopting the FEVER methodology that consists of 1,535 real-world claims regarding climate-change collected on the internet. Each claim is accompanied by five manually annotated evidence sentences retrieved from the English Wikipedia that support, refute or do not give enough information to validate the claim totalling in 7,675 claim-evidence pairs. The dataset features challenging claims that relate multiple facets and disputed cases of claims where both supporting and refuting evidence are present.

More information can be found at:

* Homepage: http://climatefever.ai
* Paper: https://arxiv.org/abs/2012.00614

Comments:
Thank you @lhoestq for your comments! 😄 I added your suggested changes, ran the tests and regenerated `dataset_infos.json` and `dummy_data`.
Can't call shape on the output of select()

I get the error `TypeError: tuple expected at most 1 argument, got 2` when calling `shape` on the output of `select()`.
It's line 531 in shape in arrow_dataset.py that causes the problem:
``return tuple(self._indices.num_rows, self._data.num_columns)``
This makes sense, since `tuple(num1, num2)` is not a valid call.
 
Full code to reproduce:

```python
dataset = load_dataset("cnn_dailymail", "3.0.0")
train_set = dataset["train"]
t = train_set.select(range(10))
print(t.shape)

Comments:
Indeed that's a typo, do you want to open a PR to fix it?

Yes, created a PR
Adding myPOS2017 dataset

myPOS Corpus (Myanmar Part-of-Speech Corpus) for Myanmar language NLP Research and Developments

Comments:
I've updated the code and Readme to reflect your comments.
Thank you very much,

looks like this PR includes changes about many other files than the ones for myPOS2017

Could you open another branch and another PR please ?
(or fix this branch)

Hi @hungluumfc ! Have you had a chance to fix this PR so that it only includes the changes for `mypos` ? 

Feel free to ping me if you have questions or if I can help :) 

Thanks for your contribution, @hungluumfc. Are you still interested in adding this dataset?

We are removing the dataset scripts from this GitHub repo and moving them to the Hugging Face Hub: https://huggingface.co/datasets

We would suggest you create this dataset there. Please, feel free to tell us if you need some help.
data loader for reading comprehension task

added doc2dial data loader and dummy data for reading comprehension task.

Comments:
Thank you for all the feedback! I have updated the dummy data with a zip under 30KB, which needs to include at least one data instance from both document domain and dialog domain. Please let me know if it is still too big. Thanks!

Thank you again for the feedback! I am not too sure what the preferable style for data instance in readme, but still added my edits. Thanks!
Can't filter language:EN on https://huggingface.co/datasets

When visiting https://huggingface.co/datasets, I don't see an obvious way to filter only English datasets. This is unexpected for me, am I missing something? I'd expect English to be selectable in the language widget. This problem reproduced on Mozilla Firefox and MS Edge:

![screenshot](https://user-images.githubusercontent.com/4547987/102792244-892e1f00-43a8-11eb-9e89-4826ca201a87.png)


Comments:
cc'ing @mapmeld 

Full language list is now deployed to https://huggingface.co/datasets ! Recommend close

Cool @mapmeld ! My 2 cents (for a next iteration), it would be cool to have a small search widget in the filter dropdown as you have a ton of languages now here! Closing this in the meantime.
added TurkishMovieSentiment dataset

This PR adds the **TurkishMovieSentiment: This dataset contains turkish movie reviews.**

- **Homepage:** [https://www.kaggle.com/mustfkeskin/turkish-movie-sentiment-analysis-dataset/tasks](https://www.kaggle.com/mustfkeskin/turkish-movie-sentiment-analysis-dataset/tasks)
- **Point of Contact:** [Mustafa Keskin](https://www.linkedin.com/in/mustfkeskin/)

Comments:
> I just generated the dataset_infos.json file
> 
> Thanks for adding this one !

Thank you very much for your support.
shuffle with torch generator 

Hi
I need to shuffle mutliple large datasets with `generator = torch.Generator()` for a distributed sampler which needs to make sure datasets are consistent across different cores, for this, this is really necessary for me to use  torch generator, based on documentation this generator is not supported with datasets, I really need to make shuffle work with this generator and I was wondering what I can do about this issue, thanks for your help 

@lhoestq 

Comments:
Is there a way one can convert the two generator? not sure overall what alternatives I could have to shuffle the datasets with a torch generator, thanks 

@lhoestq  let me please expalin in more details, maybe you could help me suggesting an alternative to solve the issue for now, I have multiple large datasets using huggingface library, then I need to define a distributed sampler on top of it, for this I need to shard the datasets and give each shard to each core, but before sharding I need to shuffle the dataset, if you are familiar with distributed sampler in pytorch, this needs to be done based on seed+epoch generator to make it consistent across the cores they do it through defining a torch generator, I was wondering if you could tell me how I can shuffle the data for now, I am unfortunately blocked by this and have a limited time left, and I greatly appreciate your help on this. thanks 

@lhoestq  Is there a way I could shuffle the datasets from this library with a custom defined shuffle function? thanks for your help on this. 

Right now the shuffle method only accepts the `seed` (optional int) or `generator` (optional `np.random.Generator`) parameters.

Here is a suggestion to shuffle the data using your own shuffle method using `select`.
`select` can be used to re-order the dataset samples or simply pick a few ones if you want.
It's what is used under the hood when you call `dataset.shuffle`.

To use `select` you must have the list of re-ordered indices of your samples.

Let's say you have a `shuffle` methods that you want to use. Then you can first build your shuffled list of indices:
```python
shuffled_indices = shuffle(range(len(dataset)))
```

Then you can shuffle your dataset using the shuffled indices with 
```python
shuffled_dataset = dataset.select(shuffled_indices)
```

Hope that helps

thank you @lhoestq  thank you very much for responding to my question, this greatly helped me and remove the blocking for continuing my work, thanks. 

@lhoestq  could you confirm the method proposed does not bring the whole data into memory? thanks 

Yes the dataset is not loaded into memory

great. thanks a lot.
shuffle does not accept seed 

Hi
I need to shuffle the dataset, but this needs to be based on epoch+seed to be consistent across the cores, when I pass seed to shuffle, this does not accept seed, could you assist me with this? thanks  @lhoestq
 

Comments:
Hi, did you check the doc on `shuffle`?
https://huggingface.co/docs/datasets/package_reference/main_classes.html?datasets.Dataset.shuffle#datasets.Dataset.shuffle

Hi Thomas
thanks for reponse, yes, I did checked it, but this does not work for me please see 

```
(internship) rkarimi@italix17:/idiap/user/rkarimi/dev$ python 
Python 3.7.9 (default, Aug 31 2020, 12:42:55) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import datasets 
2020-12-20 01:48:50.766004: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2020-12-20 01:48:50.766029: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
>>> data = datasets.load_dataset("scitail", "snli_format")
cahce dir  /idiap/temp/rkarimi/cache_home_1/datasets
cahce dir  /idiap/temp/rkarimi/cache_home_1/datasets
Reusing dataset scitail (/idiap/temp/rkarimi/cache_home_1/datasets/scitail/snli_format/1.1.0/fd8ccdfc3134ce86eb4ef10ba7f21ee2a125c946e26bb1dd3625fe74f48d3b90)
>>> data.shuffle(seed=2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: shuffle() got an unexpected keyword argument 'seed'

```

datasets version
`datasets                  1.1.2                     <pip>
`


Thanks for reporting ! 

Indeed it looks like an issue with `suffle` on `DatasetDict`. We're going to fix that.
In the meantime you can shuffle each split (train, validation, test) separately:
```python
shuffled_train_dataset = data["train"].shuffle(seed=42)
```
Not able to use 'jigsaw_toxicity_pred' dataset

 When trying to use jigsaw_toxicity_pred dataset, like this in a [colab](https://colab.research.google.com/drive/1LwO2A5M2X5dvhkAFYE4D2CUT3WUdWnkn?usp=sharing):
```
from datasets import list_datasets, list_metrics, load_dataset, load_metric

ds = load_dataset("jigsaw_toxicity_pred")
```
 
I see below error:

> FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/jigsaw_toxicity_pred/jigsaw_toxicity_pred.py

During handling of the above exception, another exception occurred:

FileNotFoundError                         Traceback (most recent call last)
FileNotFoundError: Couldn't find file at https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/jigsaw_toxicity_pred/jigsaw_toxicity_pred.py

During handling of the above exception, another exception occurred:

FileNotFoundError                         Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/datasets/load.py in prepare_module(path, script_version, download_config, download_mode, dataset, force_local_path, **download_kwargs)
    280                 raise FileNotFoundError(
    281                     "Couldn't find file locally at {}, or remotely at {} or {}".format(
--> 282                         combined_path, github_file_path, file_path
    283                     )
    284                 )

FileNotFoundError: Couldn't find file locally at jigsaw_toxicity_pred/jigsaw_toxicity_pred.py, or remotely at https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/jigsaw_toxicity_pred/jigsaw_toxicity_pred.py or https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/jigsaw_toxicity_pred/jigsaw_toxicity_pred.py

Comments:
Hi @jassimran,
The `jigsaw_toxicity_pred` dataset has not been released yet, it will be available with version 2 of `datasets`, coming soon.
You can still access it by installing the master (unreleased) version of datasets directly :
`pip install git+https://github.com/huggingface/datasets.git@master`
Please let me know if this helps

Thanks.That works for now.
adding  ted_talks_iwslt

UPDATE2: (2nd Jan) Wrote a long writeup on the slack channel. I don't think this approach is correct. Basically this created language pairs (109*108) 
Running the `pytest `went for more than 40+ hours and it was still running!  
So working on a different approach, such that the number of configs = number of languages. Will make a new pull request with that. 

UPDATE: This requires manual download dataset

This is a draft version 

Comments:
Closing this with reference to the new approach #1676
added Semantic Scholar Open Research Corpus

I picked up this dataset [Semantic Scholar Open Research Corpus](https://allenai.org/data/s2orc) but it contains 6000 files to be downloaded. I tried the current code with 100 files and it worked fine (took ~15GB space). For 6000 files it would occupy ~900GB space which I don’t have. Can someone from the HF team with that much of disk space help me with generate dataset_infos and dummy_data?

Comments:
I think we’ll need complete dataset_infos.json to create YAML tags. I ran the script again with 100 files after going through your comments and it was occupying ~16 GB space. So in total it should take ~960GB and I don’t have this much memory available with me. Also, I'll have to download the whole dataset for generating dummy data, right?
Navigation version breaking

Hi, 

when navigating docs (Chrome, Ubuntu) (e.g. on this page: https://huggingface.co/docs/datasets/loading_metrics.html#using-a-custom-metric-script) the version control dropdown has the wrong string displayed as the current version: 

![image](https://user-images.githubusercontent.com/3007947/102632187-02cad080-414f-11eb-813b-28f3c8d80def.png)

**Edit:** this actually happens _only_ if you open a link to a concrete subsection.

IMO, the best way to fix this without getting too deep into the intricacies of retrieving version numbers from the URL would be to change [this](https://github.com/huggingface/datasets/blob/master/docs/source/_static/js/custom.js#L112) line to:
```
let label = (version in versionMapping) ? version : stableVersion
```
which delegates the check to the (already maintained) keys of the version mapping dictionary & should be more robust. There's a similar ternary expression [here](https://github.com/huggingface/datasets/blob/master/docs/source/_static/js/custom.js#L97) which should also fail in this case.

I'd also suggest swapping this [block](https://github.com/huggingface/datasets/blob/master/docs/source/_static/js/custom.js#L80-L90) to `string.contains(version) for version in versionMapping` which might be more robust. I'd add a PR myself but I'm by no means competent in JS :)

I also have a side question wrt. docs versioning: I'm trying to make docs for a project which are versioned alike to your dropdown versioning. I was wondering how do you handle storage of multiple doc versions on your server? Do you update what `https://huggingface.co/docs/datasets` points to for every stable release & manually create new folders for each released version?
So far I'm building & publishing (scping) the docs to the server with a github action which works well for a single version, but would ideally need to reorder the public files triggered on a new release.

Comments:
Not relevant for our current docs :).
Add retries to HTTP requests

## What does this PR do ?

Adding retries to HTTP GET & HEAD requests, when they fail with a `ConnectTimeout` exception.

The "canonical" way to do this is to use [urllib's Retry class](https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html#urllib3.util.Retry) and wrap it in a [HttpAdapter](https://requests.readthedocs.io/en/master/api/#requests.adapters.HTTPAdapter). Seems a bit overkill to me, plus it forces us to use the `requests.Session` object. I prefer this simpler implementation. I'm open to remarks and suggestions @lhoestq @yjernite 

Fixes #1102 

Comments:
merging this one then :)