kardwalker/my_dataset huggingface dataset:
GitHub issue #2125: Is dataset timit_asr broken?
State: closed | Created: 2021-03-28T08:30:18Z

Using `timit_asr` dataset, I saw all records are the same.

``` python
from datasets import load_dataset, load_metric

timit = load_dataset("timit_asr")

from datasets import ClassLabel
import random
import pandas as pd
from IPython.display import display, HTML

def show_random_elements(dataset, num_examples=10):
    assert num_examples <= len(dataset), "Can't pick more elements than there are in the dataset."
    picks = []
    for _ in range(num_examples):
        pick = random.randint(0, len(dataset)-1)
        while pick in picks:
            pick = random.randint(0, len(dataset)-1)
        picks.append(pick)

    df = pd.DataFrame(dataset[picks])
    display(HTML(df.to_html()))


show_random_elements(timit['train'].remove_columns(["file", "phonetic_detail", "word_detail", "dialect_region", "id", 
                                                    "sentence_type", "speaker_id"]), num_examples=20)

```

`output`

<img width="312" alt="Screen Shot 2021-03-28 at 17 29 04" src="https://user-images.githubusercontent.com/42398050/112746646-21acee80-8feb-11eb-84f3-dbb5d4269724.png">


I double-checked it [here](https://huggingface.co/datasets/viewer/), and met the same problem.

<img width="1374" alt="Screen Shot 2021-03-28 at 17 32 07" src="https://user-images.githubusercontent.com/42398050/112746698-9bdd7300-8feb-11eb-97ed-5babead385f4.png">
GitHub issue #2124: Adding ScaNN library to do MIPS?
State: open | Created: 2021-03-28T00:07:00Z

@lhoestq Hi I am thinking of adding this new google library to do the MIPS similar to **add_faiss_idex**. As the paper suggests, it is really fast when it comes to retrieving the nearest neighbors. 

https://github.com/google-research/google-research/tree/master/scann

![image](https://user-images.githubusercontent.com/16892570/112738294-78ec9800-8fc6-11eb-9a5f-3d7ee5818e76.png)
GitHub issue #2123: Problem downloading GEM wiki_auto_asset_turk dataset
State: closed | Created: 2021-03-27T18:41:28Z

@yjernite 

### Summary

I am currently working on the GEM datasets and do not manage to download the wiki_auto_asset_turk data, whereas all other datasets download well with the same code.

### Steps to reproduce
Code snippet:

from datasets import load_dataset
#dataset = load_dataset('gem', 'web_nlg_en')
dataset = load_dataset('gem', 'wiki_auto_asset_turk')

```

**Expected behavior:**

I expect the dataset to start downloading (download bar appears and progresses toward 100%)

**Actual behavior:**
Instead of seeing the download bar appearing, nothing happens; the following appears in the console as expected, but nothing more:

Downloading: 36.6kB [00:00, 37.2MB/s]
Downloading: 41.7kB [00:00, ?B/s]
Downloading and preparing dataset gem/wiki_auto_asset_turk (download: 121.37 MiB, generated: 145.69 MiB, post-processed: Unknown size, total: 267.07 MiB) to C:\Users\sfmil\.cache\huggingface\datasets\gem\wiki_auto_asset_turk\1.0.0\f252756d7f1b8f019aac71a1623b2950acfe10d25d956668ac4eae4e93c58b8d...

### Is this a regression?
No, it was the first time I was trying to download this dataset (same for the other ones).

### Debug info
- Python version: Python 3.8.2
- OS version: Windows 10 Family
GitHub issue #2120: dataset viewer does not work anymore 
State: closed | Created: 2021-03-26T13:22:13Z

Hi
I normally use this link to see all datasets and how I can load them 


https://huggingface.co/datasets/viewer/

Now I am getting 

502 Bad Gateway
nginx/1.18.0 (Ubuntu)

could you bring this webpage back ? this was very helpful @lhoestq 
thanks for your help
GitHub issue #2119: copy.deepcopy os.environ instead of copy
State: closed | Created: 2021-03-26T03:58:38Z

Fixes: https://github.com/huggingface/datasets/issues/2115

- bug fix: using envrion.copy() returns a dict.
- using deepcopy(environ) returns an `_environ` object
- Changing the datatype of the _environ object can break code, if subsequent libraries perform operations using apis exclusive to the environ object, like `environ.getenv()` for example.


Testing:

Tested the change on my terminal:

```
>>> import os
>>> x = deepcopy(os.environ)
>>> y = os.environ
>>> x is y
False
>>> isinstance(x, type(os.environ))
True
>>> z = os.environ.copy()
>>> isinstance(z, type(os.environ))
False
>>> isinstance(z, dict)
True
```
GitHub issue #2117: load_metric from local "glue.py" meet error 'NoneType' object is not callable
State: closed | Created: 2021-03-26T02:35:22Z

actual_task = "mnli" if task == "mnli-mm" else task
dataset = load_dataset(path='/home/glue.py', name=actual_task)
metric = load_metric(path='/home/glue.py', name=actual_task)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-7ab77a465d81> in <module>
      1 actual_task = "mnli" if task == "mnli-mm" else task
      2 dataset = load_dataset(path='/home/jcli/glue.py', name=actual_task)
----> 3 metric = load_metric(path='/home/jcli/glue.py', name=actual_task)

~/anaconda3/envs/pytorch/lib/python3.6/site-packages/datasets/load.py in load_metric(path, config_name, process_id, num_process, cache_dir, experiment_id, keep_in_memory, download_config, download_mode, script_version, **metric_init_kwargs)
    508         keep_in_memory=keep_in_memory,
    509         experiment_id=experiment_id,
--> 510         **metric_init_kwargs,
    511     )
    512 

TypeError: 'NoneType' object is not callable

Please help
GitHub issue #2116: Creating custom dataset results in error while calling the map() function
State: closed | Created: 2021-03-26T00:37:46Z

calling `map()` of `datasets` library results into an error while defining a Custom dataset.
Reproducible example:
```
import datasets
class MyDataset(datasets.Dataset):

    def __init__(self, sentences):
        "Initialization"
        self.samples = sentences

    def __len__(self):
        "Denotes the total number of samples"
        return len(self.samples)

    def __getitem__(self, index):
        "Generates one sample of data"
        # Select sample
        # Load data and get label
        samples = self.samples[index]

        return samples

def preprocess_function_train(examples):
        inputs = examples
        labels = [example+tokenizer.eos_token for example in examples ]
        inputs = tokenizer(inputs, max_length=30, padding=True, truncation=True)
        labels = tokenizer(labels, max_length=30, padding=True, truncation=True)
        model_inputs = inputs
        model_inputs["labels"] = labels["input_ids"]
        print("about to return")
        return model_inputs


##train["sentence"] is dataframe column
train_dataset = MyDataset(train['sentence'].values.tolist())
train_dataset = train_dataset.map(
            preprocess_function,
            batched = True,
            batch_size=32
        )
```

Stack trace of error:
```
Traceback (most recent call last):
  File "dir/train_generate.py", line 362, in <module>
    main()
  File "dir/train_generate.py", line 245, in main
    train_dataset = train_dataset.map(
  File "anaconda_dir/anaconda3/envs/env1/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 1244, in map
    return self._map_single(
  File "anaconda_dir/anaconda3/envs/env1/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 149, in wrapper
    unformatted_columns = set(self.column_names) - set(self._format_columns or [])
  File "anaconda_dir/anaconda3/envs/env1/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 526, in column_names
    return self._data.column_names
AttributeError: 'MyDataset' object has no attribute '_data'
```
GitHub issue #2115: The datasets.map() implementation modifies the datatype of os.environ object
State: closed | Created: 2021-03-25T20:29:19Z

In our testing, we noticed that the datasets.map() implementation is modifying the datatype of python os.environ object from '_Environ' to 'dict'.

This causes following function calls to fail as follows:

`   
     x = os.environ.get("TEST_ENV_VARIABLE_AFTER_dataset_map", default=None)
    TypeError: get() takes no keyword arguments
`
It looks like the following line in datasets.map implementation introduced this functionality.

https://github.com/huggingface/datasets/blob/0cb1ac06acb0df44a1cf4128d03a01865faa2504/src/datasets/arrow_dataset.py#L1421

Here is the test script to reproduce this error. 


```
from datasets import load_dataset
from transformers import AutoTokenizer
import os


def test_train():
    model_checkpoint = "distilgpt2"
    datasets = load_dataset('wikitext', 'wikitext-2-raw-v1')
    tokenizer = AutoTokenizer.from_pretrained(model_checkpoint, use_fast=True)
    tokenizer.pad_token = tokenizer.eos_token


    def tokenize_function(examples):
        y = tokenizer(examples['text'], truncation=True, max_length=64)
        return y

    x = os.environ.get("TEST_ENV_VARIABLE_BEFORE_dataset_map", default=None)
    print(f"Testing environment variable: TEST_ENV_VARIABLE_BEFORE_dataset_map {x}")
    print(f"Data type of os.environ before datasets.map = {os.environ.__class__.__name__}")
    datasets.map(tokenize_function, batched=True, num_proc=2, remove_columns=["text"])
    print(f"Data type of os.environ after datasets.map = {os.environ.__class__.__name__}")
    x = os.environ.get("TEST_ENV_VARIABLE_AFTER_dataset_map", default=None)
    print(f"Testing environment variable: TEST_ENV_VARIABLE_AFTER_dataset_map {x}")


if __name__ == "__main__":
    test_train()


```
GitHub issue #2114: Support for legal NLP datasets (EURLEX, ECtHR cases and EU-REG-IR)
State: closed | Created: 2021-03-25T18:40:17Z

Add support for two legal NLP datasets:

- EURLEX (https://www.aclweb.org/anthology/P19-1636/)
- ECtHR cases (https://arxiv.org/abs/2103.13084)
- EU-REG-IR (https://arxiv.org/abs/2101.10726)
GitHub issue #2112: Support for legal NLP datasets (EURLEX and ECtHR cases)
State: closed | Created: 2021-03-25T16:24:17Z

Add support for two legal NLP datasets:
- EURLEX (https://www.aclweb.org/anthology/P19-1636/)
- ECtHR cases (https://arxiv.org/abs/2103.13084)
GitHub issue #2109: Add more issue templates and customize issue template chooser
State: closed | Created: 2021-03-25T09:41:53Z

When opening an issue, it is not evident for the users how to choose a blank issue template. There is a link at the bottom of all the other issue templates (`Don’t see your issue here? Open a blank issue.`), but this is not very visible for users. This is the reason why many users finally chose the `add-dataset` template instead (this is more visible) for issues that indeed are not requesting the addition of a new dataset.

~~With this PR, the default blank issue template would be as visible as the other templates (as the `add-dataset` template), thus making easier for the users to choose it.~~

With this PR:
- more issue templates, besides `add-dataset`, are added: `bug-report` and `feature-request`
- the issue template chooser is customized, so that it now includes a link to `Discussions` for questions
GitHub issue #2108: Is there a way to use a GPU only when training an Index  in the process of add_faisis_index?
State: open | Created: 2021-03-24T21:32:16Z

Motivation - Some FAISS indexes like IVF consist of the training step that clusters the dataset into a given number of indexes. It would be nice if we can use a GPU to do the training step and covert the index back to CPU as mention in [this faiss example](https://gist.github.com/mdouze/46d6bbbaabca0b9778fca37ed2bcccf6).
GitHub issue #2107: Metadata validation
State: closed | Created: 2021-03-24T08:52:41Z

- `pydantic` metadata schema with dedicated validators against our taxonomy
- ci script to validate new changes against this schema and start a vertuous loop
- soft validation on tasks ids since we expect the taxonomy to undergo some changes in the near future

for reference with the current validation we have ~365~ 378 datasets with invalid metadata! full error report [_here_.](https://gist.github.com/theo-m/61b3c0c47fc6121d08d3174bd4c2a26b)
GitHub issue #2106: WMT19 Dataset for Kazakh-English is not formatted correctly
State: open | Created: 2021-03-23T20:14:47Z

In addition to the bug of languages being switched from Issue @415, there are incorrect translations in the dataset because the English-Kazakh translations have a one off formatting error.

The News Commentary v14 parallel data set for kk-en from http://www.statmt.org/wmt19/translation-task.html has a bug here:

> Line 94. The Swiss National Bank, for its part, has been battling with the deflationary effects of the franc’s dramatic appreciation over the past few years.	Швейцарияның Ұлттық банкі өз тарапынан, соңғы бірнеше жыл ішінде франк құнының қатты өсуінің дефляциялық әсерімен күресіп келеді.
> 
> Line 95. Дефляциялық күштер 2008 жылы терең және ұзаққа созылған жаһандық дағдарысқа байланысты орын алған ірі экономикалық және қаржылық орын алмасулардың арқасында босатылды.  Жеке қарыз қаражаты үлесінің қысқаруы орталық банктің рефляцияға жұмсалған күш-жігеріне тұрақты соққан қарсы желдей болды.
> 
> Line 96. The deflationary forces were unleashed by the major economic and financial dislocations associated with the deep and protracted global crisis that erupted in 2008. Private deleveraging became a steady headwind to central bank efforts to reflate.	2009 жылы, алдыңғы қатарлы экономикалардың шамамен үштен бірі бағаның төмендеуін көрсетті, бұл соғыстан кейінгі жоғары деңгей болды.

As you can see, line 95 has only the Kazakh translation which should be part of line 96. This causes all of the following English-Kazakh translation pairs to be one off rendering ALL of those translations incorrect. This issue was not fixed when the dataset was imported to Huggingface. By running this code 

```
import datasets
from datasets import load_dataset
dataset = load_dataset('wmt19', 'kk-en')
for key in dataset['train']['translation']:
    if 'The deflationary forces were unleashed by the major economic and financial dislocations associated with the deep and protracted global crisis that erupted in 2008.' in key['kk']:
        print(key['en'])
        print(key['kk'])
        break
```
we get: 
> 2009 жылы, алдыңғы қатарлы экономикалардың шамамен үштен бірі бағаның төмендеуін көрсетті, бұл соғыстан кейінгі жоғары деңгей болды.
> The deflationary forces were unleashed by the major economic and financial dislocations associated with the deep and protracted global crisis that erupted in 2008. Private deleveraging became a steady headwind to central bank efforts to reflate.

which shows that the issue still persists in the Huggingface dataset. The Kazakh sentence matches up to the next English sentence in the dataset instead of the current one.

Please let me know if there's you have any ideas to fix this one-off error from the dataset or if this can be fixed by Huggingface.
GitHub issue #2105: Request to remove S2ORC dataset
State: open | Created: 2021-03-23T19:43:06Z

Hi!  I was wondering if it's possible to remove [S2ORC](https://huggingface.co/datasets/s2orc) from hosting on Huggingface's platform?  Unfortunately, there are some legal considerations about how we make this data available.  Happy to add back to Huggingface's platform once we work out those hurdles!  Thanks!
GitHub issue #2104: Trouble loading wiki_movies
State: closed | Created: 2021-03-23T18:59:54Z

Hello,
I am trying to load_dataset("wiki_movies") and it gives me this error - 

`FileNotFoundError: Couldn't find file locally at wiki_movies/wiki_movies.py, or remotely at https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/wiki_movies/wiki_movies.py or https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/wiki_movies/wiki_movies.py`

Trying to do `python run_mlm.py \
    --model_name_or_path roberta-base \
    --dataset_name wiki_movies \` also gives the same error. 

Is this something on my end? From what I can tell, this dataset was re-added by @lhoestq a few months ago. 
Thank you!
GitHub issue #2103: citation, homepage, and license fields of `dataset_info.json` are duplicated many times
State: closed | Created: 2021-03-23T17:18:09Z

This happens after a `map` operation when `num_proc` is set to `>1`. I tested this by cleaning up the json before running the `map` op on the dataset so it's unlikely it's coming from an earlier concatenation.

Example result:
```
"citation": "@ONLINE {wikidump,\n    author = {Wikimedia Foundation},\n    title  = {Wikimedia Downloads},\n    url    = {https://dumps.wikimedia.org}\n}\n\n@ONLINE {wikidump,\n    author = {Wikimedia Foundation},\n    title  = {Wikimedia Downloads},\n    url    = {https://dumps.wikimedia.org}\n}\n\n@ONLINE {wikidump,\n    author = {Wikimedia Foundation},\n    title  = {Wikimedia Downloads},\n    url    = {https://dumps.wikimedia.org}\n}\n\n@ONLINE {wikidump,\n    author = {Wikimedia Foundation},\n    title  = {Wikimedia Downloads},\n    url    = {https://dumps.wikimedia.org}\n}\n\n@ONLINE {wikidump,\n    author = {Wikimedia Foundation},\n    title  = {Wikimedia Downloads},\n    url    = {https://dumps.wikimedia.org}\n}\n\n@ONLINE {wikidump,\n    author = {Wikimedia Foundation},\n    title  = {Wikimedia Downloads},\n    url    = {https://dumps.wikimedia.org}\n}\n\n@ONLINE {wikidump,\n    author = {Wikimedia Foundation},\n    title  = {Wikimedia Downloads},\n    url    = {https://dumps.wikimedia.org}\n}\n\n@ONLINE {wikidump,\n    author = {Wikimedia Foundation},\n    title  = {Wikimedia Downloads},\n    url    = {https://dumps.wikimedia.org}\n}\n\n@ONLINE {wikidump,\n    author = {Wikimedia Foundation},\n    title  = {Wikimedia Downloads},\n    url    = {https://dumps.wikimedia.org}\n}\n\n@ONLINE {wikidump,\n    author = {Wikimedia Foundation},\n    title  = {Wikimedia Downloads},\n    url    = {https://dumps.wikimedia.org}\n}\n\n@ONLINE {wikidump,\n    author = {Wikimedia Foundation},\n    title  = {Wikimedia Downloads},\n    url    = {https://dumps.wikimedia.org}\n}\n\n@ONLINE {wikidump,\n    author = {Wikimedia Foundation},\n    title  = {Wikimedia Downloads},\n
```

@lhoestq and I believe this is happening due to the fields being concatenated `num_proc` times.
GitHub issue #2099: load_from_disk takes a long time to load local dataset
State: closed | Created: 2021-03-23T09:28:37Z

I have an extremely large tokenized dataset (24M examples) that loads in a few minutes. However, after adding a column similar to `input_ids` (basically a list of integers) and saving the dataset to disk, the load time goes to >1 hour. I've even tried using `np.uint8` after seeing #1985 but it doesn't seem to be helping (the total size seems to be smaller though).

Does anyone know what could be the issue? Or does the casting of that column to `int8` need to happen in the function that writes the arrow table instead of in the `map` where I create the list of integers?

Tagging @lhoestq since you seem to be working on these issues and PRs :)
GitHub issue #2096: CoNLL 2003 dataset not including German
State: closed | Created: 2021-03-22T19:23:56Z

Hello, thanks for all the work on developing and maintaining this amazing platform, which I am enjoying working with!

I was wondering if there is a reason why the German CoNLL 2003 dataset is not included in the [repository](https://github.com/huggingface/datasets/tree/master/datasets/conll2003), since a copy of it could be found in some places on the internet such as GitHub? I could help adding the German data to the hub, unless there are some copyright issues that I am unaware of...

This is considering that many work use the union of CoNLL 2002 and 2003 datasets for comparing cross-lingual NER transfer performance in `en`, `de`, `es`, and `nl`. E.g., [XLM-R](https://www.aclweb.org/anthology/2020.acl-main.747.pdf).

## Adding a Dataset
- **Name:** CoNLL 2003 German
- **Paper:** https://www.aclweb.org/anthology/W03-0419/
- **Data:** https://github.com/huggingface/datasets/tree/master/datasets/conll2003
GitHub issue #2089: Add documentaton for dataset README.md files
State: closed | Created: 2021-03-20T11:44:38Z

Hi,
the dataset README files have special headers.
Somehow a documenation of the allowed values and tags is missing.
Could you add that?

Just to give some concrete questions that should be answered imo:
- which values can be passted to multilinguality?
- what should be passed to language_creators?
- which values should licenses have? What do I say when it is a custom license? Should I add a link?
- how should I choose size_categories ? What are valid ranges?
- what are valid task_categories?

Thanks
Philip
GitHub issue #2084: CUAD - Contract Understanding Atticus Dataset
State: closed | Created: 2021-03-19T09:27:43Z

## Adding a Dataset
- **Name:** CUAD - Contract Understanding Atticus Dataset
- **Description:** As one of the only large, specialized NLP benchmarks annotated by experts, CUAD can serve as a challenging research benchmark for the broader NLP community.
- **Paper:** https://arxiv.org/abs/2103.06268
- **Data:** https://github.com/TheAtticusProject/cuad/
- **Motivation:** good domain specific datasets are valuable

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).
GitHub issue #2083: `concatenate_datasets` throws error when changing the order of datasets to concatenate
State: closed | Created: 2021-03-19T08:29:48Z

Hey, 

I played around with the `concatenate_datasets(...)` function: https://huggingface.co/docs/datasets/package_reference/main_classes.html?highlight=concatenate_datasets#datasets.concatenate_datasets

and noticed that when the order in which the datasets are concatenated changes an error is thrown where it should not IMO.

Here is a google colab to reproduce the error: https://colab.research.google.com/drive/17VTFU4KQ735-waWZJjeOHS6yDTfV5ekK?usp=sharing
GitHub issue #2082: Updated card using information from data statement and datasheet
State: closed | Created: 2021-03-19T00:39:38Z

I updated and clarified the REFreSD [data card](https://github.com/mcmillanmajora/datasets/blob/refresd_card/datasets/refresd/README.md) with information from the Eleftheria's [website](https://elbria.github.io/post/refresd/). I added brief descriptions where the initial card referred to the paper, and I also recreated some of the tables in the paper to show relevant dataset statistics.

I'll email Eleftheria to see if she has any comments on the card.
GitHub issue #2080: Multidimensional arrays in a Dataset
State: closed | Created: 2021-03-18T16:29:14Z

Hi,

I'm trying to put together a `datasets.Dataset` to be used with LayoutLM which is available in `transformers`. This model requires as input the bounding boxes of each of the token of a sequence. This is when I realized that `Dataset` does not support multi-dimensional arrays as a value for a column in a row.

The following code results in conversion error in pyarrow (`pyarrow.lib.ArrowInvalid: ('Can only convert 1-dimensional array values', 'Conversion failed for column bbox with type object')`)

```
from datasets import Dataset
import pandas as pd
import numpy as np

dataset = pd.DataFrame({
    'bbox': [
        np.array([[1,2,3,4],[1,2,3,4],[1,2,3,4]]),
        np.array([[1,2,3,4],[1,2,3,4],[1,2,3,4]]),
        np.array([[1,2,3,4],[1,2,3,4],[1,2,3,4]]),
        np.array([[1,2,3,4],[1,2,3,4],[1,2,3,4]])
    ],
    'input_ids': [1, 2, 3, 4]
})
dataset = Dataset.from_pandas(dataset)
```

Since I wanted to use pytorch for the downstream training task, I also tried a few ways to directly put in a column of 2-D pytorch tensor in a formatted dataset, but I can only have a list of 1-D tensors, or a list of arrays, or a list of lists.

```
import torch
from datasets import Dataset
import pandas as pd

dataset = pd.DataFrame({
    'bbox': [
        [[1,2,3,4],[1,2,3,4],[1,2,3,4]],
        [[1,2,3,4],[1,2,3,4],[1,2,3,4]],
        [[1,2,3,4],[1,2,3,4],[1,2,3,4]],
        [[1,2,3,4],[1,2,3,4],[1,2,3,4]]
    ],
    'input_ids': [1, 2, 3, 4]
})
dataset = Dataset.from_pandas(dataset)

def test(examples):
    return {'bbbox': torch.Tensor(examples['bbox'])}
dataset = dataset.map(test)
print(dataset[0]['bbox'])
print(dataset[0]['bbbox'])

dataset.set_format(type='torch', columns=['input_ids', 'bbox'], output_all_columns=True)
print(dataset[0]['bbox'])
print(dataset[0]['bbbox'])

def test2(examples):
    return {'bbbox': torch.stack(examples['bbox'])}
dataset = dataset.map(test2)

print(dataset[0]['bbox'])
print(dataset[0]['bbbox'])
```

Is is possible to support n-D arrays/tensors in datasets? 
It seems that it can also be useful for this [feature request](https://github.com/huggingface/datasets/issues/263).
GitHub issue #2078: MemoryError when computing WER metric
State: closed | Created: 2021-03-18T11:30:05Z

Hi, I'm trying to follow the ASR example to try Wav2Vec. This is the code that I use for WER calculation:

```
wer = load_metric("wer")
print(wer.compute(predictions=result["predicted"], references=result["target"]))
```

However, I receive the following exception:

`Traceback (most recent call last):
  File "/home/diego/IpGlobal/wav2vec/test_wav2vec.py", line 51, in <module>
    print(wer.compute(predictions=result["predicted"], references=result["target"]))
  File "/home/diego/miniconda3/envs/wav2vec3.6/lib/python3.6/site-packages/datasets/metric.py", line 403, in compute
    output = self._compute(predictions=predictions, references=references, **kwargs)
  File "/home/diego/.cache/huggingface/modules/datasets_modules/metrics/wer/73b2d32b723b7fb8f204d785c00980ae4d937f12a65466f8fdf78706e2951281/wer.py", line 94, in _compute
    return wer(references, predictions)
  File "/home/diego/miniconda3/envs/wav2vec3.6/lib/python3.6/site-packages/jiwer/measures.py", line 81, in wer
    truth, hypothesis, truth_transform, hypothesis_transform, **kwargs
  File "/home/diego/miniconda3/envs/wav2vec3.6/lib/python3.6/site-packages/jiwer/measures.py", line 192, in compute_measures
    H, S, D, I = _get_operation_counts(truth, hypothesis)
  File "/home/diego/miniconda3/envs/wav2vec3.6/lib/python3.6/site-packages/jiwer/measures.py", line 273, in _get_operation_counts
    editops = Levenshtein.editops(source_string, destination_string)
MemoryError`

My system has more than 10GB of available RAM. Looking at the code, I think that it could be related to the way jiwer does the calculation, as it is pasting all the sentences in a single string before calling Levenshtein editops function.
GitHub issue #2076: Issue: Dataset download error
State: open | Created: 2021-03-18T06:36:06Z

The download link in `iwslt2017.py` file does not seem to work anymore.

For example, `FileNotFoundError: Couldn't find file at https://wit3.fbk.eu/archive/2017-01-trnted/texts/zh/en/zh-en.tgz`

Would be nice if we could modify it script and use the new downloadable link?
GitHub issue #2075: ConnectionError: Couldn't reach common_voice.py
State: closed | Created: 2021-03-18T01:19:06Z

When I run: 
from datasets import load_dataset, load_metric

common_voice_train = load_dataset("common_voice", "zh-CN", split="train+validation")
common_voice_test = load_dataset("common_voice", "zh-CN", split="test")

Got:
ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/master/datasets/common_voice/common_voice.py

Version:
1.4.1

Thanks!  @lhoestq @LysandreJik @thomwolf
GitHub issue #2071: Multiprocessing is slower than single process
State: closed | Created: 2021-03-17T16:08:58Z

```python
# benchmark_filter.py
import logging
import sys
import time

from datasets import load_dataset, set_caching_enabled


if __name__ == "__main__":
    set_caching_enabled(False)
    logging.basicConfig(level=logging.DEBUG)

    bc = load_dataset("bookcorpus")

    now = time.time()
    try:
        bc["train"].filter(lambda x: len(x["text"]) < 64, num_proc=int(sys.argv[1]))
    except Exception as e:
        print(f"cancelled: {e}")
    elapsed = time.time() - now

    print(elapsed)
```

Running `python benchmark_filter.py 1` (20min+) is faster than `python benchmark_filter.py 2` (2hrs+)
GitHub issue #2068: PyTorch not available error on SageMaker GPU docker though it is installed 
State: closed | Created: 2021-03-17T10:04:27Z

I get en error when running data loading using SageMaker SDK

```
  File "main.py", line 34, in <module>
    run_training()
  File "main.py", line 25, in run_training
    dm.setup('fit')
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/core/datamodule.py", line 92, in wrapped_fn
    return fn(*args, **kwargs)
  File "/opt/ml/code/data_module.py", line 103, in setup
    self.dataset[split].set_format(type="torch", columns=self.columns)
  File "/opt/conda/lib/python3.6/site-packages/datasets/fingerprint.py", line 337, in wrapper
    out = func(self, *args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/datasets/arrow_dataset.py", line 995, in set_format
    _ = get_formatter(type, **format_kwargs)
File "/opt/conda/lib/python3.6/site-packages/datasets/formatting/__init__.py", line 114, in get_formatter
    raise _FORMAT_TYPES_ALIASES_UNAVAILABLE[format_type]
ValueError: PyTorch needs to be installed to be able to return PyTorch tensors.
```

when trying to execute dataset loading using this notebook  https://github.com/PyTorchLightning/pytorch-lightning/blob/master/notebooks/04-transformers-text-classification.ipynb, specifically lines 

```
self.columns = [c for c in self.dataset[split].column_names if c in self.loader_columns]
self.dataset[split].set_format(type="torch", columns=self.columns)
```

The SageMaker docker image used is 763104351884.dkr.ecr.eu-central-1.amazonaws.com/pytorch-training:1.4.0-gpu-py3 .

By running container interactively I have checked that torch loading completes successfully by executing `https://github.com/huggingface/datasets/blob/master/src/datasets/config.py#L39`. 

Also as a first line in the data loading module I have 

```
import os
os.environ["USE_TF"] = "0" 
os.environ["USE_TORCH"] = "1" 
````

But unfortunately the error stills persists. Any suggestions would be appreciated as I am stack.
Many Thanks!
GitHub issue #2067: Multiprocessing windows error
State: closed | Created: 2021-03-17T09:12:28Z

As described here https://huggingface.co/blog/fine-tune-xlsr-wav2vec2

When using the num_proc argument on windows the whole Python environment crashes and hanging in loop.
For example at the map_to_array part.
An error occures because the cache file already exists and windows throws and error. After this the log crashes into an loop
GitHub issue #2065: Only user permission of saved cache files, not group
State: closed | Created: 2021-03-17T00:20:22Z

Hello,

It seems when a cached file is saved from calling `dataset.map` for preprocessing, it gets the user permissions and none of the user's group permissions. As we share data files across members of our team, this is causing a bit of an issue as we have to continually reset the permission of the files. Do you know any ways around this or a way to correctly set the permissions?
GitHub issue #2060: Filtering refactor
State: closed | Created: 2021-03-16T09:23:30Z

fix https://github.com/huggingface/datasets/issues/2032

benchmarking is somewhat inconclusive, currently running on `book_corpus` with:

```python
    bc = load_dataset("bookcorpus")
    now = time.time()
    bc.filter(lambda x: len(x["text"]) < 64)
    elapsed = time.time() - now
    print(elapsed)
```

this branch does it in 233 seconds, master in 1409 seconds.
GitHub issue #2059: Error while following docs to load the `ted_talks_iwslt` dataset
State: closed | Created: 2021-03-16T09:12:19Z

I am currently trying to load the `ted_talks_iwslt` dataset into google colab.

The [docs](https://huggingface.co/datasets/ted_talks_iwslt) mention the following way of doing so.

```python
dataset = load_dataset("ted_talks_iwslt", language_pair=("it", "pl"), year="2014")
```

Executing it results in the error attached below.

```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-7dcc67154ef9> in <module>()
----> 1 dataset = load_dataset("ted_talks_iwslt", language_pair=("it", "pl"), year="2014")

4 frames
/usr/local/lib/python3.7/dist-packages/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, script_version, use_auth_token, **config_kwargs)
    730         hash=hash,
    731         features=features,
--> 732         **config_kwargs,
    733     )
    734 

/usr/local/lib/python3.7/dist-packages/datasets/builder.py in __init__(self, writer_batch_size, *args, **kwargs)
    927 
    928     def __init__(self, *args, writer_batch_size=None, **kwargs):
--> 929         super(GeneratorBasedBuilder, self).__init__(*args, **kwargs)
    930         # Batch size used by the ArrowWriter
    931         # It defines the number of samples that are kept in memory before writing them

/usr/local/lib/python3.7/dist-packages/datasets/builder.py in __init__(self, cache_dir, name, hash, features, **config_kwargs)
    241             name,
    242             custom_features=features,
--> 243             **config_kwargs,
    244         )
    245 

/usr/local/lib/python3.7/dist-packages/datasets/builder.py in _create_builder_config(self, name, custom_features, **config_kwargs)
    337             if "version" not in config_kwargs and hasattr(self, "VERSION") and self.VERSION:
    338                 config_kwargs["version"] = self.VERSION
--> 339             builder_config = self.BUILDER_CONFIG_CLASS(**config_kwargs)
    340 
    341         # otherwise use the config_kwargs to overwrite the attributes

/root/.cache/huggingface/modules/datasets_modules/datasets/ted_talks_iwslt/024d06b1376b361e59245c5878ab8acf9a7576d765f2d0077f61751158e60914/ted_talks_iwslt.py in __init__(self, language_pair, year, **kwargs)
    219             description=description,
    220             version=datasets.Version("1.1.0", ""),
--> 221             **kwargs,
    222         )
    223 

TypeError: __init__() got multiple values for keyword argument 'version'
```

How to resolve this? 

PS: Thanks a lot @huggingface team for creating this great library!
GitHub issue #2058: Is it possible to convert a `tfds` to HuggingFace `dataset`?
State: closed | Created: 2021-03-15T20:18:47Z

I was having some weird bugs with `C4`dataset version of HuggingFace, so I decided to try to download `C4`from `tfds`. I would like to know if it is possible to convert a tfds dataset to HuggingFace dataset format :)

I can also open a new issue reporting the bug I'm receiving with `datasets.load_dataset('c4','en')` in the future if you think that it would be useful.

Thanks!
GitHub issue #2054: Could not find file for ZEST dataset
State: closed | Created: 2021-03-15T09:11:58Z

I am trying to use zest dataset from Allen AI using below code in colab,
```
!pip install -q datasets
from datasets import load_dataset
dataset = load_dataset("zest")
```

I am getting the following error,
```
Using custom data configuration default

Downloading and preparing dataset zest/default (download: 5.53 MiB, generated: 19.96 MiB, post-processed: Unknown size, total: 25.48 MiB) to /root/.cache/huggingface/datasets/zest/default/0.0.0/1f7a230fbfc964d979bbca0f0130fbab3259fce547ee758ad8aa4f9c9bec6cca...
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-6-18dbbc1a4b8a> in <module>()
      1 from datasets import load_dataset
      2 
----> 3 dataset = load_dataset("zest")

9 frames
/usr/local/lib/python3.7/dist-packages/datasets/utils/file_utils.py in get_from_cache(url, cache_dir, force_download, proxies, etag_timeout, resume_download, user_agent, local_files_only, use_etag, max_retries, use_auth_token)
    612             )
    613         elif response is not None and response.status_code == 404:
--> 614             raise FileNotFoundError("Couldn't find file at {}".format(url))
    615         _raise_if_offline_mode_is_enabled(f"Tried to reach {url}")
    616         raise ConnectionError("Couldn't reach {}".format(url))

FileNotFoundError: Couldn't find file at https://ai2-datasets.s3-us-west-2.amazonaws.com/zest/zest.zip
```
GitHub issue #2053: Add bAbI QA tasks
State: closed | Created: 2021-03-14T13:04:39Z

- **Name:** *The (20) QA bAbI tasks*
- **Description:** *The (20) QA bAbI tasks are a set of proxy tasks that evaluate reading comprehension via question answering. Our tasks measure understanding in several ways: whether a system is able to answer questions via chaining facts, simple induction, deduction and many more. The tasks are designed to be prerequisites for any system that aims to be capable of conversing with a human. The aim is to classify these tasks into skill sets,so that researchers can identify (and then rectify) the failings of their systems.*
- **Paper:** [arXiv](https://arxiv.org/pdf/1502.05698.pdf)
- **Data:** [Facebook Research Page](https://research.fb.com/downloads/babi/)
- **Motivation:** This is a unique dataset with story-based Question Answering. It  is a part of the `bAbI` project by Facebook Research.

**Note**: I have currently added all the 160 configs. If this seems impractical, I can keep only a few. While each `dummy_data.zip` weighs a few KBs, overall it is around 1.3MB for all configurations. This is problematic. Let me know what is to be done.

Thanks :)


### Checkbox

- [x] Create the dataset script `/datasets/my_dataset/my_dataset.py` using the template
- [x] Fill the `_DESCRIPTION` and `_CITATION` variables
- [x] Implement `_infos()`, `_split_generators()` and `_generate_examples()`
- [x] Make sure that the `BUILDER_CONFIGS` class attribute is filled with the different configurations of the dataset and that the `BUILDER_CONFIG_CLASS` is specified if there is a custom config class.
- [x] Generate the metadata file `dataset_infos.json` for all configurations
- [x] Generate the dummy data `dummy_data.zip` files to have the dataset script tested and that they don't weigh too much (<50KB)
- [x] Add the dataset card `README.md` using the template : fill the tags and the various paragraphs
- [x] Both tests for the real data and the dummy data pass.
GitHub issue #2052: Timit_asr dataset repeats examples
State: closed | Created: 2021-03-14T11:43:43Z

Summary

When loading timit_asr dataset on datasets 1.4+, every row in the dataset is the same
Steps to reproduce

As an example, on this code there is the text from the training part:

Code snippet:
```
from datasets import load_dataset, load_metric

timit = load_dataset("timit_asr")
timit['train']['text']
#['Would such an act of refusal be useful?',
# 'Would such an act of refusal be useful?',
# 'Would such an act of refusal be useful?',
# 'Would such an act of refusal be useful?',
# 'Would such an act of refusal be useful?',
# 'Would such an act of refusal be useful?',
```
The same behavior happens for other columns

Expected behavior:

Different info on the actual timit_asr dataset

Actual behavior:

When loading timit_asr dataset on datasets 1.4+, every row in the dataset is the same. I've checked datasets 1.3 and the rows are different
Debug info

    Streamlit version: (get it with $ streamlit version)
    Python version: Python 3.6.12
    Using Conda? PipEnv? PyEnv? Pex? Using pip
    OS version: Centos-release-7-9.2009.1.el7.centos.x86_64

Additional information

You can check the same behavior on https://huggingface.co/datasets/viewer/?dataset=timit_asr
GitHub issue #2051: Add MDD Dataset
State: closed | Created: 2021-03-14T00:01:05Z

- **Name:** *MDD Dataset*
- **Description:** The Movie Dialog dataset (MDD) is designed to measure how well models can perform at goal and non-goal orientated dialog centered around the topic of movies (question answering, recommendation and discussion), from various movie reviews sources such as MovieLens and OMDb.
- **Paper:** [arXiv](https://arxiv.org/pdf/1511.06931.pdf)
- **Data:**  https://research.fb.com/downloads/babi/
- **Motivation:** This is one of the popular dialog datasets, a part of Facebook Research's "bAbI project".

### Checkbox

- [x] Create the dataset script `/datasets/my_dataset/my_dataset.py` using the template
- [x] Fill the `_DESCRIPTION` and `_CITATION` variables
- [x] Implement `_infos()`, `_split_generators()` and `_generate_examples()`
- [x] Make sure that the `BUILDER_CONFIGS` class attribute is filled with the different configurations of the dataset and that the `BUILDER_CONFIG_CLASS` is specified if there is a custom config class.
- [x] Generate the metadata file `dataset_infos.json` for all configurations
- [x] Generate the dummy data `dummy_data.zip` files to have the dataset script tested and that they don't weigh too much (<50KB)
- [x] Add the dataset card `README.md` using the template : fill the tags and the various paragraphs
- [x] Both tests for the real data and the dummy data pass.


**Note**: I haven't included the following from the data files: `entities` (the file containing list of all entities in the first three subtasks), `dictionary`(the dictionary of words they use in their models), `movie_kb`(contains the knowledge base of information about the movies, actors and other entities that are mentioned in the dialogs). Please let me know if those are needed, and if yes, should I make separate configurations for them?
GitHub issue #2050: Build custom dataset to fine-tune Wav2Vec2
State: closed | Created: 2021-03-13T22:01:10Z

Thank you for your recent tutorial on how to finetune Wav2Vec2 on a custom dataset. The example you gave here (https://huggingface.co/blog/fine-tune-xlsr-wav2vec2) was on the CommonVoice dataset. However, what if I want to load my own dataset?  I have a manifest (transcript and their audio files) in a JSON file.
GitHub issue #2049: Fix text-classification tags
State: closed | Created: 2021-03-13T19:51:42Z

There are different tags for text classification right now: `text-classification` and `text_classification`:
![image](https://user-images.githubusercontent.com/29076344/111042457-856bdf00-8463-11eb-93c9-50a30106a1a1.png).

This PR fixes it.
GitHub issue #2048: github is not always available - probably need a back up
State: closed | Created: 2021-03-13T18:03:32Z

Yesterday morning github wasn't working:

```
:/tmp$ wget https://raw.githubusercontent.com/huggingface/datasets/1.4.1/metrics/sacrebleu/sacrebleu.py--2021-03-12 18:35:59-- https://raw.githubusercontent.com/huggingface/datasets/1.4.1/metrics/sacrebleu/sacrebleu.py
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.111.133, 185.199.109.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2021-03-12 18:36:11 ERROR 500: Internal Server Error.
```

Suggestion: have a failover system and replicate the data on another system and reach there if gh isn't reachable? perhaps gh can be a master and the replicate a slave - so there is only one true source.
GitHub issue #2047: Multilingual dIalogAct benchMark (miam)
State: closed | Created: 2021-03-12T23:02:55Z

My collaborators (@EmileChapuis, @PierreColombo) and I within the Affective Computing team at Telecom Paris would like to anonymously publish the miam dataset. It is assocated with a publication currently under review. We will update the dataset with full citations once the review period is over.
GitHub issue #2046: add_faisis_index  gets very slow when doing it interatively  
State: closed | Created: 2021-03-12T20:27:18Z

As the below code suggests, I want to run add_faisis_index in every nth interaction from the training loop. I have 7.2 million documents. Usually, it takes 2.5 hours (if I run an as a separate process similar to the script given in rag/use_own_knowleldge_dataset.py). Now, this takes usually 5hrs. Is this normal?   Any way to make this process faster? 

@lhoestq 

```
   def training_step(self, batch, batch_idx) -> Dict:

    
        if (not batch_idx==0) and (batch_idx%5==0):

            print("******************************************************")
            ctx_encoder=self.trainer.model.module.module.model.rag.ctx_encoder
            model_copy =type(ctx_encoder)(self.config_dpr) # get a new instance  #this will be load in the CPU
            model_copy.load_state_dict(ctx_encoder.state_dict()) # copy weights and stuff


            list_of_gpus = ['cuda:2','cuda:3']
            c_dir='/custom/cache/dir'

            kb_dataset = load_dataset("csv", data_files=[self.custom_config.csv_path], split="train", delimiter="\t", column_names=["title", "text"],cache_dir=c_dir) 

            print(kb_dataset)

      
            n=len(list_of_gpus) #nunber of dedicated GPUs
            kb_list=[kb_dataset.shard(n, i, contiguous=True) for i in range(n)]

            #kb_dataset.save_to_disk('/hpc/gsir059/MY-Test/RAY/transformers/examples/research_projects/rag/haha-dir')


            print(self.trainer.global_rank)
            dataset_shards = self.re_encode_kb(model_copy.to(device=list_of_gpus[self.trainer.global_rank]),kb_list[self.trainer.global_rank])
            output = [None for _ in list_of_gpus]

            #self.trainer.accelerator_connector.accelerator.barrier("embedding_process")
            dist.all_gather_object(output, dataset_shards)
            

            #This creation and re-initlaization of the new index
            if (self.trainer.global_rank==0):  #saving will be done in the main process 
           
                combined_dataset = concatenate_datasets(output)
    
                passages_path =self.config.passages_path

                logger.info("saving the dataset with  ")
                #combined_dataset.save_to_disk('/hpc/gsir059/MY-Test/RAY/transformers/examples/research_projects/rag/MY-Passage')
                combined_dataset.save_to_disk(passages_path)
                logger.info("Add faiss index to the dataset that consist of embeddings") 

    
                embedding_dataset=combined_dataset
                index = faiss.IndexHNSWFlat(768, 128, faiss.METRIC_INNER_PRODUCT)
                embedding_dataset.add_faiss_index("embeddings", custom_index=index)

                embedding_dataset.get_index("embeddings").save(self.config.index_path)
GitHub issue #2045: Preserve column ordering in Dataset.rename_column
State: closed | Created: 2021-03-12T18:26:47Z

Currently `Dataset.rename_column` doesn't necessarily preserve the order of the columns:
```python
>>> from datasets import Dataset
>>> d = Dataset.from_dict({'sentences': ["s1", "s2"], 'label': [0, 1]})
>>> d
Dataset({
    features: ['sentences', 'label'],
    num_rows: 2
})
>>> d.rename_column('sentences', 'text')
Dataset({
    features: ['label', 'text'],
    num_rows: 2
})
```
This PR fixes this.
GitHub issue #2044: Add CBT dataset
State: closed | Created: 2021-03-12T18:04:19Z

This PR adds the [CBT Dataset](https://arxiv.org/abs/1511.02301).

Note that I have also added the `raw` dataset as a separate configuration. I couldn't find a suitable "task" for it in YAML tags.

The dummy files have one example each, as the examples are slightly big. For `raw` dataset, I just used top few lines, because they are entire books and would take up a lot of space.

Let me know in case of any issues.