helmo/github-issues huggingface dataset:
Build custom dataset to fine-tune Wav2Vec2
State: closed | Author: Omarnabk (NONE)
Labels: name: dataset request

Thank you for your recent tutorial on how to finetune Wav2Vec2 on a custom dataset. The example you gave here (https://huggingface.co/blog/fine-tune-xlsr-wav2vec2) was on the CommonVoice dataset. However, what if I want to load my own dataset?  I have a manifest (transcript and their audio files) in a JSON file. 


Comments: @lhoestq - We could simply use the "general" json dataset for this no? 

Sure you can use the json loader
```python
data_files = {"train": "path/to/your/train_data.json", "test": "path/to/your/test_data.json"}
train_dataset = load_dataset("json", data_files=data_files, split="train")
test_dataset = load_dataset("json", data_files=data_files, split="test")
```

You just need to make sure that the data contain the paths to the audio files.
If not, feel free to use `.map()` to add them.

Many thanks! that was what I was looking for.
Fix text-classification tags
State: closed | Author: gchhablani (CONTRIBUTOR)

There are different tags for text classification right now: `text-classification` and `text_classification`:
![image](https://user-images.githubusercontent.com/29076344/111042457-856bdf00-8463-11eb-93c9-50a30106a1a1.png).

This PR fixes it.


Comments: LGTM, thanks for fixing.
github is not always available - probably need a back up
State: closed | Author: stas00 (CONTRIBUTOR)

Yesterday morning github wasn't working:

```
:/tmp$ wget https://raw.githubusercontent.com/huggingface/datasets/1.4.1/metrics/sacrebleu/sacrebleu.py--2021-03-12 18:35:59-- https://raw.githubusercontent.com/huggingface/datasets/1.4.1/metrics/sacrebleu/sacrebleu.py
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.111.133, 185.199.109.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2021-03-12 18:36:11 ERROR 500: Internal Server Error.
```

Suggestion: have a failover system and replicate the data on another system and reach there if gh isn't reachable? perhaps gh can be a master and the replicate a slave - so there is only one true source.
Multilingual dIalogAct benchMark (miam)
State: closed | Author: eusip (CONTRIBUTOR)

My collaborators (@EmileChapuis, @PierreColombo) and I within the Affective Computing team at Telecom Paris would like to anonymously publish the miam dataset. It is assocated with a publication currently under review. We will update the dataset with full citations once the review period is over.

Comments: Hello. All aforementioned changes have been made. I've also re-run black on miam.py. :-)

I will run isort again. Hopefully it resolves the current check_code_quality test failure.

Once the review period is over, feel free to open a PR to add all the missing information ;)

Hi! I will follow up right now with one more pull request as I have new anonymous citation information to include.
Preserve column ordering in Dataset.rename_column
State: closed | Author: mariosasko (COLLABORATOR)

Currently `Dataset.rename_column` doesn't necessarily preserve the order of the columns:
```python
>>> from datasets import Dataset
>>> d = Dataset.from_dict({'sentences': ["s1", "s2"], 'label': [0, 1]})
>>> d
Dataset({
    features: ['sentences', 'label'],
    num_rows: 2
})
>>> d.rename_column('sentences', 'text')
Dataset({
    features: ['label', 'text'],
    num_rows: 2
})
```
This PR fixes this.

Comments: Not sure why CI isn't triggered.

@lhoestq Can you please help me with this? 

I don't know how to trigger it manually, but an empty commit should do the job
Add CBT dataset
State: closed | Author: gchhablani (CONTRIBUTOR)

This PR adds the [CBT Dataset](https://arxiv.org/abs/1511.02301).

Note that I have also added the `raw` dataset as a separate configuration. I couldn't find a suitable "task" for it in YAML tags.

The dummy files have one example each, as the examples are slightly big. For `raw` dataset, I just used top few lines, because they are entire books and would take up a lot of space.

Let me know in case of any issues.

Comments: Hi @lhoestq,

I have added changes from the review.

Thanks for approving @lhoestq
Support pickle protocol for dataset splits defined as ReadInstruction
State: closed | Author: mariosasko (COLLABORATOR)

Fixes #2022 (+ some style fixes) 

Comments: @lhoestq But we don't perform conversion to a `NamedSplit` if `_split` is not a string which means it **will** be a `ReadInstruction` after reloading.

Yes right ! I read it wrong.
Perfect then
Fix arrow memory checks issue in tests
State: closed | Author: lhoestq (MEMBER)

The tests currently fail on `master` because the arrow memory verification doesn't return the expected memory evolution when loading an arrow table in memory.
From my experiments, the tests fail only when the full test suite is ran.
This made me think that maybe some arrow objects from other tests were not freeing their memory until they do and cause the memory verifications to fail in other tests.

Collecting the garbage collector before checking the arrow memory usage seems to fix this issue.
I added a context manager `assert_arrow_memory_increases` that we can use in tests and that deals with the gc.
ValueError: datasets' indices [1] come from memory and datasets' indices [0] come from disk
State: closed | Author: simonschoe (NONE)

Hi there,

I am trying to concat two datasets that I've previously saved to disk via `save_to_disk()` like so (note that both are saved as `DataDict`, `PATH_DATA_CLS_*` are `Path`-objects):
```python
concatenate_datasets([load_from_disk(PATH_DATA_CLS_A)['train'], load_from_disk(PATH_DATA_CLS_B)['train']])
```
Yielding the following error:
```python
ValueError: Datasets' indices should ALL come from memory, or should ALL come from disk.
However datasets' indices [1] come from memory and datasets' indices [0] come from disk.
```
Been trying to solve this for quite some time now. Both `DataDict` have been created by reading in a `csv` via `load_dataset` and subsequently processed using the various `datasets` methods (i.e. filter, map, remove col, rename col). Can't figure out tho...

`load_from_disk(PATH_DATA_CLS_A)['train']` yields:
```python
Dataset({
    features: ['labels', 'text'],
    num_rows: 785
})
```
`load_from_disk(PATH_DATA_CLS_B)['train']` yields:
```python
Dataset({
    features: ['labels', 'text'],
    num_rows: 3341
})
```

Comments: Hi ! To help me understand the situation, can you print the values of  `load_from_disk(PATH_DATA_CLS_A)['train']._indices_data_files` and `load_from_disk(PATH_DATA_CLS_B)['train']._indices_data_files` ?
They should both have a path to an arrow file

Also note that from #2025 concatenating datasets will no longer have such restrictions.

Sure, thanks for the fast reply!

For dataset A: `[{'filename': 'drive/MyDrive/data_target_task/dataset_a/train/cache-4797266bf4db1eb7.arrow'}]`
For dataset B: `[]`

No clue why for B it returns nothing. `PATH_DATA_CLS_B` is exactly the same in `save_to_disk` and `load_from_disk`... Also I can verify that the folder physically exists under 'drive/MyDrive/data_target_task/dataset_b/'

In the next release you'll be able to concatenate any kinds of dataset (either from memory or from disk).

For now I'd suggest you to flatten the indices of the A and B datasets. This will remove the indices mapping and you will be able to concatenate them. You can flatten the indices with
```python
dataset = dataset.flatten_indices()
```

Indeed this works. Not the most elegant solution, but it does the trick. Thanks a lot!
outdated dataset_infos.json might fail verifications
State: closed | Author: songfeng (CONTRIBUTOR)

The [doc2dial/dataset_infos.json](https://github.com/huggingface/datasets/blob/master/datasets/doc2dial/dataset_infos.json) is outdated. It would fail data_loader when verifying download checksum etc..

Could you please update this file or point me how to update this file?

Thank you.

Comments: Hi ! Thanks for reporting.

To update the dataset_infos.json you can run:
```
datasets-cli test ./datasets/doc2dial --all_configs --save_infos --ignore_verifications
```

Fixed by #2041, thanks again @songfeng !
Fix: Wikipedia - save memory by replacing root.clear with elem.clear
State: closed | Author: miyamonz (CONTRIBUTOR)

see: https://github.com/huggingface/datasets/issues/2031

What I did:
- replace root.clear with elem.clear
- remove lines to get root element
- $ make style
- $ make test
  - some tests required some pip packages, I installed them.

test results on origin/master and my branch are same. I think it's not related on my modification, isn't it?
```
==================================================================================== short test summary info ====================================================================================
FAILED tests/test_arrow_writer.py::TypedSequenceTest::test_catch_overflow - AssertionError: OverflowError not raised
============================================================= 1 failed, 2332 passed, 5138 skipped, 70 warnings in 91.75s (0:01:31) ==============================================================
make: *** [Makefile:19: test] Error 1

```

Is there anything else I should do?

Comments: The error you got is minor and appeared in the last version of pyarrow, we'll fix the CI to take this into account. You can ignore it
Cannot load wikitext
State: closed | Author: Gpwner (NONE)

when I execute these codes
```
>>> from datasets import load_dataset
>>> test_dataset = load_dataset("wikitext")
```

I got an error,any help?

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/xxx/anaconda3/envs/transformer/lib/python3.7/site-packages/datasets/load.py", line 589, in load_dataset
    path, script_version=script_version, download_config=download_config, download_mode=download_mode, dataset=True
  File "/home/xxx/anaconda3/envs/transformer/lib/python3.7/site-packages/datasets/load.py", line 267, in prepare_module
    local_path = cached_path(file_path, download_config=download_config)
  File "/home/xxx/anaconda3/envs/transformer/lib/python3.7/site-packages/datasets/utils/file_utils.py", line 308, in cached_path
    use_etag=download_config.use_etag,
  File "/home/xxx/anaconda3/envs/transformer/lib/python3.7/site-packages/datasets/utils/file_utils.py", line 487, in get_from_cache
    raise ConnectionError("Couldn't reach {}".format(url))
ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.1.3/datasets/wikitext/wikitext.py
```

Comments: Solved!
Raise an error for outdated sacrebleu versions
State: closed | Author: lhoestq (MEMBER)

The `sacrebleu` metric seem to only work for sacrecleu>=1.4.12

For example using sacrebleu==1.2.10, an error is raised (from metric/sacrebleu/sacrebleu.py):
```python
    def _compute(
        self,
        predictions,
        references,
        smooth_method="exp",
        smooth_value=None,
        force=False,
        lowercase=False,
        tokenize=scb.DEFAULT_TOKENIZER,
        use_effective_order=False,
    ):
        references_per_prediction = len(references[0])
        if any(len(refs) != references_per_prediction for refs in references):
            raise ValueError("Sacrebleu requires the same number of references for each prediction")
        transformed_references = [[refs[i] for refs in references] for i in range(references_per_prediction)]
>       output = scb.corpus_bleu(
            sys_stream=predictions,
            ref_streams=transformed_references,
            smooth_method=smooth_method,
            smooth_value=smooth_value,
            force=force,
            lowercase=lowercase,
            tokenize=tokenize,
            use_effective_order=use_effective_order,
        )

E       TypeError: corpus_bleu() got an unexpected keyword argument 'smooth_method'
/mnt/cache/modules/datasets_modules/metrics/sacrebleu/b390045b3d1dd4abf6a95c4a2a11ee3bcc2b7620b076204d0ddc353fa649fd86/sacrebleu.py:114: TypeError
```

I improved the error message when users have an outdated version of sacrebleu.
The new error message tells the user to update sacrebleu.
cc @LysandreJik
Use Arrow filtering instead of writing a new arrow file for Dataset.filter
State: closed | Author: lhoestq (MEMBER)
Labels: name: enhancement

Currently the filter method reads the dataset batch by batch to write a new, filtered, arrow file on disk. Therefore all the reading + writing can take some time.

Using a mask directly on the arrow table doesn't do any read or write operation therefore it's significantly quicker.

I think there are two cases:
- if the dataset doesn't have an indices mapping, then one can simply use the arrow filtering on the main arrow table `dataset._data.filter(...)`
- if the dataset an indices mapping, then the mask should be applied on the indices mapping table `dataset._indices.filter(...)`

The indices mapping is used to map between the idx at `dataset[idx]` in `__getitem__` and the idx in the actual arrow table.

The new filter method should therefore be faster, and allow users to pass either a filtering function (that returns a boolean given an example), or directly a mask.

Feel free to discuss this idea in this thread :)

One additional note: the refactor at #2025 would make all the pickle-related stuff work directly with the arrow filtering, so that we only need to change the Dataset.filter method without having to deal with pickle.

cc @theo-m @gchhablani 

related issues: #1796 #1949 

Comments: Actually table.filter returns a new table in memory, which can fill users RAM.

Therefore it's not a good solution if we want to keep supporting bigger than RAM datastes
wikipedia.py generator that extracts XML doesn't release memory
State: closed | Author: miyamonz (CONTRIBUTOR)

I tried downloading Japanese wikipedia, but it always failed because of out of memory maybe.

I found that the generator function that extracts XML data in wikipedia.py doesn't release memory in the loop.

https://github.com/huggingface/datasets/blob/13a5b7db992ad5cf77895e4c0f76595314390418/datasets/wikipedia/wikipedia.py#L464-L502

`root.clear()` intend to clear memory, but it doesn't.
https://github.com/huggingface/datasets/blob/13a5b7db992ad5cf77895e4c0f76595314390418/datasets/wikipedia/wikipedia.py#L490
https://github.com/huggingface/datasets/blob/13a5b7db992ad5cf77895e4c0f76595314390418/datasets/wikipedia/wikipedia.py#L494
I replaced them with `elem.clear()`, then it seems to work correctly.

here is the notebook to reproduce it.
https://gist.github.com/miyamonz/dc06117302b6e85fa51cbf46dde6bb51#file-xtract_content-ipynb

Comments: Hi @miyamonz 
Thanks for investigating this issue, good job !
It would be awesome to integrate your fix in the library, could you open a pull request ?

OK! I'll send it later.
Add Romanian to XQuAD
State: closed | Author: M-Salti (CONTRIBUTOR)

On Jan 18, XQuAD was updated with a new Romanian validation file ([xquad commit link](https://github.com/deepmind/xquad/commit/60cac411649156efb6aab9dd4c9cde787a2c0345))


Comments: Hi ! Thanks for updating XQUAD :)

The slow test is failing though since there's no dummy data nor metadata in dataset_infos.json for the romanian configuration.

Could you please generate the dummy data with
```
datasets-cli dummy_data ./datasets/xquad --auto_generate --json_field data
```
This will update all the dummy data files, and also add the new one for the romanian configuration.


You can also update the metadata with
```
datasets-cli test ./datasets/xquad --name xquad.ro --save_infos
```
This will update the dataset_infos.json file with the metadata of the romanian config :)

Thanks in advance !

Hello Quentin, and thanks for your help.

I found that running

```python
datasets-cli test ./datasets/xquad --name xquad.ro --save_infos
```

was not enough to pass the slow tests, because it was not adding the new `xquad.ro.json` checksum to the other configs infos and becuase of that an `UnexpectedDownloadedFile` error was being thrown, so instead I used:

```python
datasets-cli test ./datasets/xquad --save_infos --all_configs --ignore_verifications
```

`--ignore_verifications` was necessary to bypass the same `UnexpectedDownloadedFile` error.

Additionally, I deleted `dummy_data_copy.zip` and the `copy.sh` script because they both seem now unnecessary.

The slow tests for both the real and dummy data now pass successfully, so I hope that I didn't mess anything up :)


You're right, you needed the `--ignore_verifications` flag !
Thanks for updating them :)

Although I just noticed that the new dummy_data.zip files are quite big (170KB each) because they contain the json files of all the languages, while only one json file per language is necessary. Could you remove the unnecessary json files to reduce the size of the dummy_data.zip files if you don't mind ?

Done. I created a script (`remove_unnecessary_langs.sh`) to automate the process.
ValueError when rename_column on splitted dataset
State: closed | Author: simonschoe (NONE)

Hi there,
I am loading `.tsv` file via `load_dataset` and subsequently split the rows into training and test set via the `ReadInstruction` API like so:

```python
split = {
    'train': ReadInstruction('train', to=90, unit='%'),
    'test': ReadInstruction('train', from_=-10, unit='%')
}

dataset = load_dataset(
    path='csv',               # use 'text' loading script to load from local txt-files
    delimiter='\t',           # xxx
    data_files=text_files,    # list of paths to local text files
    split=split,              # xxx
)

dataset
```

Part of output:
```python
DatasetDict({
    train: Dataset({
        features: ['sentence', 'sentiment'],
        num_rows: 900
    })
    test: Dataset({
        features: ['sentence', 'sentiment'],
        num_rows: 100
    })
})
```
Afterwards I'd like to rename the 'sentence' column to 'text' in order to be compatible with my modelin pipeline. If I run the following code I experience a `ValueError` however:
```python
dataset['train'].rename_column('sentence', 'text')
```
```python
/usr/local/lib/python3.7/dist-packages/datasets/splits.py in __init__(self, name)
    353         for split_name in split_names_from_instruction:
    354             if not re.match(_split_re, split_name):
--> 355                 raise ValueError(f"Split name should match '{_split_re}'' but got '{split_name}'.")
    356 
    357     def __str__(self):

ValueError: Split name should match '^\w+(\.\w+)*$'' but got 'ReadInstruction('.
```
In particular, these behavior does not arise if I use the deprecated `rename_column_` method. Any idea what causes the error? Would assume something in the way I defined the split.

Thanks in advance! :)

Comments: Hi,

This is a bug so thanks for reporting it. `Dataset.__setstate__`  is the problem, which is called when `Dataset.rename_column` tries to copy the dataset with `copy.deepcopy(self)`. This only happens if the `split` arg in `load_dataset` was defined as `ReadInstruction`.

To overcome this issue, use the named splits API (for now):
```python
train_ds, test_ds = load_dataset(
    path='csv',               
    delimiter='\t',          
    data_files=text_files,    
    split=['train[:90%]', 'train[-10%:]'],
)

train_ds = train_ds.rename_column('sentence', 'text')
```

This has been fixed in #2043 , thanks @mariosasko 
The fix is available on master and we'll do a new release soon :)

feel free to re-open if you still have issues
Interactively doing  save_to_disk and load_from_disk corrupts the datasets object?
State: closed | Author: shamanez (NONE)

 dataset_info.json file saved after using  save_to_disk gets corrupted as follows. 
 
 
![image](https://user-images.githubusercontent.com/16892570/110568474-ed969880-81b7-11eb-832f-2e5129656016.png)

Is there a way to disable the cache that will save to /tmp/huggiface/datastes ? 
I have a feeling there is a serious issue with cashing.

Comments: Hi,

Can you give us a minimal reproducible example? This [part](https://huggingface.co/docs/datasets/master/processing.html#controling-the-cache-behavior) of the docs explains how to control caching.
Replace print with logging in dataset scripts
State: closed | Author: mariosasko (COLLABORATOR)

Replaces `print(...)` in the dataset scripts with the library logger.

Comments: @lhoestq Maybe a script or even a test in `test_dataset_common.py` that verifies that a dataset script meets some set of quality standards (print calls and todos from the dataset script template are not present, etc.) could be added?

Yes definitely !
Md gender card update
State: closed | Author: mcmillanmajora (CONTRIBUTOR)

I updated the descriptions of the datasets as they appear in the HF repo and the descriptions of the source datasets according to what I could find from the paper and the references. I'm still a little unclear about some of the fields of the different configs, and there was little info on the word list and name list. I'll contact the authors to see if they have any additional information or suggested changes.

Comments: Link to the card: https://github.com/mcmillanmajora/datasets/blob/md-gender-card/datasets/md_gender_bias/README.md

dataset card* @sgugger :p 

Ahah that's what I wanted to say @lhoestq, thanks for fixing. Not used to review the Datasets side ;-)
Add TF-based Features to handle different modes of data
State: closed | Author: gchhablani (CONTRIBUTOR)

Hi,

I am creating this draft PR to work on add features similar to [TF datasets](https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/core/features). I'll be starting with `Tensor` and `FeatureConnector` classes, and build upon them to add other features as well. This is a work in progress.
No upstream branch
State: closed | Author: theo-m (CONTRIBUTOR)
Labels: name: documentation

Feels like the documentation on adding a new dataset is outdated?

https://github.com/huggingface/datasets/blob/987df6b4e9e20fc0c92bc9df48137d170756fd7b/ADD_NEW_DATASET.md#L49-L54

There is no upstream branch on remote. 

Comments: What's the issue exactly ?

Given an `upstream` remote repository with url `https://github.com/huggingface/datasets.git`, you can totally rebase from `upstream/master`.

It's mentioned at the beginning how to add the `upstream` remote repository

https://github.com/huggingface/datasets/blob/987df6b4e9e20fc0c92bc9df48137d170756fd7b/ADD_NEW_DATASET.md#L10-L14

~~What difference is there with the default `origin` remote that is set when you clone the repo?~~ I've just understood that this applies to **forks** of the repo 🤡