AniketTec/github-issues huggingface dataset:
Title: why the stratify option is omitted from test_train_split function?
State: closed
Labels: description: New feature or request
name: enhancement

description: Issues a bit more difficult than "Good First" issues
name: good second issue

why the stratify option is omitted from test_train_split function?

is there any other way implement the stratify option while splitting the dataset? as it is important point to be considered while splitting the dataset.

Comments:
Hi ! It's simply not added yet :)

If someone wants to contribute to add the `stratify` parameter I'd be happy to give some pointers.

In the meantime, I guess you can use `sklearn` or other tools to do a stratified train/test split over the **indices** of your dataset and then do
```
train_dataset = dataset.select(train_indices)
test_dataset = dataset.select(test_indices)
```

Hi @lhoestq I would like to add `stratify` parameter, can you give me some pointers for adding the same ?

Hi ! Sure :)

The `train_test_split` method is defined here: 

https://github.com/huggingface/datasets/blob/dc62232fa1b3bcfe2fbddcb721f2d141f8908943/src/datasets/arrow_dataset.py#L3253-L3253

and inside `train_test_split ` we need to create the right `train_indices` and `test_indices` that are passed here to `.select()`:

https://github.com/huggingface/datasets/blob/dc62232fa1b3bcfe2fbddcb721f2d141f8908943/src/datasets/arrow_dataset.py#L3450-L3464

For example if your dataset is like
|    |   label |
|---:|--------:|
|  0 |       1 |
|  1 |       1 |
|  2 |       0 |
|  3 |       0 |

and the user passes `stratify=dataset["label"]`, then you should get indices that look like this
```
train_indices = [0, 2]
test_indices = [1, 3]
```

these indices will be passed to `.select` to return the stratified train and test splits :)

Feel free to îng me if you have any question !

@lhoestq 
I just added the implementation for `stratify` option here #4322
Title: [Staging] Update dataset repos automatically on the Hub
State: closed

Let's have a script that updates the dataset repositories on staging for now. This way we can make sure it works fine before going in prod.

Related to https://github.com/huggingface/datasets/issues/3341

The script runs on each commit on `master`. It checks the datasets that were changed, and it pushes the changes to the corresponding repositories on the Hub.

If there's a new dataset, then a new repository is created.

If the commit is a new release of `datasets`, it also pushes the tag to all the repositories.

Comments:
do keep us updated on how it's going in staging! cc @SBrandeis 

Sure ! For now it works smoothly. We'll also do a new release today.

I can send you some repos to explore on staging, in case you want to see how they look like after being updated.
For example [swahili_news](https://moon-staging.huggingface.co/datasets/swahili_news/tree/main)
Title: Unexpected behavior doing Split + Filter
State: closed
Labels: description: Something isn't working
name: bug

## Describe the bug
I observed unexpected behavior when applying 'train_test_split' followed by 'filter' on dataset. Elements of the training dataset eventually end up in the test dataset (after applying the 'filter') 

## Steps to reproduce the bug
```
from datasets import Dataset
import pandas as pd
dic = {'x': [1,2,3,4,5,6,7,8,9], 'y':['q','w','e','r','t','y','u','i','o']}
df = pd.DataFrame.from_dict(dic)
dataset = Dataset.from_pandas(df)
split_dataset = dataset.train_test_split(test_size=0.5, shuffle=False, seed=42)
train_dataset = split_dataset["train"]
eval_dataset = split_dataset["test"]
eval_dataset_2 = eval_dataset.filter(lambda example: example['x'] % 2 == 0)
print( eval_dataset['x'])
print(eval_dataset_2['x'])
```
One observes that elements in eval_dataset2 are actually coming from the training dataset...

## Expected results
The expected results would be that the filtered eval dataset would only contain elements from the original eval dataset.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: Windows 10
- Python version: 3.7
- PyArrow version: 5.0.0


Comments:
Hi ! This is an issue with `datasets` 1.12. Sorry for the inconvenience. Can you update to `>=1.13` ?
see https://github.com/huggingface/datasets/issues/3190

Maybe we should also backport the bug fix to `1.12` (in a new version `1.12.2`)
Title: Add `__add__()`, `__iadd__()` and similar to `Dataset` class
State: closed
Labels: description: New feature or request
name: enhancement

description: Generic discussion on the library
name: generic discussion

**Is your feature request related to a problem? Please describe.**
No.

**Describe the solution you'd like**
I would like to be able to concatenate datasets as follows:
```python
>>> dataset["train"] += dataset["validation"]
```

... instead of using `concatenate_datasets()`:
```python
>>> raw_datasets["train"] = concatenate_datasets([raw_datasets["train"], raw_datasets["validation"]])
>>> del raw_datasets["validation"]
```

**Describe alternatives you've considered**
Well, I have considered `concatenate_datasets()` 😀

**Additional context**
N.a.


Comments:
I was going through the codebase, and I believe the implementation of __add__() and __iadd__() will be similar to concatenate_datasets() after the elimination of code for arguments other than the list of datasets (info, split, axis). 
(Assuming elimination of axis means concatenating over axis 1.)

Most data frame libraries (Polars, Pandas, etc.) override `__add__` to perform (mathematical) summation, so having different behavior could lead to confusion.
Title: JSONDecodeError with HuggingFace dataset viewer
State: closed
Labels: description: Related to the dataset viewer on huggingface.co
name: dataset-viewer

## Dataset viewer issue for 'pubmed_neg'

**Link:** https://huggingface.co/datasets/IGESML/pubmed_neg

I am getting the error: 
Status code:   400
Exception:     JSONDecodeError
Message:       Expecting property name enclosed in double quotes: line 61 column 2 (char 1202)

I have checked all files - I am not using single quotes anywhere. Not sure what is causing this issue.

Am I the one who added this dataset ? Yes


Comments:
Hi ! I think the issue comes from the dataset_infos.json file: it has the "flat" field twice.

Can you try deleting this file and regenerating it please ?

Thanks! That fixed that, but now I am getting:
Server Error
Status code:   400
Exception:     KeyError
Message:       'feature'

I checked the dataset_infos.json and pubmed_neg.py script, I don't use 'feature' anywhere as a key. Is the dataset viewer expecting that I do?

It seems that the `feature` key is missing from some feature type definition in your dataset_infos.json:
```json
			"tokens": {
				"dtype": "list",
				"id": null,
				"_type": "Sequence"
			},
			"tags": {
				"dtype": "list",
				"id": null,
				"_type": "Sequence"
			}
```
They should be
```json
			"tokens": {
				"dtype": "list",
				"id": null,
				"_type": "Sequence"
                                 "feature": {"dtype": "string", "id": null, "_type": "Value"}
			},
			"tags": {
				"dtype": "list",
				"id": null,
				"_type": "Sequence",
                                 "feature": {"num_classes": 5, "names": ["-", "S", "H", "N", "C"], "names_file": null, "id": null, "_type": "ClassLabel"}
			}
```

Note that you can generate the dataset_infos.json automatically to avoid mistakes:
```bash
datasets-cli test ./path/to/dataset --save_infos
```
Title: Remove redundant local path information in audio/image datasets
State: closed
Labels: description: Contribution to a dataset script
name: dataset contribution

Remove the redundant path information in the audio/image dataset as discussed in https://github.com/huggingface/datasets/pull/3430#issuecomment-994734828

TODOs:
* [ ] merge https://github.com/huggingface/datasets/pull/3430
* [ ] merge https://github.com/huggingface/datasets/pull/3364
* [ ] re-generate the info files of the updated audio datasets

cc:  @patrickvonplaten @anton-l @nateraw (I expect this to break the audio/vision examples in Transformers; after this change you'll be able to access underlying paths as follows `dset = dset.cast_column("audio", Audio(..., decode=False)); path = dset[0]["audio"]`)


Comments:
Cool, I'm in favor of this PR. Our official examples in speech already make use of `"audio"` so no need to change anything there. It would be great if we could prominently feature how one can get the audio path without decoding in the docs.

@patrickvonplaten Yes, I agree.

cc @stevhliu we should add an example where decoding is disabled (to read paths) to  [this section](https://github.com/huggingface/datasets/blob/master/docs/source/audio_process.rst#audio-datasets) in the docs and remove the mentions of `path`/`file` (if we merge this PR).

Thanks @mariosasko. Are you planning to finish this before we remove the dataset scripts from GitHub?
Title: Add QuALITY dataset
State: open
Labels: description: Requesting to add a new dataset
name: dataset request

## Adding a Dataset
- **Name:** QuALITY
- **Description:** A challenging question answering with very long contexts (Twitter [thread](https://twitter.com/sleepinyourhat/status/1471225421794529281?s=20))
- **Paper:** No ArXiv link yet, but draft is [here](https://github.com/nyu-mll/quality/blob/main/quality_preprint.pdf)
- **Data:** GitHub repo [here](https://github.com/nyu-mll/quality)
- **Motivation:** This dataset would serve as a nice way to benchmark long-range Transformer models like BigBird, Longformer and their descendants. In particular, it would be very interesting to see how the S4 model fares on this given it's impressive performance on the Long Range Arena

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Comments:
I'll take this one if no one hasn't yet!
Title: Update BLEURT hyperlink
State: closed

The description of BLEURT on the hf.co website has a strange use of URL hyperlinking. This PR attempts to fix this, although I am not 100% sure Markdown syntax is allowed on the frontend or not.

![Screen Shot 2021-12-15 at 17 31 27](https://user-images.githubusercontent.com/26859204/146226432-c83cbdaf-f57d-4999-b53c-85da718ff7fb.png)




Comments:
seems like a very very low-prio improvement :)

@albertvillanova thanks for the feedback! I removed the formatting altogether since I think this is a bit simpler tor read than non-rendered Markdown
Title: Add The People's Speech
State: closed
Labels: description: Requesting to add a new dataset
name: dataset request

name: speech

## Adding a Dataset
- **Name:** The People's Speech
- **Description:** a massive English-language dataset of audio transcriptions of full sentences. 
- **Paper:** https://openreview.net/pdf?id=R8CwidgJ0yT
- **Data:** https://mlcommons.org/en/peoples-speech/
- **Motivation:** With over 30,000 hours of speech, this dataset is the largest and most diverse freely available English speech recognition corpus today.


[The article](https://thegradient.pub/new-datasets-to-democratize-speech-recognition-technology-2/) which may be useful when working on the dataset.

cc: @anton-l 

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Comments:
This dataset is now available on the Hub here: https://huggingface.co/datasets/MLCommons/peoples_speech
Title: Add Multilingual Spoken Words dataset
State: closed
Labels: description: Requesting to add a new dataset
name: dataset request

name: speech

## Adding a Dataset
- **Name:** Multilingual Spoken Words
- **Description:** Multilingual Spoken Words Corpus is a large and growing audio dataset of spoken words in 50 languages for academic research and commercial applications in keyword spotting and spoken term search, licensed under CC-BY 4.0. The dataset contains more than 340,000 keywords, totaling 23.4 million 1-second spoken examples (over 6,000 hours). 

  Read more: https://mlcommons.org/en/news/spoken-words-blog/
- **Paper:** https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/file/fe131d7f5a6b38b23cc967316c13dae2-Paper-round2.pdf
- **Data:** https://mlcommons.org/en/multilingual-spoken-words/
- **Motivation:** 

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


Comments:
Title: Unable to resolve any data file after loading once
State: closed

when I rerun my program, it occurs this error
" Unable to resolve any data file that matches '['**train*']' at /data2/whr/lzy/open_domain_data/retrieval/wiki_dpr with any supported extension ['csv', 'tsv', 'json', 'jsonl', 'parquet', 'txt', 'zip']", so how could i deal with this problem?
thx.
And below is my code .
![image](https://user-images.githubusercontent.com/84694183/146023446-d75fdec8-65c1-484f-80d8-6c20ff5e994b.png)


Comments:
Hi ! `load_dataset` accepts as input either a local dataset directory or a dataset name from the Hugging Face Hub.

So here you are getting this error the second time because it tries to load the local `wiki_dpr` directory, instead of `wiki_dpr` from the Hub. It doesn't work since it's a **cache** directory, not a **dataset** directory in itself.

To fix that you can use another cache directory like `cache_dir="/data2/whr/lzy/open_domain_data/retrieval/cache"`

thx a lot
Title: Make cast cacheable (again) on Windows
State: closed

`cast` currently emits the following warning when called on Windows:
```
Parameter 'function'=<function Dataset.cast.<locals>.<lambda> at 0x000001C930571EA0> of the transform datasets.arrow_dataset.Dataset._map_single couldn't be hashed properly, a random hash was used instead. Make sure your transforms and parameters are serializable with pickle or dill for the dataset fingerprinting 
and caching to work. If you reuse this transform, the caching mechanism will consider it to be different 
from the previous calls and recompute everything. This warning is only showed once. Subsequent hashing failures won't be showed.
```

It seems like the issue stems from the `config.PYARROW_VERSION` object not being serializable on Windows (tested with `dumps(lambda: config.PYARROW_VERSION)`), so I'm fixing this by capturing `config.PYARROW_VERSION.major` before the lambda definition.

Comments:
Title: Getting configs names takes too long
State: open
Labels: description: Something isn't working
name: bug


## Steps to reproduce the bug
```python
from datasets import get_dataset_config_names
get_dataset_config_names("allenai/c4")
```

## Expected results

I would expect to get the answer quickly, at least in less than 10s

## Actual results

It takes about 45s on my environment

## Environment info

- `datasets` version: 1.16.1
- Platform: Linux-5.11.0-1022-aws-x86_64-with-glibc2.31
- Python version: 3.9.6
- PyArrow version: 4.0.1

Comments:
maybe related to https://github.com/huggingface/datasets/issues/2859


It looks like it's currently calling `HfFileSystem.ls()` ~8 times at the root and for each subdirectory:
- ""
- "en.noblocklist"
- "en.noclean"
- "en"
- "multilingual"
- "realnewslike"

Currently `ls` is slow because it iterates on all the files inside the repository.

An easy optimization would be to cache the result of each call to `ls`.
We can also optimize `ls` by using a tree structure per directory instead of a list of all the files.


ok
Title: Add RedCaps dataset
State: closed

Add the RedCaps dataset. I'm not adding the generated `dataset_infos.json` file for now due to its size (11 MB). 

TODOs:
- [x] dummy data
- [x] dataset card

Close #3316 

Comments:
Cool ! If you want you can include `dataset_infos.json` but only for the main configurations. That's what we do for example for translation datasets when there are too many configs

@lhoestq I've added an example that uses `map` to download the images.
Title: Error about load_metric
State: closed
Labels: description: Something isn't working
name: bug

## Describe the bug
  File "/opt/conda/lib/python3.8/site-packages/datasets/load.py", line 1371, in load_metric
    metric = metric_cls(
TypeError: 'NoneType' object is not callable

## Steps to reproduce the bug
```python
metric = load_metric("glue", "sst2")
```


## Environment info
- `datasets` version: 1.16.1
- Platform: Linux-4.15.0-161-generic-x86_64-with-glibc2.10
- Python version: 3.8.3
- PyArrow version: 6.0.1


Comments:
Hi ! I wasn't able to reproduce your error.

Can you try to clear your cache at `~/.cache/huggingface/modules` and try again ?
Title: Adding mMARCO dataset
State: closed
Labels: description: Contribution to a dataset script
name: dataset contribution

Adding mMARCO (v1.1) to HF datasets.

Comments:
Hi @albertvillanova we've made a major overhaul of the loading script including all configurations we're making available. Could you please review it again?

@albertvillanova :ping_pong: 

Thanks @lhbonifacio for adding this dataset.
Hi there, i got an error about mmarco:
ConnectionError: Couldn't reach 'unicamp-dl/mmarco' on the Hub (ConnectionError)
code:
`from datasets import list_datasets, load_dataset
dataset = load_dataset('unicamp-dl/mmarco', language='portuguese')`

Any help will be appreciated!

Hi @catqaq, we updated the loading script. Now you can load the datasets with:

```python
dataset = load_dataset('unicamp-dl/mmarco', 'portuguese')
```

You can check the list of supported languages and usage examples in [this link](https://huggingface.co/datasets/unicamp-dl/mmarco). Feel free to contact us if you have any issues.




> 



> Hi @catqaq, we updated the loading script. Now you can load the datasets with:
> 
> ```python
> dataset = load_dataset('unicamp-dl/mmarco', 'portuguese')
> ```
> 
> You can check the list of supported languages and usage examples in [this link](https://huggingface.co/datasets/unicamp-dl/mmarco). Feel free to contact us if you have any issues.

Thanks for your quick updates. So, how can i get the fixed version, install from the source? It seems that the merging is blocked.

@catqaq you can load mMARCO using the namespace `unicamp-dl/mmarco` while this PR remains under review.

Thanks for your contribution, @lhbonifacio and  @hugoabonizio. And sorry for the late response.

We are removing the dataset scripts from this GitHub repo and moving them to the Hugging Face Hub: https://huggingface.co/datasets

As you already created this dataset under your organization namespace (https://huggingface.co/datasets/unicamp-dl/mmarco), I think we can safely close this PR.

We would suggest you complete your dataset card with the YAML tags, to make it searchable and discoverable.

Please, feel free to tell us if you need some help.
Title: Add eli5_category dataset
State: closed

This pull request adds a categorized Long-form question answering dataset `ELI5_Category`. It's a new variant of the [ELI5](https://huggingface.co/datasets/eli5) dataset that uses the Reddit tags to alleviate the training/validation overlapping in the origin ELI5 dataset.

A [report](https://celeritasml.netlify.app/posts/2021-12-01-eli5c/)(Section 2) on this dataset. 

Comments:
> Thanks a lot for adding this dataset ! Good job with the dataset card and the dataset scripts - they're really good :)
> 
> I just added minor changes

Thanks for fixing typos!
Title: Add Wikisource dataset
State: closed
Labels: description: Contribution to a dataset script
name: dataset contribution

Add loading script for Wikisource dataset.

Fix #3399.

CC: @geohci, @yjernite

Comments:
As we are removing the dataset scripts from GitHub and moving them to the Hugging Face Hub, I am going to transfer this script to the repo: https://huggingface.co/datasets/wikimedia/wikisource
Title: Fix type of bridge field in QED
State: closed

Use `Value("string")` instead of `Value("bool")` for the feature type of the `"bridge"` field in the QED dataset. If the value is `False`, set to `None`.

The following paragraph in the QED repo explains the purpose of this field:
>Each annotation in referential_equalities is a pair of spans, the question_reference and the sentence_reference, corresponding to an entity mention in the question and the selected_sentence respectively. As described in the paper, sentence_references can be "bridged in", in which case they do not correspond with any actual span in the selected_sentence. Hence, sentence_reference spans contain an additional field, bridge, which is a prepositional phrase when a reference is bridged, and is False otherwise. Prepositional phrases serve to link bridged references to an anchoring phrase in the selected_sentence. In the case a sentence_reference is bridged, the start and end, as well as the span string, map to such an anchoring phrase in the selected_sentence.

Fix #3346

cc @VictorSanh 

Comments:
Title: disaster_response_messages unavailable
State: closed
Labels: description: Related to the dataset viewer on huggingface.co
name: dataset-viewer

## Dataset viewer issue for '* disaster_response_messages*'

**Link:** https://huggingface.co/datasets/disaster_response_messages

Dataset unavailable. Link dead: https://datasets.appen.com/appen_datasets/disaster_response_data/disaster_response_messages_training.csv

Am I the one who added this dataset ?No


Comments:
Hi, thanks for reporting! This is a duplicate of https://github.com/huggingface/datasets/issues/3240. We are working on a fix.
Title: Non-deterministic tests: CI tests randomly fail
State: closed
Labels: description: Something isn't working
name: bug

## Describe the bug
Some CI tests fail randomly.

1. In https://github.com/huggingface/datasets/pull/3375/commits/c10275fe36085601cb7bdb9daee9a8f1fc734f48, there were 3 failing tests, only on Linux:
   ```
   =========================== short test summary info ============================
   FAILED tests/test_streaming_download_manager.py::test_streaming_dl_manager_get_extraction_protocol[https://drive.google.com/uc?export=download&id=1k92sUfpHxKq8PXWRr7Y5aNHXwOCNUmqh-zip]
   FAILED tests/test_streaming_download_manager.py::test_streaming_gg_drive - Fi...
   FAILED tests/test_streaming_download_manager.py::test_streaming_gg_drive_zipped
   = 3 failed, 3553 passed, 2950 skipped, 2 xfailed, 1 xpassed, 125 warnings in 192.79s (0:03:12) =
   ```

2. After re-running the CI (without any change in the code) in https://github.com/huggingface/datasets/pull/3375/commits/57bfe1f342cd3c59d2510b992d5f06a0761eb147, there was only 1 failing test (one on Linux and a different one on Windows):
   - On Linux:
      ```
      =========================== short test summary info ============================
      FAILED tests/test_streaming_download_manager.py::test_streaming_gg_drive_zipped
      = 1 failed, 3555 passed, 2950 skipped, 2 xfailed, 1 xpassed, 125 warnings in 199.76s (0:03:19) =
      ```
   - On Windows:
      ```
      =========================== short test summary info ===========================
      FAILED tests/test_load.py::test_load_dataset_builder_for_community_dataset_without_script
      = 1 failed, 3551 passed, 2954 skipped, 2 xfailed, 1 xpassed, 121 warnings in 478.58s (0:07:58) =
      ```

   The test `tests/test_streaming_download_manager.py::test_streaming_gg_drive_zipped` passes locally.

3. After re-running again the CI (without any change in the code) in https://github.com/huggingface/datasets/pull/3375/commits/39f32f2119cf91b86867216bb5c356c586503c6a, ALL the tests passed.



Comments:
I think it might come from two different issues:
1. Google Drive is an unreliable host, mainly because of quota limitations
2. the staging environment can sometimes raise some errors

For Google Drive tests we could set up some retries with backup URLs if necessary I guess.
For staging on the other hand, I guess we can investigate what causes this and discuss with the back-end team

Closed by:
- #3982