Minimartzz/github-issues huggingface dataset:
Support streaming zipped dataset repo by passing only repo name
State: closed

Proposed solution:
- I have added the method `iter_files` to DownloadManager and StreamingDownloadManager
- I use this in modules: "csv", "json", "text"
- I test for CSV/JSONL/TXT zipped (and non-zipped) files, both in streaming and non-streaming modes

Fix #3373.

Comments:
I just tested and I think this only opens one file ? If there are several files in the ZIP, only the first one is opened. To open several files from a ZIP, one has to call `open` several times.

What about updating the CSV loader to make it `download_and_extract` zip files, and open each extracted file ?

I have implemented the  glob of ZIP files in the packaged modules:
- csv
- json
- text

Also for streaming and non-streaming.

In https://github.com/huggingface/datasets/pull/3375/commits/c10275fe36085601cb7bdb9daee9a8f1fc734f48, there were 3 failing tests, only on Linux:
```
=========================== short test summary info ============================
FAILED tests/test_streaming_download_manager.py::test_streaming_dl_manager_get_extraction_protocol[https://drive.google.com/uc?export=download&id=1k92sUfpHxKq8PXWRr7Y5aNHXwOCNUmqh-zip]
FAILED tests/test_streaming_download_manager.py::test_streaming_gg_drive - Fi...
FAILED tests/test_streaming_download_manager.py::test_streaming_gg_drive_zipped
= 3 failed, 3553 passed, 2950 skipped, 2 xfailed, 1 xpassed, 125 warnings in 192.79s (0:03:12) =
```

After re-running the CI in https://github.com/huggingface/datasets/pull/3375/commits/57bfe1f342cd3c59d2510b992d5f06a0761eb147, there was only 1 failing test:
- On Linux:
```
=========================== short test summary info ============================
FAILED tests/test_streaming_download_manager.py::test_streaming_gg_drive_zipped
= 1 failed, 3555 passed, 2950 skipped, 2 xfailed, 1 xpassed, 125 warnings in 199.76s (0:03:19) =
```
- On Windows:
```
=========================== short test summary info ===========================
FAILED tests/test_load.py::test_load_dataset_builder_for_community_dataset_without_script
= 1 failed, 3551 passed, 2954 skipped, 2 xfailed, 1 xpassed, 121 warnings in 478.58s (0:07:58) =
```

The test `tests/test_streaming_download_manager.py::test_streaming_gg_drive_zipped` passes locally.

I guess the issue is caused by those tests and has nothing to do with this PR.

@lhoestq my final proposed solution:
- I have added the method `iter_files` to DownloadManager and StreamingDownloadManager
- I use this in modules: "csv", "json", "text"
- I test for CSV/JSONL/TXT zipped (and non-zipped) files, both in streaming and non-streaming modes

> Note that at one point we might consider switching to using `iter_archive` for ZIP files in the json/text/csv loaders since it should be faster.

As far as the functionality is kept... ;)
NonMatchingChecksumError for the CLUE:cluewsc2020, chid, c3 and tnews
State: closed

Hi, it seems like there are updates in cluewsc2020, chid, c3 and tnews, since i could not load them due to the checksum error.

Comments:
Seems like the issue still exists,:
`Downloading and preparing dataset clue/chid (download: 127.15 MiB, generated: 259.71 MiB, post-processed: Unknown size, total: 386.86 MiB) to /mnt/cache/tanhaochen/.cache/huggingface/datasets/clue/chid/1.0.0/e55b490cb7809dcd8db31b9a87119f2e2ec87cdc060da8a9ac070b070ca3e379...
Traceback (most recent call last):
  File "/mnt/cache/tanhaochen/PromptCLUE/test_datasets.py", line 3, in <module>
    cluewsc2020 = datasets.load_dataset("clue","chid")
  File "/mnt/cache/tanhaochen/dependencies/datasets/src/datasets/load.py", line 1667, in load_dataset
    builder_instance.download_and_prepare(
  File "/mnt/cache/tanhaochen/dependencies/datasets/src/datasets/builder.py", line 593, in download_and_prepare
    self._download_and_prepare(
  File "/mnt/cache/tanhaochen/dependencies/datasets/src/datasets/builder.py", line 663, in _download_and_prepare
    verify_checksums(
  File "/mnt/cache/tanhaochen/dependencies/datasets/src/datasets/utils/info_utils.py", line 40, in verify_checksums
    raise NonMatchingChecksumError(error_msg + str(bad_urls))
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://storage.googleapis.com/cluebenchmark/tasks/chid_public.zip']
`

Hi,

the fix hasn't been merged yet (it should be merged early next week).
Support streaming zipped CSV dataset repo by passing only repo name
State: closed
Labels: description: New feature or request
name: enhancement

Given a community 🤗 dataset repository containing only a zipped CSV file (only raw data, no loading script), I would like to load it in streaming mode without passing `data_files`:
```
ds_name = "bigscience-catalogue-data/vietnamese_poetry_from_fsoft_ai_lab"
ds = load_dataset(ds_name, split="train", streaming=True, use_auth_token=True)
item = next(iter(ds))
```

Currently, it gives a `FileNotFoundError` because there is no glob (no "\*" after "zip://": "zip://*") in the passed URL:
```
'zip://::https://huggingface.co/datasets/bigscience-catalogue-data/vietnamese_poetry_from_fsoft_ai_lab/resolve/e5d45f1bd9a8a798cc14f0a45ebc1ce91907c792/poems_dataset.zip'
```


Comments:
[SEO improvement] Add Dataset Metadata to make datasets indexable
State: closed
Labels: description: New feature or request
name: enhancement

Some people who host datasets on github seem to include a table of metadata at the end of their README.md to make the dataset indexable by [Google Dataset Search](https://datasetsearch.research.google.com/) (See [here](https://github.com/google-research/google-research/tree/master/goemotions#dataset-metadata) and [here](https://github.com/cvdfoundation/google-landmark#dataset-metadata)). This could be a useful addition to canonical datasets; perhaps even community datasets.

I'll include a screenshot (as opposed to markdown) as an example so as not to have a github issue indexed as a dataset:

> ![image](https://user-images.githubusercontent.com/3664563/144496173-953428cf-633a-4571-b75b-f099c6b2ed65.png)



**_PS: It might very well be the case that this is already covered by some other markdown magic I'm not aware of._**


Comments:
New: Americas NLI dataset
State: closed

This PR adds the [Americas NLI](https://arxiv.org/abs/2104.08726) dataset, extension of XNLI to 10 low-resource indigenous languages spoken in the Americas: Ashaninka, Aymara, Bribri, Guarani, Nahuatl, Otomi, Quechua, Raramuri, Shipibo-Konibo, and Wixarika.

One odd thing (not sure) is that I had to set

`datasets-cli dummy_data ./datasets/americas_nli/ --auto_generate --n_lines 7500`

`n_lines` very large to successfully generate the dummy files for all the subsets. Happy to get some guidance here.

Otherwise, I hope everything is in order :)

e: missed a step, onto fixing the tests
e2: there you go -- hope it's ok to have added more languages with their ISO codes to `languages.json`, need those tests to pass :laughing: 

Comments:
Document a training loop for streaming dataset
State: closed

I added some docs about streaming dataset. In particular I added two subsections:
- one on how to use `map` for preprocessing
- one on how to use a streaming dataset in a pytorch training loop

cc @patrickvonplaten @stevhliu if you have some comments
cc @Rocketknight1 later we can add the one for TF and I might need your help ^^'

Comments:
[Audio] Allow resampling for audio datasets in streaming mode
State: closed
Labels: description: New feature or request
name: enhancement

Many audio datasets like Common Voice always need to be resampled. This can very easily be done in non-streaming mode as follows:

```python
from datasets import load_dataset

ds = load_dataset("common_voice", "ab", split="test")

ds = ds.cast_column("audio", Audio(sampling_rate=16_000))
```

However in streaming mode it fails currently:

```python
from datasets import load_dataset

ds = load_dataset("common_voice", "ab", split="test", streaming=True)

ds = ds.cast_column("audio", Audio(sampling_rate=16_000))
```

with the following error:

```
AttributeError: 'IterableDataset' object has no attribute 'cast_column'  
```

It would be great if we could add such a feature (I'm not 100% sure though how complex this would be)

Comments:
This requires implementing `cast_column` for iterable datasets, it could be a very nice addition !


It can also be useful to be able to disable the audio/image decoding for the dataset viewer (see PR https://github.com/huggingface/datasets/pull/3430) cc @severo 

EDIT: actually following https://github.com/huggingface/datasets/issues/3145 the dataset viewer might not need it anymore

Just to clarify a bit. This feature is **always** needed when using the common voice dataset in streaming mode. So I think it's quite important
Add multimodal datasets
State: open
Labels: description: Requesting to add a new dataset
name: dataset request

Epic issue to track the addition of multimodal datasets:
- [ ] #2526
- [x] #1842
- [ ] #1810

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

@VictorSanh feel free to add and sort by priority any interesting dataset. I have added the multimodal dataset requests which were already present as issues.

Comments:
Add task tags for multimodal datasets
State: closed
Labels: description: New feature or request
name: enhancement

## **Is your feature request related to a problem? Please describe.**

Currently, task tags are either exclusively related to text or speech processing:
- https://github.com/huggingface/datasets/blob/master/src/datasets/utils/resources/tasks.json

## **Describe the solution you'd like**

We should also add tasks related to:
- multimodality
- image
- video

CC: @VictorSanh @lewtun @lhoestq @merveenoyan @SBrandeis 

Comments:
The Hub pulls these tags from [here](https://github.com/huggingface/hub-docs/blob/main/js/src/lib/interfaces/Types.ts) (allows multimodal tasks) now, so I'm closing this issue.
Use the Audio feature in the AutomaticSpeechRecognition template
State: closed

This updates the ASR template and all supported datasets to use the `Audio` feature

Comments:
Cool !

I noticed that you removed the `audio_file_path_column` field of the template, note that you also have to update all the dataset_infos.json file that still contain this outdated field. For example in the common_voice you can find this:
```
"task_templates": [{"task": "automatic-speech-recognition", "audio_file_path_column": "path", "transcription_column": "sentence"}]
```

Yes, will do that. I'm just busy with the bigscience task.

After we merge this, we should also update the following dataset scripts: https://huggingface.co/datasets?task_ids=task_ids:automatic-speech-recognition

Closing in favor of https://github.com/huggingface/datasets/pull/4006
Adapt image datasets
State: closed

This PR:
* adapts the ImageClassification template to use the new Image feature
* adapts the following datasets to use the new Image feature:
  * beans (+ fixes streaming)
  * cast_vs_dogs (+ fixes streaming)
  * cifar10
  * cifar100
  * fashion_mnist
  * mnist
  * head_qa

cc @nateraw

Comments:
This PR can be merged after #3163 is merged (this PR is pretty big because I was working on the forked branch).

@lhoestq @albertvillanova Could you please take a look at the changes in `src/datasets/utils/streaming_download_manager.py`? These changes were required to support streaming of the `cats_vs_dogs` and the `beans` datasets.

The CI failures are due to the missing fields in the README files.

and thanks for adding support for Path.name and Path.parent for streaming :)
Add The Pile Free Law subset
State: closed

Add:
- Free Law subset of The Pile: "free_law" config

Close bigscience-workshop/data_tooling#75.

CC: @StellaAthena

Comments:
@albertvillanova Is there a specific reason you’re adding the Pile under “the” instead of under “pile”? That does not appear to be consistent with other datasets.

Hi @StellaAthena,

I asked myself the same question, but at the end I decided to be consistent with previously added Pile subsets:
- #2817

I guess the reason is to stress that the definite article is always used before the name of the dataset (your site says: "The Pile. An 800GB Dataset of Diverse Text for Language Modeling"). Other datasets are not usually preceded by the definite article, like "the SQuAD" or "the GLUE" or "the Common Voice"...

CC: @lhoestq 

> I guess the reason is to stress that the definite article is always used before the name of the dataset (your site says: "The Pile. An 800GB Dataset of Diverse Text for Language Modeling").

Yes that's because of this that it starts with "the"
add new field, and get errors
State: closed

after adding new field **tokenized_examples["example_id"]**, and get errors below,
I think it is due to changing data to tensor, and **tokenized_examples["example_id"]** is string list 
**all fields**
```
***************** train_dataset 1: Dataset({
    features: ['attention_mask', 'end_positions', 'example_id', 'input_ids', 'start_positions', 'token_type_ids'],
    num_rows: 87714
})
```

**Errors**
```
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 705, in convert_to_tensors
    tensor = as_tensor(value)
ValueError: too many dimensions 'str'
```

Comments:
Hi, 

could you please post this question on our [Forum](https://discuss.huggingface.co/) as we keep issues for bugs and feature requests? 

> Hi,
> 
> could you please post this question on our [Forum](https://discuss.huggingface.co/) as we keep issues for bugs and feature requests?

ok.
to_tf_dataset() refactor
State: closed

This is the promised cleanup to `to_tf_dataset()` now that the course is out of the way! The main changes are:

- A collator is always required (there was way too much hackiness making things like labels work without it)
- Lots of cleanup and a lot of code moved to `_get_output_signature`
- Should now handle it gracefully when the data collator adds unexpected columns

Comments:
Also, please don't merge yet - I need to make sure all the code samples and notebooks have a collate_fn specified, since we're removing the ability for this method to work without one!

Hi @lhoestq @mariosasko, the other PRs this was depending on in Transformers and huggingface/notebooks are now merged, so this is ready to go. Do you want to take one more look at it, or are you happy at this point?

The documentation for the method is fine, it doesn't need to be changed, but the tutorial notebook definitely looks a little out of date. Let me see what I can do!

@lhoestq I rewrote the last bit of the notebook - let me know what you think!

Cool thank you ! It's much nicer that what we had :)

I also spotted other things I'd like to update in the notebook (especially the beginning) but it can be fixed later
Extend support for streaming datasets that use pd.read_excel
State: closed

This PR fixes error:
```
ValueError: Cannot seek streaming HTTP file
```

CC: @severo 

Comments:
TODO in the future: https://github.com/huggingface/datasets/pull/3355#discussion_r761138011
- If we finally find a use case where the `pd.read_excel()` can work in streaming mode (using fsspec), that is, without using the `.read()`, I propose to try this first, catch the ValueError and then try with `.read`, but all implemented in `xpandas_read_excel`.
raise exception instead of using assertions.
State: closed

fix for the remaining files https://github.com/huggingface/datasets/issues/3171

Comments:
@mariosasko - Thanks for the review & suggestions. Updated as per the suggestions. 

@mariosasko - Hello, Are there any additional changes required from my end??. Wondering if this PR can be merged or still pending on additional steps.

@mariosasko - The approved changes in the PR now has conflicts with the master branch. Would you like me to resolve the conflicts??.  Let me know.   

@mariosasko @lhoestq - Gentle reminder about my previous question. 

Hi ! Thanks for the heads up :)
I just resolved the conflicts, it should be alright now

Merging, thanks for the help @manisnesan !
BLEURT: Match key names to correspond with filename
State: closed

In order to properly locate downloaded ckpt files key name needs to match filename. Correcting change introduced in #3235 

Comments:
Thanks for the suggestion! I think the current checked-in `CHECKPOINT_URLS` is already not working. I believe anyone who tried using the new ckpts (`BLEURT-20-X`) can't unless this fix is in. The zip file from bleurt side unzips to directory name matching the filename (capitalized for new ones). For example without current changes I'd get the following error

```
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-5-f6832fe20f84> in <module>()
      1 predictions = ["hello there", "general kenobi"]
      2 references = ["hello there", "general kenobi"]
----> 3 bleurt = datasets.load_metric("bleurt", "bleurt-20")
      4 results = bleurt.compute(predictions=predictions, references=references)

4 frames
/usr/local/lib/python3.7/dist-packages/bleurt/checkpoint.py in read_bleurt_config(path)
     84   """Reads and checks config file from a BLEURT checkpoint."""
     85   assert tf.io.gfile.exists(path), \
---> 86       "Could not find BLEURT checkpoint {}".format(path)
     87   config_path = os.path.join(path, CONFIG_FILE)
     88   assert tf.io.gfile.exists(config_path), \

AssertionError: Could not find BLEURT checkpoint /root/.cache/huggingface/metrics/bleurt/bleurt-20/downloads/extracted/e34c60f1a05394ecda54e253a10413ca7b5d59f9a23f3cc73258c6b78ffa2f50/bleurt-20
```
inspecting specified path I see that directory name is `BLEURT-20` instead of `bleurt-20`. 
Other solution similar to your suggestion is meddle with `dl_manager.download_and_extract` to unzip to paths with lowering all the paths but I imagine this will affect other parts of the library. 

Indeed, good catch ! Your solution that fixes `CHECKPOINT_URLS ` is simple and works well, thanks :)

Furthermore to avoid breaking changes though we could also keep the support for the lowercase one:
```python
        if self.config_name.lower() in CHECKPOINT_URLS:
            checkpoint_name = self.config_name.lower()
        elif self.config_name.upper() in CHECKPOINT_URLS:
            checkpoint_name = self.config_name.upper()
        else:
            raise KeyError(
                f"{self.config_name} model not found. You should supply the name of a model checkpoint for bleurt in {CHECKPOINT_URLS.keys()}"
            )
```
and then we can use `checkpoint_name` instead of `self.config_name` to download and instantiate the model:
```python
        model_path = dl_manager.download_and_extract(CHECKPOINT_URLS[checkpoint_name])
        self.scorer = score.BleurtScorer(os.path.join(model_path, checkpoint_name))
```

Please let me know if that sounds reasonable to you !

Thanks for the suggestion! I believe your suggestion should work to make keys case insensitive. Changes are committed to the PR now.
iter_archive for zip files 
State: closed

* In this PR, I added the option to iterate through zipfiles for `download_manager.py` only. 
* Next PR will be the same applied to `streaming_download_manager.py`.
* Related issue #3272.
## Comments : 
* There is no `.isreg()` equivalent in zipfile library to check if file is Regular so I used `.is_dir()` instead to skip directories.
* For now I got `streaming_download_manager.py`  working for local zip files, but not for urls. I get the following error when I test it on an archive in google drive, so still working on it.  `BlockSizeError: Got more bytes so far (>2112) than requested (22)`


## Tasks : 
- [x] download_manager.py
- [ ] streaming_download_manager.py

Comments:
And also don't always try streaming with Google Drive - it can have issues because of how Google Drive works (with quotas, restrictions, etc.) and it can indeed cause `BlockSizeError`.

Feel free to host your test data elsewhere, such as in a dataset repository on https://huggingface.co (see [here](https://huggingface.co/docs/datasets/upload_dataset.html#upload-your-files) for a tutorial on how to upload files)
Better error message when download fails
State: closed

From our discussions in https://github.com/huggingface/datasets/issues/3269 and https://github.com/huggingface/datasets/issues/3282 it would be nice to have better messages if a download fails.

In particular the error now shows:
- the error from the HEAD request if there's one
- otherwise the response code of the HEAD request

I also added an error to tell users to pass `use_auth_token` when the Hugging Face Hub returns 401 (Unauthorized).

While paying around with this I also fixed a minor issue with the `force_download` parameter that was not always taken into account

Comments:
Fix ASSET dataset data URLs
State: closed

Change the branch name "master" to "main" in the data URLs, since facebookresearch has changed that.

Comments:
> Hi @tianjianjiang, thanks for the fix.
> The links should also be updated in the `dataset_infos.json` file.
> The failing tests are due to the missing tag in the header of the `README.md` file:

Hi @albertvillanova, thank you for the info! My apologies for the messy PR.
Mirror the canonical datasets to the Hugging Face Hub
State: closed
Labels: description: New feature or request
name: enhancement

- [ ] create a repo on https://hf.co/datasets for every canonical dataset
- [ ] on every commit related to a dataset, update the hf.co repo

See https://github.com/huggingface/moon-landing/pull/1562

@SBrandeis: I let you edit this description if needed to precise the intent.

Comments:
I created a GitHub project to keep track of what needs to be done:
https://github.com/huggingface/datasets/projects/3

I also store my code in a (private for now) repository at https://github.com/huggingface/mirror_canonical_datasets_on_hub

I understand that the datasets are mirrored on the Hub now, right? Might I close @lhoestq @SBrandeis?
Fix JSON ClassLabel casting for integers
State: closed

Loading a JSON dataset with ClassLabel feature types currently fails if the JSON data already has integers. Indeed currently it tries to convert the strings to integers without even checking if the data are not integers already.

For example this currently fails:
```python
from datasets import load_dataset, Features, ClassLabel

path = "data.json"
f = Features({"a": ClassLabel(names=["neg", "pos"])})
d = load_dataset("json", data_files=path, features=f)
```
data.json
```json
{"a": 0}
{"a": 1}
```

I fixed that by adding a line that checks the type of the JSON data before trying to convert them

cc @albertvillanova let me know if it sounds good to you

Comments: