rIsHu009/github-issues-updated huggingface dataset:
Title: ImageFolder dataset builder does not read the validation data set if it is named as "val"
Author: akt42 (CONTRIBUTOR)
State: closed
Created: 2022-08-12T13:26:00Z
Labels: enhancement

**Is your feature request related to a problem? Please describe.**

Currently, the `'imagefolder'` data set builder in [`load_dataset()`](https://github.com/huggingface/datasets/blob/2.4.0/src/datasets/load.py#L1541] ) only [supports](https://github.com/huggingface/datasets/blob/6c609a322da994de149b2c938f19439bca99408e/src/datasets/data_files.py#L31) the following names as the validation data set directory name: `["validation", "valid", "dev"]`. When the validation directory is named as `'val'`, the Data set will not have a validation split. I expected this to be a trivial task but ended up spending a lot of time before knowing that only the above names are supported.

Here's a minimal example of `val` not being recognized:

```python
import os
import numpy as np
import cv2   
from datasets import load_dataset

# creating a dummy data set with the following structure:

# ROOT
# | -- train
# | ---- class_1
# | ---- class_2
# | -- val
# | ---- class_1
# | ---- class_2


ROOT = "data"


for which in ["train", "val"]:
  for class_name in ["class_1", "class_2"]:
    dir_name = os.path.join(ROOT, which, class_name)
    if not os.path.exists(dir_name):
      os.makedirs(dir_name)
    for i in range(10):
       cv2.imwrite(
           os.path.join(dir_name, f"{i}.png"),
           np.random.random((224, 224))
           )

# trying to create a data set
dataset = load_dataset(
    "imagefolder", 
    data_dir=ROOT
)

>> dataset
DatasetDict({
    train: Dataset({
        features: ['image', 'label'],
        num_rows: 20
    })
})

# ^ note how the dataset only has a 'train' subset

```

**Describe the solution you'd like**

The suggestion is to include `"val"` to  [that list ](https://github.com/huggingface/datasets/blob/6c609a322da994de149b2c938f19439bca99408e/src/datasets/data_files.py#L31) as that's a commonly used phrase to name the validation directory. 

Also, In the documentation, explicitly mention that only such directory names are supported as train/val/test directories to avoid confusion.

**Describe alternatives you've considered**

In the documentation, explicitly mention that only such directory names are supported as train/val/test directories without adding `val` to the above list.


**Additional context**

A question asked in the forum: [
Loading an imagenet-style image dataset with train/val directories](https://discuss.huggingface.co/t/loading-an-imagenet-style-image-dataset-with-train-val-directories/21554)

Comments:
#take
Title: Fix documentation card of adv_glue dataset
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-08-12T13:15:26Z

Fix documentation card of adv_glue dataset.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The failing test has nothing to do with this PR:
```
FAILED tests/test_upstream_hub.py::TestPushToHub::test_push_dataset_dict_to_hub_multiple_files
```
Title: Add support for CSV metadata files to ImageFolder
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2022-08-12T11:19:18Z

Fix #4814

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Cool thanks ! Maybe let's include this change after the refactoring from FolderBasedBuilder in #3963 to avoid dealing with too many unpleasant conflicts ?

@lhoestq I resolved the conflicts (AudioFolder also supports CSV metadata now). Let me know what you think.


@lhoestq Thanks for the suggestion! Indeed it makes more sense to use CSV as the default format in the folder-based builders.
Title: Is it possible to pass multiple links to a split in load script?
Author: sadrasabouri (NONE)
State: open
Created: 2022-08-12T11:06:11Z
Labels: enhancement

**Is your feature request related to a problem? Please describe.**
I wanted to use a python loading script in hugging face datasets that use different sources of text (it's somehow a compilation of multiple datasets + my own dataset) based on how `load_dataset` [works](https://huggingface.co/docs/datasets/loading) I assumed I could do something like bellow in my loading script:

```python
...
_URL = "MY_DATASET_URL/resolve/main/data/"
_URLS = {
    "train": [
        "FIRST_URL_TO.txt",
        _URL + "train-00000-of-00001-676bfebbc8742592.parquet"
    ]
}
...
```
but when loading the dataset it raises the following error:
```python
File ~/.local/lib/python3.8/site-packages/datasets/builder.py:704, in DatasetBuilder.download_and_prepare(self, download_config, download_mode, ignore_verifications, try_from_hf_gcs, dl_manager, base_path, use_auth_token, **download_and_prepare_kwargs)
    702         logger.warning("HF google storage unreachable. Downloading and preparing it from source")
    703 if not downloaded_from_gcs:
--> 704     self._download_and_prepare(
    705         dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
...
    668     if isinstance(a, str):
    669         # Force-cast str subclasses to str (issue #21127)
    670         parts.append(str(a))

TypeError: expected str, bytes or os.PathLike object, not list
```

**Describe the solution you'd like**
I believe since it's possible for `load_dataset` to get list of URLs instead of just a URL for `train` split it can be possible here too.

**Describe alternatives you've considered**
An alternative solution would be to download all needed datasets locally and `push_to_hub` them all, but since the datasets I'm talking about are huge it's not among my options. 

**Additional context**
I think loading `text` beside the `parquet` is completely a different issue but I believe I can figure it out by proposing a config for my dataset to load each entry of `_URLS['train']` separately either by `load_dataset("text", ...` or `load_dataset("parquet", ...`.

Comments:
Title: Fix missing tags in dataset cards
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-08-12T09:04:52Z

Fix missing tags in dataset cards:
- boolq
- break_data
- definite_pronoun_resolution
- emo
- kor_nli
- pg19
- quartz
- sciq
- squad_es
- wmt14
- wmt15
- wmt16
- wmt17
- wmt18
- wmt19
- wmt_t2t

This PR partially fixes the missing tags in dataset cards. Subsequent PRs will follow to complete this task.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Misalignment between card tag validation and docs
Author: albertvillanova (MEMBER)
State: open
Created: 2022-08-11T14:44:45Z
Labels: bug

## Describe the bug
As pointed out in other issue: https://github.com/huggingface/datasets/pull/4827#discussion_r943536284
the validation of the dataset card tags is not aligned with its documentation: e.g.
- implementation: `license: List[str]`
- docs: `license: Union[str, List[str]]`

They should be aligned.

CC: @julien-c

Comments:
(Note that the doc is aligned with the hub validation rules, and the "ground truth" is the hub validation rules given that they apply to all datasets, not just the canonical ones)

Instead of our own implementation, we now use `huggingface_hub`'s `DatasetCardData`, which has the correct type hint, so I think we can close this issue.
Title: Support PIL Image objects in `add_item`/`add_column`
Author: mariosasko (COLLABORATOR)
State: open
Created: 2022-08-11T14:25:45Z

Fix #4796 

PS: We should also improve the type inference in `OptimizedTypeSequence` to make it possible to also infer the complex types (only `Image` currently) in nested arrays (e.g.  `[[pil_image], [pil_image, pil_image]]` or `[{"img": pil_image}`]), but I plan to address this in a separate PR.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4828). All of your documentation changes will be reflected on that endpoint.

Hey @mariosasko could we please merge this? I'm still getting the original error at #4796 .

Are you planning to continue working on this?
Title: [Windows] Fix Access Denied when using os.rename()
Author: DougTrajano (CONTRIBUTOR)
State: closed
Created: 2022-08-11T11:57:15Z

In this PR, we are including an additional step when `os.rename()` raises a PermissionError.

Basically, we will use `shutil.move()` on the temp files.

Fix #2937

Comments:
Cool thank you ! Maybe we can just replace `os.rename` by `shutil.move` instead ?

> Cool thank you ! Maybe we can just replace `os.rename` by `shutil.move` instead ?

Yes, I think that could be a better solution, but I didn't test it in Linux (e.g. Ubuntu) to guarantee that `os.rename()` could be completely replaced by `shutil.move()`.

AFAIK `shutil.move` does call `os.rename` first before doing extra work to make it work on windows, so this is should be a safe safe change for linux ;)

> AFAIK `shutil.move` does call `os.rename` first before doing extra work to make it work on windows, so this is should be a safe safe change for linux ;)

alright, let me change the PR then.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4825). All of your documentation changes will be reflected on that endpoint.

Hi @lhoestq looks like one of the tests failed, but is not related to this change, do I need to do something from my side?
Title: Terminating: fork() called from a process already using GNU OpenMP, this is unsafe.
Author: talhaanwarch (NONE)
State: closed
Created: 2022-08-10T19:42:33Z
Labels: bug

Hi, when i try to run prepare_dataset function in [fine tuning ASR tutorial 4](https://colab.research.google.com/github/patrickvonplaten/notebooks/blob/master/Fine_tuning_Wav2Vec2_for_English_ASR.ipynb) , i got this error.
I got this error
Terminating: fork() called from a process already using GNU OpenMP, this is unsafe.
There is no other logs available, so i have no clue what is the cause of it.
```

def prepare_dataset(batch):
    audio = batch["path"]
    # batched output is "un-batched"
    batch["input_values"] = processor(audio["array"], sampling_rate=audio["sampling_rate"]).input_values[0]
    batch["input_length"] = len(batch["input_values"])
    with processor.as_target_processor():
        batch["labels"] = processor(batch["text"]).input_ids
    return batch

data = data.map(prepare_dataset, remove_columns=data.column_names["train"],
                      num_proc=4)
```


Specify the actual results or traceback.
There is no traceback except
`Terminating: fork() called from a process already using GNU OpenMP, this is unsafe.`
## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.4.0
- Platform: Linux-5.15.0-43-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyArrow version: 9.0.0
- Pandas version: 1.4.3

Comments:
Fixed by installing either resampy<3 or resampy>=4
Title: Add add cc-by-sa-2.5 license tag
Author: polinaeterna (CONTRIBUTOR)
State: closed
Created: 2022-08-10T17:18:39Z

- [ ] add it to moon-landing
- [ ] add it to hub-docs

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4818). All of your documentation changes will be reflected on that endpoint.

I think we can close this PR because the `standard_licenses.tsv` file was removed from this repo and we no longer perform any dataset card validation.
Title: Support CSV as metadata file format in AudioFolder/ImageFolder
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2022-08-09T14:36:49Z
Labels: enhancement

Requested here: https://discuss.huggingface.co/t/how-to-structure-an-image-dataset-repo-using-the-image-folder-approach/21004. CSV is also used in AutoTrain for specifying metadata in image datasets.

Comments:
Title: Fix loading example in opus dataset cards
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-08-09T13:47:38Z

This PR:
- fixes the examples to load the datasets, with the corrected dataset name, in their dataset cards for:
  - opus_dgt
  - opus_paracrawl
  - opus_wikipedia
- fixes their dataset cards with the missing required information: title, data instances/fields/splits
- enumerates the supported languages
- adds a missing citation reference for opus_wikipedia

Related to:
- #4806

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Fix bug in function validate_type for Python >= 3.9
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-08-09T10:32:42Z

Fix `validate_type` function, so that it uses `get_origin` instead. This makes the function forward compatible.

This fixes #4811 because:
```python
In [4]: typing.Optional[str]
Out[4]: typing.Optional[str]

In [5]: get_origin(typing.Optional[str])
Out[5]: typing.Union
```

Fix #4811.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Bug in function validate_type for Python >= 3.9
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-08-09T10:25:21Z
Labels: bug

## Describe the bug
The function `validate_type` assumes that the type `typing.Optional[str]` is automatically transformed to `typing.Union[str, NoneType]`.
```python
In [4]: typing.Optional[str]
Out[4]: typing.Union[str, NoneType]
```

However, this is not the case for Python 3.9:
```python
In [3]: typing.Optional[str]
Out[3]: typing.Optional[str]
```

Comments:
Title: Add description to hellaswag dataset
Author: julien-c (MEMBER)
State: closed
Created: 2022-08-09T10:21:14Z
Labels: dataset contribution


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Are the `metadata JSON file` not on their way to deprecation? 😆😇

IMO, more generally than this particular PR, the contribution process should be simplified now that many validation checks happen on the hub side.

Keeping this open in the meantime to get more potential feedback!
Title: Complete the mlqa dataset card
Author: el2e10 (CONTRIBUTOR)
State: closed
Created: 2022-08-09T07:38:06Z

I fixed the issue #4808 

Details of PR:
- Added languages included in the dataset. 
- Added task id and task category. 
- Updated the citation information. 

Fix #4808.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> Thanks for your contribution, @eldhoittangeorge.
> 
> The CI error message: https://github.com/huggingface/datasets/runs/7743526624?check_suite_focus=true
> 
> ```
> E           ValueError: The following issues have been found in the dataset cards:
> E           YAML tags:
> E           __init__() missing 5 required positional arguments: 'annotations_creators', 'language_creators', 'license', 'size_categories', and 'source_datasets'
> ```

I will fix the CI error.

@eldhoittangeorge, thanks again for all the fixes. Just a minor one before we can merge this PR: https://github.com/huggingface/datasets/runs/7744885754?check_suite_focus=true
```
E           YAML tags:
E           Could not validate the metadata, found the following errors:
E           * field 'language_creators':
E           	['unknown'] are not registered tags for 'language_creators', reference at https://github.com/huggingface/datasets/tree/main/src/datasets/utils/resources/creators.json
```

> 

Thanks, I updated the file. 
A small suggestion can you mention this link https://github.com/huggingface/datasets/tree/main/src/datasets/utils/resources/  in the contribution page. So that others will know the acceptable values for the tags.
Title: document fix in opus_gnome dataset
Author: gojiteji (CONTRIBUTOR)
State: closed
Created: 2022-08-09T06:38:13Z

I fixed a issue #4805.

I changed `"gnome"` to `"opus_gnome"` in[ README.md](https://github.com/huggingface/datasets/tree/main/datasets/opus_gnome#dataset-summary).

Comments:
Duplicate:
- #4806
Title: Wrong example in opus_gnome dataset card
Author: gojiteji (CONTRIBUTOR)
State: closed
Created: 2022-08-09T03:21:27Z
Labels: bug

## Describe the bug
I found that [the example on opus_gone dataset ](https://github.com/huggingface/datasets/tree/main/datasets/opus_gnome#dataset-summary) doesn't work.

## Steps to reproduce the bug
```python
 load_dataset("gnome", lang1="it", lang2="pl")
```
`"gnome"` should be `"opus_gnome"`

## Expected results
```bash
100%
1/1 [00:00<00:00, 42.09it/s]
DatasetDict({
    train: Dataset({
        features: ['id', 'translation'],
        num_rows: 8368
    })
})
```

## Actual results
```bash
 Couldn't find 'gnome' on the Hugging Face Hub either: FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/huggingface/datasets/main/datasets/gnome/gnome.py
```

## Environment info
- `datasets` version: 2.4.0
- Platform: Linux-5.4.0-120-generic-x86_64-with-glibc2.27
- Python version: 3.9.13
- PyArrow version: 9.0.0
- Pandas version: 1.4.3

Comments:
Title: Support `pipeline` argument in inspect.py functions
Author: severo (COLLABORATOR)
State: open
Created: 2022-08-08T16:01:24Z
Labels: enhancement

**Is your feature request related to a problem? Please describe.**

The `wikipedia` dataset requires a `pipeline` argument to build the list of splits:

https://huggingface.co/datasets/wikipedia/blob/main/wikipedia.py#L937

But this is currently not supported in `get_dataset_config_info`:

https://github.com/huggingface/datasets/blob/main/src/datasets/inspect.py#L373-L375

which is called by other functions, e.g. `get_dataset_split_names`.

**Additional context**

The dataset viewer is not working out-of-the-box on `wikipedia` for this reason:

https://huggingface.co/datasets/wikipedia/viewer

<img width="637" alt="Capture d’écran 2022-08-08 à 12 01 16" src="https://user-images.githubusercontent.com/1676121/183461838-5330783b-0269-4ba7-a999-314cde2023d8.png">

Comments:
Now: the preview (first-rows) works, but not the conversion to parquet. See https://huggingface.co/datasets/wikipedia/viewer/20220301.de/train

```
_split_generators() missing 1 required positional argument: 'pipeline'

Error code:   UnexpectedError
```
Title: Fix fine classes in trec dataset
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-08-08T05:11:02Z

This PR:
- replaces the fine labels, so that there are 50 instead of 47
- once more labels are added, all they (fine and coarse) have been re-ordered, so that they align with the order in: https://cogcomp.seas.upenn.edu/Data/QA/QC/definition.html
- the feature names have been fixed: `fine_label` instead of `label-fine`
  - to sneak-case (underscores instead of hyphens)
  - words have been reordered

Fix #4790.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: video dataset loader/parser
Author: mccrearyd (CONTRIBUTOR)
State: closed
Created: 2022-08-07T01:54:12Z
Labels: enhancement

you know how you can [use `load_dataset` with any arbitrary csv file](https://huggingface.co/docs/datasets/loading#csv)? and you can also [use it to load a local image dataset](https://huggingface.co/docs/datasets/image_load#local-files)?

could you please add functionality to load a video dataset? it would be really cool if i could point it to a bunch of video files and use pytorch to start looping through batches of videos. like if my batch size is 16, each sample in the batch is a frame from a video. i'm competing in the [minerl challenge](https://www.aicrowd.com/challenges/neurips-2022-minerl-basalt-competition), and it would be awesome to use the HF ecosystem.

Comments:
Hi! We've just started discussing the video support in `datasets` (decoding backends, video feature type, etc.), so I believe we should have something tangible by the end of this year.

Also, if you have additional video features in mind that you would like to see, feel free to let us know

Coool thanks @mariosasko 

Hey @mariosasko, I was wondering if there's a way to load video data currently in the library? 
Alternatively is there a way I could hack it through the dataset.from_dict() method? I tried to hack it, but the issue I run into is that earlier I was doing a `cast_column()` call for the `Image` feature, but now I'm not sure about to do if I want the dataset to have the following keys when I call from_dict on it:
`{"caption":[list of text captions], "video_frames": [list of image lists with one image list corresponding to one video]}`

Maybe something like `cast_column("video_frames", List(Image))` ..
(This is assuming I have already extracted frames from video)
Title: Torgo dataset creation
Author: YingLi001 (NONE)
State: closed
Created: 2022-08-05T14:18:26Z


Comments:
Hi @YingLi001, thanks for your proposal to add this dataset.

However, now we add datasets directly to the Hub (instead of our GitHub repository). You have the instructions in our docs: 
- [Create a dataset loading script](https://huggingface.co/docs/datasets/dataset_script)
- [Create a dataset card](https://huggingface.co/docs/datasets/dataset_card)
- [Share](https://huggingface.co/docs/datasets/share)

Feel free to ask if you need any additional support/help.
Title: Missing MBPP splits
Author: stadlerb (NONE)
State: closed
Created: 2022-08-05T06:51:01Z
Labels: bug

(@albertvillanova)
The [MBPP dataset on the Hub](https://huggingface.co/datasets/mbpp) has only a test split for both its "full" and its "sanitized" subset, while the [paper](https://arxiv.org/abs/2108.07732) states in subsection 2.1 regarding the full split:
> In the experiments described later in the paper, we hold out 10 problems for **few-shot prompting**, another 500 as our **test** dataset (which is used to evaluate both few-shot inference and fine-tuned models), 374 problems for **fine-tuning**, and the rest for **validation**.

If the dataset on the Hub should reproduce most closely what the original authors use, I guess this four-way split should be reflected. 

The paper doesn't explicitly state the task_id ranges of the splits, but the [GitHub readme](https://github.com/google-research/google-research/tree/master/mbpp) referenced in the paper specifies exact task_id ranges, although it misstates the total number of samples:
> We specify a train and test split to use for evaluation. Specifically:
> 
> * Task IDs 11-510 are used for evaluation.
> * Task IDs 1-10 and 511-1000 are used for training and/or prompting. We typically used 1-10 for few-shot prompting, although you can feel free to use any of the training examples.

I.e. the few-shot, train and validation splits are combined into one split, with a soft suggestion of using the first ten for few-shot prompting. It is not explicitly stated whether the 374 fine-tuning samples mentioned in the paper have task_id 511 to 784 or 601 to 974 or are randomly sampled from task_id 511 to 974.

Regarding the "sanitized" split the paper states the following:
> For evaluations involving the edited dataset, we perform comparisons with 100 problems that appear in both the original and edited dataset, using the same held out 10 problems for few-shot prompting and 374 problems for fine-tuning. 

The statement doesn't appear to be very precise, as among the 10 few-shot problems, those with task_id 1, 5 and 10 are not even part of the sanitized variant, and many from the task_id range from 511 to 974 are missing (e.g. task_id 511 to 553). I suppose the idea the task_id ranges for each split remain the same, even if some of the task_ids are not present. That would result in 7 few-shot, 257 test, 141 train and 22 validation examples in the sanitized split.

Comments:
Thanks for reporting this as well, @stadlerb.

I suggest waiting for the answer of the data owners... 

@albertvillanova The first author of the paper responded to the upstream issue:
> Task IDs 11-510 are the 500 test problems. We use 90 problems (511-600) for validation and then remaining 374 for fine-tuning (601-974). The other problems can be used as desired, either for training or few-shot prompting (although this should be specified).

Thanks for the follow-up, @stadlerb.

Would you be willing to open a Pull Request to address this issue? :wink: 

Opened a [PR](https://github.com/huggingface/datasets/pull/4943) to implement this--lmk if you have any feedback
Title: Add DocVQA
Author: NielsRogge (CONTRIBUTOR)
State: open
Created: 2022-08-04T13:07:26Z
Labels: dataset request

## Adding a Dataset
- **Name:** DocVQA
- **Description:** Document Visual Question Answering (DocVQA) seeks to inspire a “purpose-driven” point of view in Document Analysis and Recognition research, where the document content is extracted and used to respond to high-level tasks defined by the human consumers of this information. 
- **Paper:** https://arxiv.org/abs/2007.00398
- **Data:** https://www.docvqa.org/datasets/docvqa
- **Motivation:** Models like LayoutLM and Donut in the Transformers library are fine-tuned on DocVQA. Would be very handy to directly load this dataset from the hub.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/main/ADD_NEW_DATASET.md).

Comments:
Thanks for proposing, @NielsRogge.

Please, note this dataset requires registering in their website and their Terms and Conditions state we cannot distribute their URL:
```
1. You will NOT distribute the download URLs
...
```
Title: Dataset Viewer issue for Team-PIXEL/rendered-wikipedia-english
Author: xplip (NONE)
State: closed
Created: 2022-08-04T12:49:16Z
Labels: dataset-viewer

### Link

https://huggingface.co/datasets/Team-PIXEL/rendered-wikipedia-english/viewer/rendered-wikipedia-en/train

### Description

The dataset can be loaded fine but the viewer shows this error:

```
Server Error
Status code:   400
Exception:     Status400Error
Message:       The dataset does not exist.
```

I'm guessing this is because I recently renamed the dataset. Based on related issues (e.g. https://github.com/huggingface/datasets/issues/4759) , is there something server-side that needs to be refreshed?

### Owner

Yes

Comments:
Thanks for reporting. It's a known issue that should be fixed soon. Meanwhile, I had to manually trigger the dataset viewer. It's OK now.
Note that the extreme aspect ratio of the images generates another issue, that we're inspecting.
Title: Issue with fine classes in trec dataset
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-08-04T12:28:51Z
Labels: bug

## Describe the bug
According to their paper, the TREC dataset contains 2 kinds of classes:
- 6 coarse classes: TREC-6
- 50 fine classes: TREC-50

However, our implementation only has 47 (instead of 50) fine classes. The reason for this is that we only considered the last segment of the label, which is repeated for several coarse classes:
- We have one `desc` fine label instead of 2:
  - `DESC:desc`
  - `HUM:desc`
- We have one `other` fine label instead of 3:
  - `ENTY:other`
  - `LOC:other`
  - `NUM:other`

From their paper:
> We define a two-layered taxonomy, which represents a natural semantic classification for typical answers in the TREC task. The hierarchy contains 6 coarse classes and 50 fine classes,

> Each coarse class contains a non-overlapping set of fine classes.

Comments:
Title: Fix NonMatchingChecksumError in mbpp dataset
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-08-04T08:17:40Z

Fix issue reported on the Hub: https://huggingface.co/datasets/mbpp/discussions/1

Fix #4787.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you for the quick response! Before noticing that you already had implemented the fix, I already had implemened my own version. I'd also suggest bumping the major version because the contents of the dataset changed, even if only slightly.
I'll attach my version of the affected files: [mbpp-checksum-changes.zip](https://github.com/huggingface/datasets/files/9258161/mbpp-checksum-changes.zip).

Hi @stadlerb, thanks for your feedback.

We normally update the major version whenever there is a new dataset release, usually with a breaking change in schema. The patch version is updated whenever there is a small correction in the dataset that does not change its schema.

As a side note for future contributions, please note that this dataset is hosted in our library GitHub repository. Therefore, the PRs to GitHub-hosted datasets needs being done through GitHub.

Currently added datasets are hosted on the Hub and for them, PRs can be done through the Hub.

I just noticed another problem with the dataset: The [GitHub page](https://github.com/google-research/google-research/tree/master/mbpp) and the [paper](http://arxiv.org/abs/2108.07732) mention a train-test split, which is not reflected in the dataloader. I'll open a new issue regarding this later.
Title: .save_to_disk('path', fs=s3) TypeError 
Author: h-k-dev (NONE)
State: closed
Created: 2022-08-03T14:49:29Z
Labels: bug

The following code:
```python
import datasets

train_dataset, test_dataset = load_dataset("imdb", split=["train", "test"])
s3 = datasets.filesystems.S3FileSystem(key=aws_access_key_id, secret=aws_secret_access_key)
train_dataset.save_to_disk("s3://datasets/", fs=s3)

```
produces following traceback:

```shell
File "C:\Users\Hong Knop\AppData\Local\Programs\Python\Python310\lib\site-packages\botocore\auth.py", line 374, in scope
    return '/'.join(scope)

```
I invoke print(scope) in <auth.py> (line 373) and find this:

```python
[('4VA08VLL3VTKQJKCAI8M',), '20220803', 'us-east-1', 's3', 'aws4_request']

```

Comments:
Title: Require torchaudio<0.12.0 in docs
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-08-03T13:32:00Z

This PR adds to docs the requirement of torchaudio<0.12.0 to avoid RuntimeError.

Subsequent to PR:
- #4777

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Add Multiface dataset
Author: osanseviero (CONTRIBUTOR)
State: open
Created: 2022-08-02T21:00:22Z
Labels: dataset request

vision

## Adding a Dataset
- **Name:** Multiface dataset
- **Description:** f high quality recordings of the faces of 13 identities, each captured in a multi-view capture stage performing various facial expressions. An average of 12,200 (v1 scripts) to 23,000 (v2 scripts) frames per subject with capture rate at 30 fps
- **Data:** https://github.com/facebookresearch/multiface

The whole dataset is 65TB though, so I'm not sure

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/main/ADD_NEW_DATASET.md).

Comments:
Hi @osanseviero I would like to add this dataset.

Hey @nandwalritik! Thanks for offering to help!

This dataset might be somewhat complex and I'm concerned about it being 65 TB, which would be quite expensive to host. @lhoestq @mariosasko I would love your input if you think it's worth adding this dataset.

Thanks for proposing this interesting dataset, @osanseviero.

Please note that the data files are already hosted in a third-party server: e.g. the index of data files for entity "6795937" is at https://fb-baas-f32eacb9-8abb-11eb-b2b8-4857dd089e15.s3.amazonaws.com/MugsyDataRelease/v0.0/identities/6795937/index.html 
- audio.tar: https://fb-baas-f32eacb9-8abb-11eb-b2b8-4857dd089e15.s3.amazonaws.com/MugsyDataRelease/v0.0/identities/6795937/audio.tar
- ...

Therefore, in principle, we don't need to host them on our Hub: it would be enough to just implement a loading script in the corresponding Hub dataset repo, e.g. "facebook/multiface"...
Title: Docs for creating a loading script for image datasets
Author: stevhliu (MEMBER)
State: closed
Created: 2022-08-02T20:36:03Z
Labels: documentation

This PR is a first draft of creating a loading script for image datasets. Feel free to let me know if there are any specificities I'm missing for this. 🙂 

To do:
- [x] Document how to create different configurations.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

IMO it would make more sense to add a "Create image dataset" page with two main sections - a no-code approach with `imagefolder` + metadata (preferred way), and with a loading script (advanced). It should be clear when to choose which. If we leave this as-is, the user who jumps straight to the Vision section could be under the impression that writing a loading script is the preferred way to share a vision dataset due to how this subsection starts:
```
Write a dataset loading script to share a dataset.
```
  
Also, I think a note explaining how to make a dataset gated/disable the viewer to hide the data would be beneficial (it's pretty common to require submitting a form to access a CV dataset).

Great suggestion @mariosasko! I added your suggestions, let me know what you think. For gated dataset access, I just added a tip referring users to the relevant docs since it's more of a Hub feature than `datasets` feature.

Thanks, looks much better now :). I would also move the sections explaining how to create an `imagefolder` for the specific task from the [loading page](https://raw.githubusercontent.com/huggingface/datasets/main/docs/source/image_load.mdx) to this one. IMO it makes more sense to have the basic info (imagefolder structure + `load_dataset` call) there + a link to this page for info on how to create an image folder dataset.

Good idea! Moved everything about `imagefolder` + metadata to the create an image dataset section since the `load_dataset` call is the same for different computer vision tasks. 

Thanks for all the feedbacks! 🥰

What do you think about creating how to share an `ImageFolder` dataset in a separate PR? I think we should create a new section under `Vision` for how to share an image dataset.

I love it thanks ! I think moving forward we can use CSV instead of JSON Lines in the docs ;)
Title: pyarrow.lib.ArrowCapacityError: array cannot contain more than 2147483646 bytes, have 2147483648
Author: conceptofmind (NONE)
State: closed
Created: 2022-08-02T18:36:05Z
Labels: bug

## Describe the bug
Following the example in CodeParrot, I receive an array size limitation error when deduplicating larger datasets.

## Steps to reproduce the bug
```python
dataset_name = "the_pile"
ds = load_dataset(dataset_name, split="train")
ds = ds.map(preprocess, num_proc=num_workers)
uniques = set(ds.unique("hash"))
```
Gists for minimum reproducible example:
https://gist.github.com/conceptofmind/c5804428ea1bd89767815f9cd5f02d9a
https://gist.github.com/conceptofmind/feafb07e236f28d79c2d4b28ffbdb6e2

## Expected results
Chunking and writing out a deduplicated dataset. 

## Actual results
```
return dataset._data.column(column).unique().to_pylist()
File "pyarrow/table.pxi", line 394, in pyarrow.lib.ChunkedArray.unique
File "pyarrow/_compute.pyx", line 531, in pyarrow._compute.call_function
File "pyarrow/_compute.pyx", line 330, in pyarrow._compute.Function.call
File "pyarrow/error.pxi", line 143, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 124, in pyarrow.lib.check_status
pyarrow.lib.ArrowCapacityError: array cannot contain more than 2147483646 bytes, have 2147483648
```

Comments:
Thanks for reporting @conceptofmind.

Could you please give details about your environment? 
```
## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:
- Platform:
- Python version:
- PyArrow version:
```

Hi @albertvillanova ,

Here is the environment information:
```
- `datasets` version: 2.3.2
- Platform: Linux-5.4.0-122-generic-x86_64-with-glibc2.27
- Python version: 3.9.12
- PyArrow version: 7.0.0
- Pandas version: 1.4.2
```
Thanks,

Enrico

I think this issue is solved here https://discuss.huggingface.co/t/minhash-deduplication/19992/12?u=loubnabnl, this only happens for very large datasets we will update it in CodeParrot code

Hi @loubnabnl,

Yes, the issue is solved in the discussion thread.

I will close this issue.

Thank you again for all of your help.

Enrico

Thanks @loubnabnl for pointing out the solution to this issue.
Title: Update local loading script docs
Author: stevhliu (MEMBER)
State: closed
Created: 2022-08-01T20:21:07Z
Labels: documentation

This PR clarifies the local loading script section to include how to load a dataset after you've modified the local loading script (closes #4732).

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4778). All of your documentation changes will be reflected on that endpoint.

I would rather have a section in the docs that explains how to modify the script of an existing dataset (`inspect_dataset` + modification + `load_dataset`) instead of focusing on the GH datasets bundled with the source (only applicable for devs).

Good idea! I went with @mariosasko's suggestion to use `inspect_dataset` instead of cloning a dataset repository since it's a good opportunity to show off more of the library's lesser-known functions if that's ok with everyone :)

One advantage of cloning the repo is that it fetches potential data files referenced inside a script using relative paths, so if we decide to use `inspect_dataset`, we should at least add a tip to explain this limitation and how to circumvent it.

Oh you're right. Calling `load_dataset` on the modified script without having the files that come with it is not ideal. I agree it should be `git clone` instead - and inspect is for inspection only ^^'
Title: RuntimeError when using torchaudio 0.12.0 to load MP3 audio file
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-08-01T14:11:23Z

Current version of `torchaudio` (0.12.0) raises a RuntimeError when trying to use `sox_io` backend but non-Python dependency `sox` is not installed:
https://github.com/pytorch/audio/blob/2e1388401c434011e9f044b40bc8374f2ddfc414/torchaudio/backend/sox_io_backend.py#L21-L29
```python
def _fail_load(
    filepath: str,
    frame_offset: int = 0,
    num_frames: int = -1,
    normalize: bool = True,
    channels_first: bool = True,
    format: Optional[str] = None,
) -> Tuple[torch.Tensor, int]:
    raise RuntimeError("Failed to load audio from {}".format(filepath))
```

Maybe we should raise a more actionable error message so that the user knows how to fix it.

UPDATE:
- this is an incompatibility of latest torchaudio (0.12.0) and the sox backend

TODO:
- [x] as a temporary solution, we should recommend installing torchaudio<0.12.0
  - #4777 
  - #4785
- [ ] however, a stable solution must be found for torchaudio>=0.12.0

Related to: 
- https://github.com/huggingface/transformers/issues/18379

Comments:
Requiring torchaudio<0.12.0 isn't really a viable solution because that implies torch<0.12.0 which means no sm_86 CUDA support which means no RTX 3090 support in PyTorch.

But in my case, the error only occurs if `_fallback_load` resolves to `_fail_load` inside torchaudio 0.12.0 which is only the case if FFMPEG initialization failed:  https://github.com/pytorch/audio/blob/b1f510fa5681e92ee82bdc6b2d1ed896799fc32c/torchaudio/backend/sox_io_backend.py#L36-L47

That means the proper solution for torchaudio>=0.12.0 is to check `torchaudio._extension._FFMPEG_INITIALIZED` and if it is False, then we need to remind the user to install a dynamically linked ffmpeg 4.1.8 and then maybe call `torchaudio._extension._init_ffmpeg()` to force a user-visible exception showing the missing ffmpeg dynamic library name.

On my system, installing 

- libavcodec.so.58 
- libavdevice.so.58  
- libavfilter.so.7  
- libavformat.so.58  
- libavutil.so.56  
- libswresample.so.3  
- libswscale.so.5

from ffmpeg 4.1.8 made HF datasets 2.3.2 work just fine with  torchaudio 0.12.1+cu116:

```python3
import sox, torchaudio, datasets
print('torchaudio', torchaudio.__version__)
print('datasets', datasets.__version__)
torchaudio._extension._init_ffmpeg()
print(torchaudio._extension._FFMPEG_INITIALIZED)
waveform, sample_rate = torchaudio.load('/workspace/.cache/huggingface/datasets/downloads/extracted/8e5aa88585efa2a4c74c6664b576550d32b7ff9c3d1d17cc04f44f11338c3dc6/cv-corpus-8.0-2022-01-19/en/clips/common_voice_en_100038.mp3', format='mp3')
print(waveform.shape)
```

```
torchaudio 0.12.1+cu116
datasets 2.3.2
True
torch.Size([1, 369792])
```

Related: https://github.com/huggingface/datasets/issues/4889

Closing as we no longer use `torchaudio` for decoding MP3 files.
Title: Streaming not supported in Theivaprakasham/wildreceipt
Author: NitishkKarra (NONE)
State: closed
Created: 2022-08-01T09:46:17Z
Labels: streaming

### Link

_No response_

### Description

_No response_

### Owner

_No response_

Comments:
Thanks for reporting @NitishkKarra.

The root source of the issue is that streaming mode is not supported out-of-the-box for that dataset, because it contains a TAR file.

We have opened a discussion in the corresponding Hub dataset page, pointing out this issue: https://huggingface.co/datasets/Theivaprakasham/wildreceipt/discussions/1

I'm closing this issue here, so this discussion is transferred there instead.
Title: Training hangs at the end of epoch, with set_transform/with_transform+multiple workers
Author: memray (NONE)
State: open
Created: 2022-07-31T06:32:28Z
Labels: bug

## Describe the bug
I use load_dataset() (I tried with [wiki](https://huggingface.co/datasets/wikipedia) and my own json data) and use set_transform/with_transform for preprocessing. But it hangs at the end of the 1st epoch if dataloader_num_workers>=1. No problem with single worker. 


## Steps to reproduce the bug
```python
train_dataset = datasets.load_dataset("wikipedia", "20220301.en",
                                      split='train', 
                                      cache_dir=model_args.cache_dir,
                                      streaming=False)
train_dataset.set_transform(psg_parse_fn)
train_dataloader = DataLoader(
            train_dataset,
            batch_size=args.train_batch_size,
            sampler=DistributedSampler(train_dataset),
            collate_fn=data_collator,
            drop_last=args.dataloader_drop_last,
            num_workers=args.dataloader_num_workers,
        )
```

## Expected results


## Actual results
It simply hangs. The ending step is num_example/batch_size (one epoch).

## Environment info
- `datasets` version: 2.4.1.dev0
- Platform: Linux-5.4.170+-x86_64-with-glibc2.17
- Python version: 3.8.12
- PyArrow version: 8.0.0
- Pandas version: 1.4.1

Comments:
Title: Document loading from relative path
Author: stevhliu (MEMBER)
State: closed
Created: 2022-07-29T23:32:21Z
Labels: documentation

This PR describes loading a dataset from the Hub by specifying a relative path in `data_dir` or `data_files` in `load_dataset` (see #4757).

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks for the feedback!

I agree that adding it to `load_hub.mdx` is probably a bit too specific, especially for beginners reading the tutorials. Since this clarification is closely related to loading from the Hub (the only difference being the presence/absence of a loading script), I think it makes the most sense to keep it somewhere in `loading.mdx`. What do you think about adding a Warning in Loading >>> Hugging Face Hub that explains the difference between relative/absolute paths when there is a script?

What about updating the section about "manual download" ? I think it goes there no ?

https://huggingface.co/docs/datasets/v2.4.0/en/loading#manual-download

Updated the manual download section :)

Thanks ! Pinging @albertvillanova to review this change, and then I think we're good to merge
Title: fix typo
Author: Jiaxin-Wen (CONTRIBUTOR)
State: closed
Created: 2022-07-29T11:46:12Z

By defaul -> By default

Comments:
good catch thanks ! Can you check if the same typo is also present in `add_elasticsearch_index` ? It has a very similar signature

> good catch thanks ! Can you check if the same typo is also present in `add_elasticsearch_index` ? It has a very similar signature

fixed
Title: Unpin rouge_score test dependency
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-07-29T08:17:40Z

Once `rouge-score` has made the 0.1.2 release to fix their issue https://github.com/google-research/google-research/issues/1212, we can unpin it.

Related to:
- #4735

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: More rigorous shape inference in to_tf_dataset
Author: Rocketknight1 (MEMBER)
State: closed
Created: 2022-07-28T18:04:15Z

`tf.data` needs to know the shape of tensors emitted from a `tf.data.Dataset`. Although `None` dimensions are possible, overusing them can cause problems - Keras uses the dataset tensor spec at compile-time, and so saying that a dimension is `None` when it's actually constant can hurt performance, or even cause training to fail for dimensions that are needed to determine the shape of weight tensors!

The compromise I used here was to sample several batches from the underlying dataset and apply the `collate_fn` to them, and then to see which dimensions were "empirically variable". There's an obvious problem here, though - if you sample 10 batches and they all have the same shape on a certain dimension, there's still a small chance that the 11th batch will be different, and Keras will throw an error  if a dataset tries to emit a tensor whose shape doesn't match the spec.

I encountered this bug in practice once or twice for datasets that were mostly-but-not-totally constant on a given dimension, and I still don't have a perfect solution, but this PR should greatly reduce the risk. It samples many more batches, and also samples very small batches (size 2) - this increases the variability, making it more likely that a few outlier samples will be detected.

Ideally, of course, we'd determine the full output shape analytically, but that's surprisingly tricky when the `collate_fn` can be any arbitrary Python code!

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Title: Improve features resolution in streaming
Author: lhoestq (MEMBER)
State: closed
Created: 2022-07-28T17:28:11Z

`IterableDataset._resolve_features` was returning the features sorted alphabetically by column name, which is not consistent with non-streaming. I changed this and used the order of columns from the data themselves. It was causing some inconsistencies in the dataset viewer as well.

I also fixed `interleave_datasets` that was not filling missing columns with None, because it was not using the columns from `IterableDataset._resolve_features`

cc @severo

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Just took your comment into account @mariosasko , let me know if it's good for you now :)
Title: Dataset Viewer issue for Toygar/turkish-offensive-language-detection
Author: tanyelai (NONE)
State: closed
Created: 2022-07-28T11:21:43Z
Labels: dataset-viewer

### Link

https://huggingface.co/datasets/Toygar/turkish-offensive-language-detection

### Description

Status code:   400
Exception:     Status400Error
Message:       The dataset does not exist.

Hi, I provided train.csv, test.csv and valid.csv files. However, viewer says dataset does not exist. 
Should I need to do anything else?

### Owner

Yes

Comments:
I refreshed the dataset viewer manually, it's fixed now. Sorry for the inconvenience.
<img width="1557" alt="Capture d’écran 2022-07-28 à 09 17 39" src="https://user-images.githubusercontent.com/1676121/181514666-92d7f8e1-ddc1-4769-84f3-f1edfdb902e8.png">
Title: Document better when relative paths are transformed to URLs
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-07-28T08:46:27Z
Labels: documentation

As discussed with @ydshieh, when passing a relative path as `data_dir` to `load_dataset` of a dataset hosted on the Hub, the relative path is transformed to the corresponding URL of the Hub dataset.

Currently, we mention this in our docs here: [Create a dataset loading script > Download data files and organize splits](https://huggingface.co/docs/datasets/v2.4.0/en/dataset_script#download-data-files-and-organize-splits)
> If the data files live in the same folder or repository of the dataset script, you can just pass the relative paths to the files instead of URLs.

Maybe we should document better how relative paths are handled, not only when creating a dataset loading script, but also when passing to `load_dataset`:
- `data_dir`
- `data_files`

CC: @stevhliu

Comments: