kardwalker/my_dataset huggingface dataset:
GitHub issue #5096: Transfer some canonical datasets under an organization namespace
State: closed | Created: 2022-10-10T15:44:31Z

As discussed during our @huggingface/datasets meeting, we are planning to move some "canonical" dataset scripts under their corresponding organization namespace (if this does not exist).

On the contrary, if the dataset already exists under the organization namespace, we are deprecating the canonical one (and eventually delete it).

First, we should test it using a dummy dataset/organization.

TODO:
- [x] Test with a dummy dataset
  - [x] Create dummy canonical dataset: https://huggingface.co/datasets/dummy_canonical_dataset
  - [x] Create dummy organization: https://huggingface.co/dummy-canonical-org
  - [x] Transfer dummy canonical dataset to dummy organization
- [ ] Transfer datasets
  - [x] babi_qa => facebook
  - [x] blbooks => TheBritishLibrary/blbooks
  - [x] blbooksgenre => TheBritishLibrary/blbooksgenre
  - [x] common_gen => allenai
  - [x] commonsense_qa => tau
  - [x] competition_math => hendrycks/competition_math
  - [x] cord19 => allenai
  - [x] emotion => dair-ai
  - [ ] gem => GEM
  - [x] hellaswag => Rowan
  - [x] hendrycks_test => cais/mmlu
  - [x] indonlu => indonlp
  - [ ] multilingual_librispeech => facebook
    - It already exists "facebook/multilingual_librispeech"
  - [ ] oscar => oscar-corpus
  - [x] peer_read => allenai
  - [x] qasper => allenai
  - [x] reddit => webis/tldr-17
  - [x] russian_super_glue => russiannlp
  - [x] rvl_cdip => aharley
  - [x] s2orc => allenai
  - [x] scicite => allenai
  - [x] scifact => allenai
  - [x] scitldr => allenai
  - [x] swiss_judgment_prediction => rcds
  - [x] the_pile => EleutherAI
  - [ ] wmt14, wmt15, wmt16, wmt17, wmt18, wmt19,... => wmt
- [ ] Deprecate (and eventually remove) datasets that cannot be transferred because they already exist
  - [x] banking77 => PolyAI
  - [x] common_voice => mozilla-foundation
  - [x] german_legal_entity_recognition => elenanereiss
- ... 

EDIT: the list above is continuously being updated
GitHub issue #5094: Multiprocessing with `Dataset.map` and  `PyTorch` results in deadlock 
State: closed | Created: 2022-10-10T13:50:56Z

## Describe the bug
There seems to be an issue with using multiprocessing with `datasets.Dataset.map` (i.e. setting `num_proc` to a value greater than one) combined with a function that uses `torch` under the hood.  The subprocesses that `datasets.Dataset.map` spawns [a this step](https://github.com/huggingface/datasets/blob/1b935dab9d2f171a8c6294269421fe967eb55e34/src/datasets/arrow_dataset.py#L2663) go into wait mode forever.  

## Steps to reproduce the bug
The below code goes into  deadlock when `NUMBER_OF_PROCESSES` is greater than one.
```python

NUMBER_OF_PROCESSES = 2

from transformers import AutoTokenizer, AutoModel
from datasets import load_dataset

dataset = load_dataset("glue", "mrpc", split="train")
tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")
model = AutoModel.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")
model.to("cpu")

def cls_pooling(model_output):
    return model_output.last_hidden_state[:, 0]

def generate_embeddings_batched(examples):
    sentences_batch = list(examples['sentence1'])
    encoded_input = tokenizer(
        sentences_batch, padding=True, truncation=True, return_tensors="pt"
    )
    encoded_input = {k: v.to("cpu") for k, v in encoded_input.items()}
    model_output = model(**encoded_input)
    embeddings = cls_pooling(model_output)
    examples['embeddings'] = embeddings.detach().cpu().numpy() # 64, 384
    return examples

embeddings_dataset = dataset.map(
    generate_embeddings_batched, 
    batched=True,
    batch_size=10,
    num_proc=NUMBER_OF_PROCESSES
)
```
While debugging it I've seen that it gets "stuck" when calling `torch.nn.Embedding.forward` but some testing shows that the same happens with other functions from `torch.nn`. 

## Environment info
- Platform: Linux-5.14.0-1052-oem-x86_64-with-glibc2.31
- Python version: 3.9.14
- PyArrow version: 9.0.0
- Pandas version: 1.5.0

Not sure if this is a HF problem, a PyTorch problem or something I'm doing wrong..

Thanks!
GitHub issue #5093: Mismatch between tutoriel and doc
State: closed | Created: 2022-10-10T10:23:53Z

## Describe the bug
In the "Process text data" tutorial, [`map` has `return_tensors` as kwarg](https://huggingface.co/docs/datasets/main/en/nlp_process#map). It does not seem to appear in the [function documentation](https://huggingface.co/docs/datasets/main/en/package_reference/main_classes#datasets.Dataset.map), nor to work.

## Steps to reproduce the bug
MWE:
```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")

from datasets import load_dataset
dataset  = load_dataset("lhoestq/demo1", split="train")

dataset = dataset.map(lambda examples: tokenizer(examples["review"]), batched=True, return_tensors="pt")

```

## Expected results
return_tensors to be a valid kwarg :smiley: 

## Actual results
```python
>> TypeError: map() got an unexpected keyword argument 'return_tensors'
```

## Environment info
- `datasets` version: 2.3.2
- Platform: Linux-5.14.0-1052-oem-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyArrow version: 8.0.0
- Pandas version: 1.4.3
GitHub issue #5092: Use HTML relative paths for tiles in the docs
State: closed | Created: 2022-10-10T07:24:27Z

This PR replaces the absolute paths in the landing page tiles with relative ones so that one can test navigation both locally in and in future PRs (see [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_5084/en/index) for an example PR where the links don't work).

I encountered this while working on the `optimum` docs and figured I'd fix it elsewhere too :)

Internal Slack thread: https://huggingface.slack.com/archives/C02GLJ5S0E9/p1665129710176619
GitHub issue #5091: Allow connection objects in `from_sql` + small doc improvement
State: closed | Created: 2022-10-07T12:39:44Z

Allow connection objects in `from_sql` (emit a warning that they are cachable) and add a tip that explains the format of the con parameter when provided as a URI string.

PS: ~~This PR contains a parameter link, so https://github.com/huggingface/doc-builder/pull/311 needs to be merged before it's "ready for review".~~ Done!
GitHub issue #5090: Review sync issues from GitHub to Hub
State: closed | Created: 2022-10-07T12:31:56Z

## Describe the bug
We have discovered that sometimes there were sync issues between GitHub and Hub datasets, after a merge commit to main branch.

For example:
- this merge commit: https://github.com/huggingface/datasets/commit/d74a9e8e4bfff1fed03a4cab99180a841d7caf4b
- was not properly synced with the Hub: https://github.com/huggingface/datasets/actions/runs/3002495269/jobs/4819769684
```
[main 9e641de]  Add Papers with Code ID to scifact dataset (#4941)
 Author: Albert Villanova del Moral <albertvillanova@users.noreply.huggingface.co>
 1 file changed, 42 insertions(+), 14 deletions(-)
push failed !
GitCommandError(['git', 'push'], 1, b'remote: ----------------------------------------------------------        \nremote: Sorry, your push was rejected during YAML metadata verification:        \nremote: - Error: "license" does not match any of the allowed types        \nremote: ----------------------------------------------------------        \nremote: Please find the documentation at:        \nremote: https://huggingface.co/docs/hub/models-cards#model-card-metadata        \nremote: ----------------------------------------------------------        \nTo [https://huggingface.co/datasets/scifact.git\n](https://huggingface.co/datasets/scifact.git/n) ! [remote rejected] main -> main (pre-receive hook declined)\nerror: failed to push some refs to \'[https://huggingface.co/datasets/scifact.git\](https://huggingface.co/datasets/scifact.git/)'', b'')
```
We are reviewing sync issues in previous commits to recover them and repushing to the Hub.

TODO: Review
- [x] #4941
  - scifact
- [x] #4931
  - scifact
- [x] #4753
  - wikipedia 
- [x] #4554
  - wmt17, wmt19, wmt_t2t
  - Fixed with "Release 2.4.0" commit: https://github.com/huggingface/datasets/commit/401d4c4f9b9594cb6527c599c0e7a72ce1a0ea49
    - https://huggingface.co/datasets/wmt17/commit/5c0afa83fbbd3508ff7627c07f1b27756d1379ea
    - https://huggingface.co/datasets/wmt19/commit/b8ad5bf1960208a376a0ab20bc8eac9638f7b400
    - https://huggingface.co/datasets/wmt_t2t/commit/b6d67191804dd0933476fede36754a436b48d1fc
- [x] #4607
- [x] #4416
  - lccc
  - Fixed with "Release 2.3.0" commit: https://huggingface.co/datasets/lccc/commit/8b1f8cf425b5653a0a4357a53205aac82ce038d1
- [x] #4367
GitHub issue #5089: Resume failed process
State: open | Created: 2022-10-07T08:07:03Z

**Is your feature request related to a problem? Please describe.**
When a process (`map`, `filter`, etc.) crashes part-way through, you lose all progress.

**Describe the solution you'd like**
It would be good if the cache reflected the partial progress, so that after we restart the script, the process can restart where it left off.

**Describe alternatives you've considered**
Doing processing outside of `datasets`, by writing the dataset to json files and building a restart mechanism myself.

**Additional context**
N/A
GitHub issue #5088: load_datasets("json", ...) don't read local .json.gz properly
State: open | Created: 2022-10-07T02:16:58Z

## Describe the bug
I have a local file `*.json.gz` and it can be read by `pandas.read_json(lines=True)`, but cannot be read by `load_datasets("json")` (resulting in 0 lines)

## Steps to reproduce the bug
```python
fpath = '/data/junwang/.cache/general/57b6f2314cbe0bc45dda5b78f0871df2/test.json.gz'
ds_panda = DatasetDict(
    test=Dataset.from_pandas(
        pd.read_json(fpath, lines=True)
    )
)
ds_direct = load_dataset(
    'json', data_files={
        'test': fpath
    }, features=Features(
        text_input=Value(dtype="string", id=None),
        text_output=Value(dtype="string", id=None)
    )
)
len(ds_panda['test']), len(ds_direct['test'])
```

## Expected results
Lines of `ds_panda['test']` and `ds_direct['test']` should match.

## Actual results
```
Using custom data configuration default-c0ef2598760968aa
Downloading and preparing dataset json/default to /data/junwang/.cache/huggingface/datasets/json/default-c0ef2598760968aa/0.0.0/e6070c77f18f01a5ad4551a8b7edfba20b8438b7cad4d94e6ad9378022ce4aab...
Dataset json downloaded and prepared to /data/junwang/.cache/huggingface/datasets/json/default-c0ef2598760968aa/0.0.0/e6070c77f18f01a5ad4551a8b7edfba20b8438b7cad4d94e6ad9378022ce4aab. Subsequent calls will reuse this data.

(62087, 0)
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:
- Platform: Ubuntu 18.04.4 LTS 
- Python version: 3.8.13
- PyArrow version: 9.0.0
GitHub issue #5086: HTTPError: 404 Client Error: Not Found for url
State: closed | Created: 2022-10-06T19:48:58Z

## Describe the bug

I was following chap 5 from huggingface course: https://huggingface.co/course/chapter5/6?fw=tf

However, I'm not able to download the datasets, with a 404 erros


<img width="1160" alt="iShot2022-10-06_15 54 50" src="https://user-images.githubusercontent.com/54015474/194406327-ae62c2f3-1da5-4686-8631-13d879a0edee.png">





## Steps to reproduce the bug
```python
from huggingface_hub import hf_hub_url

data_files = hf_hub_url(
    repo_id="lewtun/github-issues",
    filename="datasets-issues-with-hf-doc-builder.jsonl",
    repo_type="dataset",
)
from datasets import load_dataset

issues_dataset = load_dataset("json", data_files=data_files, split="train")
issues_dataset
```


## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.5.2
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.9.12
- PyArrow version: 9.0.0
- Pandas version: 1.4.4
GitHub issue #5085: Filtering on an empty dataset returns a corrupted dataset.
State: closed | Created: 2022-10-06T18:18:49Z

## Describe the bug
When filtering a dataset twice, where the first result is an empty dataset, the second dataset seems corrupted.

## Steps to reproduce the bug
```python
  datasets = load_dataset("glue", "sst2")
  dataset_split = datasets['validation']
  ds_filter_1 = dataset_split.filter(lambda x: False)  # Some filtering condition that leads to an empty dataset
  assert ds_filter_1.num_rows == 0
  sentences = ds_filter_1['sentence']
  assert len(sentences) == 0
  ds_filter_2 = ds_filter_1.filter(lambda x: False)  # Some other filtering condition
  assert ds_filter_2.num_rows == 0
  assert 'sentence' in ds_filter_2.column_names
  sentences = ds_filter_2['sentence']
```

## Expected results
The last line should be returning an empty list, same as 4 lines above.

## Actual results
The last line currently raises `IndexError: index out of bounds`.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.5.2
- Platform: macOS-11.6.6-x86_64-i386-64bit
- Python version: 3.9.11
- PyArrow version: 7.0.0
- Pandas version: 1.4.1
GitHub issue #5084: IterableDataset formatting in numpy/torch/tf/jax
State: closed | Created: 2022-10-06T16:53:38Z

This code now returns a numpy array:
```python
from datasets import load_dataset

ds = load_dataset("imagenet-1k", split="train", streaming=True).with_format("np")
print(next(iter(ds))["image"])
```

It also works with "arrow", "pandas", "torch", "tf" and "jax"

### Implementation details:

I'm using the existing code to format an Arrow Table to the right output format for simplicity.
Therefore it's probbaly not the most optimized approach.

For example to output PyTorch tensors it does this for every example:

python data -> arrow table -> numpy extracted data -> pytorch formatted data

### Releasing this feature

Even though I consider this as a bug/inconsistency, this change is a breaking change.
And I'm sure some users were relying on the torch iterable dataset to return PIL Image and used data collators to convert to pytorch.

So I guess this is `datasets` 3.0 ?

### TODO

- [x] merge https://github.com/huggingface/datasets/pull/5072
- [ ] docs
- [ ] tests

Close https://github.com/huggingface/datasets/issues/5083
GitHub issue #5083: Support numpy/torch/tf/jax formatting for IterableDataset
State: closed | Created: 2022-10-06T15:14:58Z

Right now `IterableDataset` doesn't do any formatting.

In particular this code should return a numpy array:
```python
from datasets import load_dataset

ds = load_dataset("imagenet-1k", split="train", streaming=True).with_format("np")
print(next(iter(ds))["image"])
```

Right now it returns a PIL.Image.

Setting `streaming=False` does return a numpy array after #5072
GitHub issue #5080: Use hfh for caching
State: open | Created: 2022-10-06T05:51:58Z

## Is your feature request related to a problem?

As previously discussed in our meeting with @Wauplin and agreed on our last datasets team sync meeting, I'm investigating how `datasets` can use `hfh` for caching.

## Describe the solution you'd like

Due to the peculiarities of the `datasets` cache, I would propose adopting `hfh` caching system in stages.

First, we could easily start using `hfh` caching for:
- dataset Python scripts
- dataset READMEs
- dataset infos JSON files (now deprecated)

Second, we could also use `hfh` caching for data files downloaded from the Hub.

Further investigation is needed for:
- files downloaded from non-Hub hosts
- extracted files from downloaded archive/compressed files
- generated Arrow files

## Additional context

Docs about the `hfh` caching system:
- [Manage huggingface_hub cache-system](https://huggingface.co/docs/huggingface_hub/main/en/how-to-cache)
- [Cache-system reference](https://huggingface.co/docs/huggingface_hub/main/en/package_reference/cache)

The `transformers` library has already adopted `hfh` for caching. See:
- huggingface/transformers#18438
- huggingface/transformers#18857
- huggingface/transformers#18966
GitHub issue #5079: refactor: replace AssertionError with more meaningful exceptions (#5074)
State: closed | Created: 2022-10-06T01:39:35Z

Closes #5074

Replaces `AssertionError` in the following files with more descriptive exceptions:

- `src/datasets/arrow_reader.py`
- `src/datasets/builder.py`
- `src/datasets/utils/version.py`

The issue listed more files that needed to be fixed, but the rest of them were contained in the top-level `datasets` directory, which was removed when #4974 was merged
GitHub issue #5078: Fix header level in Audio docs
State: closed | Created: 2022-10-05T20:22:44Z

Fixes header level so `Dataset features` is the doc title instead of `The Audio type`:

![Screen Shot 2022-10-05 at 1 22 02 PM](https://user-images.githubusercontent.com/59462357/194155840-eeb5d62f-f4eb-411e-b281-8494c5fffdce.png)
GitHub issue #5075: Throw EnvironmentError when token is not present
State: closed | Created: 2022-10-05T14:14:18Z

Throw EnvironmentError instead of OSError ([link](https://github.com/huggingface/datasets/blob/6ad430ba0cdeeb601170f732d4bd977f5c04594d/src/datasets/arrow_dataset.py#L4306) to the line) in `push_to_hub` when the Hub token is not present.
GitHub issue #5074: Replace AssertionErrors with more meaningful errors
State: closed | Created: 2022-10-05T14:03:55Z

Replace the AssertionErrors with more meaningful errors such as ValueError, TypeError, etc.

The files with AssertionErrors that need to be replaced:
```
src/datasets/arrow_reader.py
src/datasets/builder.py
src/datasets/utils/version.py
```
GitHub issue #5073: Restore saved format state in `load_from_disk`
State: closed | Created: 2022-10-05T13:51:47Z

Hello! @mariosasko 

This pull request relates to issue #5050 and intends to add the format to datasets loaded from disk. 
All I did was add a set_format in the Dataset.load_from_disk, as DatasetDict.load_from_disk relies on the first.

I don't know if I should add a test and where, so let me know if I should and I can work on that as well!
GitHub issue #5072: Image & Audio formatting for numpy/torch/tf/jax
State: closed | Created: 2022-10-05T13:07:03Z

Added support for image and audio formatting for numpy, torch, tf and jax.

For images, the dtype used is the one of the image (the one returned by PIL.Image), e.g. uint8

I also added support for string, binary and None types. In particular for torch and jax, strings are kept unchanged (previously it was returning an error because you can't create a tensor of strings)
GitHub issue #5070: Support default config name when no builder configs
State: closed | Created: 2022-10-04T19:49:35Z

**Is your feature request related to a problem? Please describe.**
As discussed with @stas00, we could support defining a default config name, even if no predefined allowed config names are set. That is, support `DEFAULT_CONFIG_NAME`, even when `BUILDER_CONFIGS` is not defined. 

**Additional context**
In order to support creating configs on the fly **by name** (not using kwargs), the list of allowed builder configs `BUILDER_CONFIGS` must not be set.

However, if so, then `DEFAULT_CONFIG_NAME` is not supported.
GitHub issue #5067: Fix CONTRIBUTING once dataset scripts transferred to Hub
State: closed | Created: 2022-10-04T14:16:05Z

This PR updates the `CONTRIBUTING.md` guide, once the all dataset scripts have been removed from the GitHub repo and transferred to the HF Hub:
- #4974

See diff here: https://github.com/huggingface/datasets/commit/e3291ecff9e54f09fcee3f313f051a03fdc3d94b

Additionally, this PR fixes the line separator that by some previous mistake was CRLF instead of LF.
GitHub issue #5064: Align signature of create/delete_repo with latest hfh
State: closed | Created: 2022-10-04T09:54:53Z

This PR aligns the signature of `create_repo`/`delete_repo` with the current one in hfh, by removing deprecated `name` and `organization`, and using `repo_id` instead.

Related to:
- #5063

CC: @lhoestq
GitHub issue #5063: Align signature of list_repo_files with latest hfh
State: closed | Created: 2022-10-04T08:51:46Z

This PR aligns the signature of `list_repo_files` with the current one in `hfh`, by renaming deprecated `token` to  `use_auth_token`.

This is already the case for `dataset_info`.

CC: @lhoestq
GitHub issue #5062: Fix CI hfh token warning
State: closed | Created: 2022-10-04T06:36:54Z

In our CI, we get warnings from `hfh` about using deprecated `token`: https://github.com/huggingface/datasets/actions/runs/3174626525/jobs/5171672431
```
tests/test_upstream_hub.py::TestPushToHub::test_push_dataset_dict_to_hub_private
tests/test_upstream_hub.py::TestPushToHub::test_push_dataset_dict_to_hub
tests/test_upstream_hub.py::TestPushToHub::test_push_dataset_dict_to_hub_multiple_files
tests/test_upstream_hub.py::TestPushToHub::test_push_dataset_dict_to_hub_multiple_files_with_max_shard_size
tests/test_upstream_hub.py::TestPushToHub::test_push_dataset_dict_to_hub_overwrite_files
  C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\site-packages\huggingface_hub\utils\_deprecation.py:97: FutureWarning: Deprecated argument(s) used in 'dataset_info': token. Will not be supported from version '0.12'.
    warnings.warn(message, FutureWarning)
```

This PR fixes the tests in `TestPushToHub` so that we fix these warnings.

Continuation of:
- #5031

CC: @lhoestq
GitHub issue #5058: Mark CI tests as xfail when 502 error
State: closed | Created: 2022-10-03T15:53:55Z

To make CI more robust, we could mark as xfail when the Hub raises a 502 error (besides 500 error):

- FAILED tests/test_upstream_hub.py::TestPushToHub::test_push_dataset_to_hub_skip_identical_files
  - https://github.com/huggingface/datasets/actions/runs/3174626525/jobs/5171672431
  ```
  >           raise HTTPError(http_error_msg, response=self)
  E           requests.exceptions.HTTPError: 502 Server Error: Bad Gateway for url: https://hub-ci.huggingface.co/datasets/__DUMMY_TRANSFORMERS_USER__/test-16648055339047.git/info/lfs/objects/batch
  ```

- FAILED tests/test_upstream_hub.py::TestPushToHub::test_push_dataset_dict_to_hub_overwrite_files
  - https://github.com/huggingface/datasets/actions/runs/3145587033/jobs/5113074889
  ```
  >           raise HTTPError(http_error_msg, response=self)
  E           requests.exceptions.HTTPError: 502 Server Error: Bad Gateway for url: https://hub-ci.huggingface.co/datasets/__DUMMY_TRANSFORMERS_USER__/test-16643866807322.git/info/lfs/objects/verify
  ```

Currently, we mark as xfail when 500 error:
- #4845
GitHub issue #5055: Fix backward compatibility for dataset_infos.json
State: closed | Created: 2022-10-03T10:30:14Z

While working on https://github.com/huggingface/datasets/pull/5018 I noticed a small bug introduced in #4926 regarding backward compatibility for dataset_infos.json

Indeed, when a dataset repo had both dataset_infos.json and README.md, the JSON file was ignored. This is unexpected: in practice it should be ignored only if the README.md has a dataset_info field, which has precedence over the data in the JSON file.
GitHub issue #5054: Fix license/citation information of squadshifts dataset card
State: closed | Created: 2022-10-03T05:19:13Z

This PR fixes the license/citation information of squadshifts dataset card, once the dataset owners have responded to our request for information:
- https://github.com/modestyachts/squadshifts-website/issues/1

Additionally, we have updated the mention in their website to our `datasets` library (they were referring old name `nlp`):
- https://github.com/modestyachts/squadshifts-website/pull/2#event-7500953009
GitHub issue #5051: Revert task removal in folder-based builders
State: closed | Created: 2022-09-30T14:50:03Z

Reverts the removal of `task_templates` in the folder-based builders. I also added the `AudioClassifaction` task for consistency.

This is needed to fix https://github.com/huggingface/transformers/issues/19177.

I think we should soon deprecate and remove the current task API (and investigate if it's possible to integrate the `train eval index` API), but we need to update the Transformers examples before that so we don't break them.

cc @NielsRogge
GitHub issue #5050: Restore saved format state in `load_from_disk`
State: closed | Created: 2022-09-30T12:40:07Z

Even though we save the `format` state in `save_to_disk`, we don't restore it in `load_from_disk`. We should fix that.
 
Reported here: https://discuss.huggingface.co/t/save-to-disk-loses-formatting-information/23815
GitHub issue #5048: Fix bug with labels of eurlex config of lex_glue dataset
State: closed | Created: 2022-09-30T09:47:12Z

Fix for a critical bug in the EURLEX dataset label list to make LexGLUE EURLEX results replicable. 

In LexGLUE (Chalkidis et al., 2022), the following is mentioned w.r.t. EUR-LEX: _"It supports four different label granularities, comprising 21, 127, 567, 7390 EuroVoc concepts, respectively. We use the 100 most frequent concepts from level 2 [...]”._ The current label list has all 127 labels, which leads to different (lower) results, as communicated by users.

Thanks!
GitHub issue #5046: Audiofolder creates empty Dataset if files same level as metadata
State: closed | Created: 2022-09-29T19:17:23Z

## Describe the bug
When audio files are at the same level as the metadata (`metadata.csv` or `metadata.jsonl` ), the `load_dataset` returns a `DatasetDict` with no rows but the correct columns.

https://github.com/huggingface/datasets/blob/1ea4d091b7a4b83a85b2eeb8df65115d39af3766/docs/source/audio_dataset.mdx?plain=1#L88

## Steps to reproduce the bug
`metadata.csv`:
```csv
file_name,duration,transcription
./2063_fe9936e7-62b2-4e62-a276-acbd344480ce_1.wav,10.768,hello
```


```python
>>> audio_dataset = load_dataset("audiofolder", data_dir="/audio-data/")
>>> audio_dataset
DatasetDict({
    train: Dataset({
        features: ['audio', 'duration', 'transcription'],
        num_rows: 0
    })
    validation: Dataset({
        features: ['audio', 'duration', 'transcription'],
        num_rows: 0
    })
})
```

I've tried, with no success,:
- setting `split` to something else so I don't get a `DatasetDict`,
- removing the `./`,
- using `.jsonl`.


## Expected results
```
 Dataset({
        features: ['audio', 'duration', 'transcription'],
        num_rows: 1
})
```

## Actual results
```
DatasetDict({
    train: Dataset({
        features: ['audio', 'duration', 'transcription'],
        num_rows: 0
    })
    validation: Dataset({
        features: ['audio', 'duration', 'transcription'],
        num_rows: 0
    })
})
```

## Environment info
- `datasets` version: 2.5.1
- Platform: Linux-5.13.0-1025-aws-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyArrow version: 9.0.0
- Pandas version: 1.5.0
GitHub issue #5045: Automatically revert to last successful commit to hub when a push_to_hub is interrupted
State: closed | Created: 2022-09-29T18:08:12Z

**Is your feature request related to a problem? Please describe.**
I pushed a modification of a large dataset (remove a column) to the hub. The push was interrupted after some files were committed to the repo. This left the dataset to raise an error on load_dataset() (ValueError couldn’t cast … because column names don’t match). Only by specifying the previous (complete) commit as revision=commit_hash in load_data(), I was able to repair this and after a successful, complete push, the dataset loads without error again.

**Describe the solution you'd like**
Would it make sense to detect an incomplete push_to_hub() and automatically revert to the previous commit/revision?

**Describe alternatives you've considered**
Leave everything as is, the revision parameter in load_dataset() allows to manually fix this problem.

**Additional context**
Provide useful defaults
GitHub issue #5039: Hendrycks Checksum
State: closed | Created: 2022-09-29T06:56:20Z

Hi, 
The checksum for [hendrycks_test](https://huggingface.co/datasets/hendrycks_test) does not compare correctly, I guess it has been updated on the remote.

```
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://people.eecs.berkeley.edu/~hendrycks/data.tar']
```
GitHub issue #5038: `Dataset.unique` showing wrong output after filtering
State: closed | Created: 2022-09-28T16:20:35Z

## Describe the bug
After filtering a dataset, and if no samples remain, `Dataset.unique` will return the unique values of the unfiltered dataset.

## Steps to reproduce the bug
```python
from datasets import Dataset

dataset = Dataset.from_dict({'id': [0]})
dataset = dataset.filter(lambda _: False)
print(dataset.unique('id'))
```

## Expected results
The above code should return an empty list since the dataset is empty.

## Actual results
```bash
[0]
```

## Environment info
- `datasets` version: 2.5.1
- Platform: Linux-5.18.19-100.fc35.x86_64-x86_64-with-glibc2.34
- Python version: 3.9.14
- PyArrow version: 7.0.0
- Pandas version: 1.3.5
GitHub issue #5036: Add oversampling strategy iterable datasets interleave
State: closed | Created: 2022-09-28T10:10:23Z

Hello everyone,
Following the issue #4893 and the PR #4831, I propose here an oversampling strategy for a `IterableDataset` list.  
The `all_exhausted` strategy stops building the new dataset as soon as all samples in each dataset have been added at least once. 
It follows roughly the same logic behind #4831, namely:
- if ``probabilities`` is `None` and the strategy is `all_exhausted`, it simply performs a round robin interleaving that stops when the longest dataset is out of samples. Here the new dataset length will be $maxLengthDataset*nbDataset$.
- if ``probabilities`` is not `None` and the strategy is `all_exhausted`, it keeps trace of the datasets which were out of samples but continues to add them to the new dataset, and stops as soons as every dataset runs out of samples at least once.

In order to be consistent and also to align with the `Dataset` behavior, please note that the behavior of the default strategy (`first_exhausted`) has been changed. Namely, it really stops when a dataset is out of samples whereas it used to stop when receiving the `StopIteration` error.  
To give an example of the last note, with the following snippet:
```
>>> from tests.test_iterable_dataset import *
>>> d1 = IterableDataset(ExamplesIterable((lambda: (yield from [(i, {"a": i}) for i in [0, 1, 2]])), {}))
>>> d2 = IterableDataset(ExamplesIterable((lambda: (yield from [(i, {"a": i}) for i in [10, 11, 12, 13]])), {}))
>>> d3 = IterableDataset(ExamplesIterable((lambda: (yield from [(i, {"a": i}) for i in [20, 21, 22, 23, 24]])), {}))
>>> dataset = interleave_datasets([d1, d2, d3])
>>> [x["a"] for x in dataset]
```
The result here will then be `[10, 0, 11, 1, 2]` instead of `[10, 0, 11, 1, 2, 20, 12, 13]`.

I modified the behavior because I found it to be consistent with the under/oversampling approach and because it unified the undersampling and oversampling code, but I stay open to any suggestions.
GitHub issue #5032: new dataset type: single-label and multi-label video classification
State: open | Created: 2022-09-27T19:40:11Z

**Is your feature request related to a problem? Please describe.**
In my research, I am dealing with multi-modal (audio+text+frame sequence) video classification. It would be great if the datasets library supported generating multi-modal batches from a video dataset.

**Describe the solution you'd like**
Assume I have video files having single/multiple labels. I want to train a single/multi-label video classification model. I want datasets to support generating multi-modal batches (audio+frame sequence) from video files. Audio waveform and frame sequence can be extracted from each video clip then I can use any audio, image and video model from transformers library to extract features which will be fed into my model.

**Describe alternatives you've considered**
Currently, I am using https://github.com/facebookresearch/pytorchvideo dataloaders. There seems to be not much alternative.

**Additional context**
I am wiling to open a PR but don't know where to start.
GitHub issue #5031: Support hfh 0.10 implicit auth
State: closed | Created: 2022-09-27T18:37:49Z

In huggingface-hub 0.10 the `token` parameter is deprecated for dataset_info and list_repo_files in favor of use_auth_token.
Moreover if use_auth_token=None then the user's token is used implicitly.

I took those two changes into account

Close https://github.com/huggingface/datasets/issues/4990

TODO:
- [x] fix tests

We should wait hfh 0.10 to be relased first to make sure it works correctly before merging
GitHub issue #5030: Fast dataset iter
State: closed | Created: 2022-09-27T16:44:51Z

Use `pa.Table.to_reader` to make iteration over examples/batches faster in `Dataset.{__iter__, map}`

TODO:
* [x] benchmarking (the only benchmark for now - iterating over (single) examples of `bookcorpus` (75 mil examples) in Colab is approx. 2.3x faster)
* [x] check if iterating over bigger chunks + slicing to fetch individual examples in `_iter` yields better performance
GitHub issue #5028: passing parameters to the method passed to Dataset.from_generator()
State: closed | Created: 2022-09-26T15:20:06Z

Big thanks for providing dataset creation via a generator.
I want to ask whether there is any way that parameters can be passed to the method Dataset.from_generator() method, like as follows. 

```
from datasets import Dataset

def gen(param1):
    for idx in len(custom_dataset):
        yield custom_dataset[idx] + param1

ds = Dataset.from_generator(gen(param1))
```
GitHub issue #5026: patch CI_HUB_TOKEN_PATH with Path instead of str
State: closed | Created: 2022-09-26T13:19:01Z

Should fix the tests for `huggingface_hub==0.10.0rc0` prerelease (see [failed CI](https://github.com/huggingface/datasets/actions/runs/3127805250/jobs/5074879144)). 
Related to [this thread](https://huggingface.slack.com/archives/C02V5EA0A95/p1664195165294559) (internal link).

Note: this should be a backward compatible fix (e.g. works also with previous versions of `huggingface_hub`)

I am not sure where to put the changes so feel free to cherry-pick the commit and close this one without merging.

cc @lhoestq
GitHub issue #5023: Text strings are split into lists of characters in xcsr dataset
State: closed | Created: 2022-09-26T11:11:50Z

## Describe the bug
Text strings are split into lists of characters.

Example for "X-CSQA-en":
```
{'id': 'd3845adc08414fda',
 'lang': 'en',
 'question': {'stem': ['T',
   'h',
   'e',
   ' ',
   'd',
   'e',
   'n',
   't',
   'a',
   'l',
   ' ',
   'o',
   'f',
   'f',
   'i',
   'c',
   'e',
   ' ',
   'h',
   'a',
   'n',
   'd',
   'l',
   'e',
   'd',
   ' ',
   'a',
   ' ',
   'l',
   'o',
   't',
   ' ',
   'o',
   'f',
   ' ',
   'p',
   'a',
   't',
   'i',
   'e',
   'n',
   't',
   's',
   ' ',
   'w',
   'h',
   'o',
   ' ',
   'e',
   'x',
   'p',
   'e',
   'r',
   'i',
   'e',
   'n',
   'c',
   'e',
   'd',
   ' ',
   't',
   'r',
   'a',
   'u',
   'm',
   'a',
   't',
   'i',
   'c',
   ' ',
   'm',
   'o',
   'u',
   't',
   'h',
   ' ',
   'i',
   'n',
   'j',
   'u',
   'r',
   'y',
   ',',
   ' ',
   'w',
   'h',
   'e',
   'r',
   'e',
   ' ',
   'w',
   'e',
   'r',
   'e',
   ' ',
   't',
   'h',
   'e',
   's',
   'e',
   ' ',
   'p',
   'a',
   't',
   'i',
   'e',
   'n',
   't',
   's',
   ' ',
   'c',
   'o',
   'm',
   'i',
   'n',
   'g',
   ' ',
   'f',
   'r',
   'o',
   'm',
   '?'],
  'choices': [{'label': ['A'], 'text': ['t', 'o', 'w', 'n']},
   {'label': ['B'], 'text': ['m', 'i', 'c', 'h', 'i', 'g', 'a', 'n']},
   {'label': ['C'], 'text': ['h', 'o', 's', 'p', 'i', 't', 'a', 'l']},
   {'label': ['D'], 'text': ['s', 'c', 'h', 'o', 'o', 'l', 's']},
   {'label': ['E'],
    'text': ['o',
     'f',
     'f',
     'i',
     'c',
     'e',
     ' ',
     'b',
     'u',
     'i',
     'l',
     'd',
     'i',
     'n',
     'g']}]},
 'answerKey': 'C'}

## Steps to reproduce the bug
```python
ds = load_dataset("datasets/xcsr", "X-CSQA-en", split="validation", streaming=True)
item = next(iter(ds))
item
```

## Expected results
```
{'id': 'd3845adc08414fda',
 'lang': 'en',
 'question': {'stem': 'The dental office handled a lot of patients who experienced traumatic mouth injury, where were these patients coming from?',
  'choices': {'label': ['A', 'B', 'C', 'D', 'E'],
   'text': ['town', 'michigan', 'hospital', 'schools', 'office building']}},
 'answerKey': 'C'}
```
GitHub issue #5021: Split is inferred from filename and overrides metadata.jsonl
State: closed | Created: 2022-09-26T03:22:14Z

## Describe the bug

Including the strings "test" or "train" anywhere in a filename causes `datasets` to infer the split and silently ignore all other files.

This behavior is documented for directory names but not filenames: https://huggingface.co/docs/datasets/image_dataset#imagefolder

## Steps to reproduce the bug
`metadata.jsonl`
```json
{"file_name": "photo of a cat.jpg", "text": "a photo of a cat"}
{"file_name": "photo of a dog.jpg", "text": "a photo of a dog"}
{"file_name": "photo of a train.jpg", "text": "a photo of a train"}
{"file_name": "photo of test tubes.jpg", "text": "a photo of test tubes"}
```

`bug.py`
```python
from datasets import load_dataset

dataset = load_dataset("dataset")

print(dataset)
# DatasetDict({
#     train: Dataset({
#         features: ['image', 'text'],
#         num_rows: 1
#     })
#     test: Dataset({
#         features: ['image', 'text'],
#         num_rows: 1
#     })
# })

for split in dataset:
    for n in dataset[split]:
        print(n['text'])
        # a photo of a train
        # a photo of test tubes
```

## Expected results
One single dataset with all four images / a warning for unused files / documentation of this behavior

## Actual results
Only the images with "test" or "train" in the name are loaded

## Environment info
- `datasets` version: 2.5.1
- Platform: macOS-12.5.1-x86_64-i386-64bit
- Python version: 3.10.4
- PyArrow version: 9.0.0
- Pandas version: 1.5.0
GitHub issue #5020: Fix URLs of sbu_captions dataset
State: closed | Created: 2022-09-24T14:00:33Z

Forbidden
You don't have permission to access /~vicente/sbucaptions/sbu-captions-all.tar.gz on this server.

Additionally, a 403 Forbidden error was encountered while trying to use an ErrorDocument to handle the request.

Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.2k-fips PHP/5.4.16 mod_fcgid/2.3.9 mod_wsgi/3.4 Python/2.7.5 mod_perl/2.0.11 Perl/v5.16.3 Server at [www.cs.virginia.edu](mailto:csroot@virginia.edu) Port 443
GitHub issue #5019: Update swiss judgment prediction
State: closed | Created: 2022-09-24T13:28:57Z

Hi,

I updated the dataset to include additional data made available recently. When I test it locally, it seems to work. However, I get the following error with the dummy data creation: 
`Dummy data generation done but dummy data test failed since splits ['train', 'validation', 'test'] have 0 examples for config 'fr'`. Do you know why this could be the case?

Cheers,
Joel
GitHub issue #5018: Create all YAML dataset_info
State: closed | Created: 2022-09-23T18:08:15Z

Following https://github.com/huggingface/datasets/pull/4926

Creates all the `dataset_info` YAML fields in the dataset cards
The JSON are also updated using the simplified backward compatible format added in https://github.com/huggingface/datasets/pull/4926

Needs https://github.com/huggingface/datasets/pull/4926 to be merged first
GitHub issue #5017: xcsr: X-CSQA simply uses english for all alleged non-english data
State: closed | Created: 2022-09-23T16:11:54Z

## Describe the bug
All the alleged non-english subcollections for the X-CSQA task in the [xcsr benchmark dataset ](https://huggingface.co/datasets/xcsr) seem to be copies of the english subcollection, rather than translations. This is in contrast to the data description:

> we automatically translate the original CSQA and CODAH datasets, which only have English versions, to 15 other languages, forming development and test sets for studying X-CSR
 
## Steps to reproduce the bug
```python
# let's say you want to load the french X-CSQA subcollection
french = datasets.load_dataset("xcsr", "X-CSQA-fr")
# for good measure, let's load english too
english = datasets.load_dataset("xcsr", "X-CSQA-en")

# let's inspect
"".join(english['test'][0]['question']['stem'])
# output: 'The people wanted to stop the parade, so what did they set up to thwart it?'
"".join(french['test'][0]['question']['stem'])
# output: 'The people wanted to stop the parade, so what did they set up to thwart it?'
# what? Why are they both in english?

# I've checked this for validation and train splits too, across many datapoints. It's all the same english dataset
# maybe i need to look better?
french['test'].unique('lang')
# output: ['en']
# no, it's all english

```

## Expected results
Accessing a subcollection in language X should return a subcollection containg samples in language X

## Actual results
Accessing a subcollection in language X returns a subcollection containing samples in English.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->

- `datasets` version: 2.5.1
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.13
- PyArrow version: 9.0.0
- Pandas version: 1.4.3
GitHub issue #5016: Fix tar extraction vuln
State: closed | Created: 2022-09-23T14:22:21Z

Fix for CVE-2007-4559

Description:
Directory traversal vulnerability in the (1) extract and (2) extractall functions in the tarfile
module in Python allows user-assisted remote attackers to overwrite arbitrary files via a .. (dot dot)
sequence in filenames in a TAR archive, a related issue to CVE-2001-1267.

I fixed it by using the solution proposed in https://stackoverflow.com/questions/10060069/safely-extract-zip-or-tar-using-python

It blocks extraction of files with an absolute path or double dots and symlinks.
GitHub issue #5015: Transfer dataset scripts to Hub
State: closed | Created: 2022-09-23T08:48:10Z

Before merging:
- #4974 

TODO:
- [x] Create label: ["dataset contribution"](https://github.com/huggingface/datasets/pulls?q=label%3A%22dataset+contribution%22)
- [x] Create project: [Datasets: Transfer datasets to Hub](https://github.com/orgs/huggingface/projects/22/)
- [x] PRs:
  - [x] Add dataset: we should recommend transfer all additions of datasets to the Hub, under the appropriate namespace; no more additions of datasets on GitHub
  - [x] Update dataset: in general, we should merge bug fixes; enhancements should be considered on a case-by-case basis, depending on whether there is a more suitable namespace on the Hub
- [ ] Issues

Finally:
- [x] #4974

Let me know what you think! :hugs:
GitHub issue #5012: Force JSON format regardless of file naming on S3
State: closed | Created: 2022-09-22T18:28:15Z

I have a file on S3 created by Data Version Control, it looks like `s3://dvc/ac/badff5b134382a0f25248f1b45d7b2` but contains a json file. If I run 
```python
dataset = load_dataset(
    "json",
    data_files='s3://dvc/ac/badff5b134382a0f25248f1b45d7b2' 
)
```
It gives me 
```
InvalidSchema: No connection adapters were found for 's3://dvc/ac/badff5b134382a0f25248f1b45d7b2'
```
However, I cannot go ahead and change the names of the s3 file. Is there a way to "force" load a S3 url with certain decoder (JSON, CSV, etc.) regardless of s3 URL naming?
GitHub issue #5010: Add deprecation warning to multilingual_librispeech dataset card
State: closed | Created: 2022-09-22T11:41:59Z

Besides the current deprecation warning in the script of `multilingual_librispeech`, this PR adds a deprecation warning to its dataset card as well.

The format of the deprecation warning is aligned with the one in the library documentation when docstrings contain the `<Deprecated/>` tag.

Related to:
- #4060
GitHub issue #5005: Release 2.5.0 breaks transformers CI
State: closed | Created: 2022-09-21T13:39:19Z

## Describe the bug
As reported by @lhoestq:

> see https://app.circleci.com/pipelines/github/huggingface/transformers/47634/workflows/b491886b-e66e-4edb-af96-8b459e72aa25/jobs/564563
this is used here: [https://github.com/huggingface/transformers/blob/3b19c0317b6909e2d7f11b5053895ac55[…]torch/speech-pretraining/run_wav2vec2_pretraining_no_trainer.py](https://github.com/huggingface/transformers/blob/3b19c0317b6909e2d7f11b5053895ac55250e7da/examples/pytorch/speech-pretraining/run_wav2vec2_pretraining_no_trainer.py#L482-L488)
GitHub issue #5002: Dataset Viewer issue for loubnabnl/humaneval-x
State: closed | Created: 2022-09-21T09:06:17Z

### Link

https://huggingface.co/datasets/loubnabnl/humaneval-x/viewer/

### Description

The dataset has subsets but the viewer gets stuck in the default subset even when I select another one (the data loading of the subsets works fine)

### Owner

Yes
GitHub issue #4999: Add EmptyDatasetError
State: closed | Created: 2022-09-20T15:28:05Z

examples:

from the hub:
```python
Traceback (most recent call last):
  File "playground/ttest.py", line 3, in <module>
    print(load_dataset("lhoestq/empty"))
  File "/Users/quentinlhoest/Desktop/hf/nlp/src/datasets/load.py", line 1686, in load_dataset
    **config_kwargs,
  File "/Users/quentinlhoest/Desktop/hf/nlp/src/datasets/load.py", line 1458, in load_dataset_builder
    data_files=data_files,
  File "/Users/quentinlhoest/Desktop/hf/nlp/src/datasets/load.py", line 1171, in dataset_module_factory
    raise e1 from None
  File "/Users/quentinlhoest/Desktop/hf/nlp/src/datasets/load.py", line 1162, in dataset_module_factory
    download_mode=download_mode,
  File "/Users/quentinlhoest/Desktop/hf/nlp/src/datasets/load.py", line 760, in get_module
    else get_data_patterns_in_dataset_repository(hfh_dataset_info, self.data_dir)
  File "/Users/quentinlhoest/Desktop/hf/nlp/src/datasets/data_files.py", line 678, in get_data_patterns_in_dataset_repository
    ) from None
datasets.data_files.EmptyDatasetError: The dataset repository at 'lhoestq/empty' doesn't contain any data file.
```

from local directory:
```python
Traceback (most recent call last):
  File "playground/ttest.py", line 3, in <module>
    print(load_dataset("playground/empty"))
  File "/Users/quentinlhoest/Desktop/hf/nlp/src/datasets/load.py", line 1686, in load_dataset
    **config_kwargs,
  File "/Users/quentinlhoest/Desktop/hf/nlp/src/datasets/load.py", line 1458, in load_dataset_builder
    data_files=data_files,
  File "/Users/quentinlhoest/Desktop/hf/nlp/src/datasets/load.py", line 1107, in dataset_module_factory
    path, data_dir=data_dir, data_files=data_files, download_mode=download_mode
  File "/Users/quentinlhoest/Desktop/hf/nlp/src/datasets/load.py", line 625, in get_module
    else get_data_patterns_locally(base_path)
  File "/Users/quentinlhoest/Desktop/hf/nlp/src/datasets/data_files.py", line 460, in get_data_patterns_locally
    raise EmptyDatasetError(f"The directory at {base_path} doesn't contain any data file") from None
datasets.data_files.EmptyDatasetError: The directory at playground/empty doesn't contain any data file
```

Close https://github.com/huggingface/datasets/issues/4995
GitHub issue #4998: Don't add a tag on the Hub on release
State: closed | Created: 2022-09-20T13:54:57Z

Datasets with no namespace on the Hub have tags to redirect to the version of datasets where they come from.
I’m about to remove them all because I think it looks bad/unexpected in the UI and it’s not actually useful

Therefore I'm also disabling tagging.

Note that the CI job will be completely removed in https://github.com/huggingface/datasets/pull/4974 anyway