Dataset wikipedia and Bookcorpusopen cannot be fetched from dataloader.

Something isn't working

Hi ! I think this issue is more related to the `transformers` project. Could you open an issue on https://github.com/huggingface/transformers ?

Anyway I think the issue could be that both wikipedia and bookcorpusopen have an additional "title" column, contrary to wikitext which only has a "text" column. After calling `load_dataset`, can you try doing `dataset = dataset.remove_columns("title")` ?

Removing the "title" column works! Thanks for your advice.

Maybe I should still create an issue to `transformers' to mark this solution?

## Describe the bug
I have used the previous version of `transformers` and `datasets`. The dataset `wikipedia` can be successfully used. Recently, I upgrade them to the newest version and find it raises errors. I also tried other datasets. The `wikitext` works and the `bookcorpusopen` raises the same errors as `wikipedia`.

## Steps to reproduce the bug
Run the `run_mlm_no_trainer.py` and the given script on this [link](https://github.com/huggingface/transformers/tree/master/examples/pytorch/language-modeling). Change the dataset from wikitext to wikipedia or bookcorpusopen. BTW, the library transformers is of version 4.11.3.

## Expected results
The data batchs are fetched from the data loader and train.

## Actual results
The first time to fetch data batch occurs error.
`Traceback (most recent call last):
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 705, in convert_to_tensors
    tensor = as_tensor(value)
ValueError: too many dimensions 'str'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "src/original_run_mlm_no_trainer.py", line 528, in <module>
    main()
  File "src/original_run_mlm_no_trainer.py", line 488, in main
    for step, batch in enumerate(train_dataloader):
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/accelerate/data_loader.py", line 303, in __iter__
    for batch in super().__iter__():
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 517, in __next__
    data = self._next_data()
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 557, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/transformers/data/data_collator.py", line 41, in __call__
    return self.torch_call(features)
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/transformers/data/data_collator.py", line 671, in torch_call
    batch = self.tokenizer.pad(examples, return_tensors="pt", pad_to_multiple_of=self.pad_to_multiple_of)
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 2774, in pad
    return BatchEncoding(batch_outputs, tensor_type=return_tensors)
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 210, in __init__
    self.convert_to_tensors(tensor_type=tensor_type, prepend_batch_axis=prepend_batch_axis)
  File "/home/zyli/anaconda3/envs/LatestStacking/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 722, in convert_to_tensors
    "Unable to create tensor, you should probably activate truncation and/or padding "
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length.
`

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: Linux-5.8.0-59-generic-x86_64-with-debian-bullseye-sid
- Python version: 3.7.6
- PyArrow version: 5.0.0


Error in per class precision computation

bug

Something isn't working

Hi @tidhamecha2, thanks for reporting.

Indeed, we fixed this issue just one week ago: #3008

The fix will be included in our next version release.

In the meantime, you can incorporate the fix by installing `datasets` from the master branch:
```
pip install -U git+ssh://git@github.com/huggingface/datasets.git@master#egg=datasest
```
or
```
pip install -U git+https://github.com/huggingface/datasets.git@master#egg=datasets
```

## Describe the bug
When trying to get the per class precision values by providing `average=None`, following error is thrown `ValueError: can only convert an array of size 1 to a Python scalar`

## Steps to reproduce the bug
```python
from datasets import load_dataset, load_metric
precision_metric = load_metric("precision")
predictions = [0, 2, 1, 0, 0, 1]
references = [0, 1, 2, 0, 1, 2]
results = precision_metric.compute(predictions=predictions, references=references, average=None)
```

## Expected results
`  {'precision': array([0.66666667, 0.        , 0.        ])}`
as per https://github.com/huggingface/datasets/blob/master/metrics/precision/precision.py

## Actual results
```
   output = self._compute(predictions=predictions, references=references, **kwargs)
  File "~/.cache/huggingface/modules/datasets_modules/metrics/precision/94709a71c6fe37171ef49d3466fec24dee9a79846c9f176dff66a649e9811690/precision.py", line 110, in _compute
    sample_weight=sample_weight,
ValueError: can only convert an array of size 1 to a Python scalar
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: linux
- Python version: 3.6.9
- PyArrow version: 5.0.0


CI test suite fails after meteor metric update

Something isn't working

## Describe the bug
CI test suite fails: https://app.circleci.com/pipelines/github/huggingface/datasets/8110/workflows/f059ba43-9154-4632-bebb-82318447ddc9/jobs/50010

Stack trace:
```
___________________ LocalMetricTest.test_load_metric_meteor ____________________
[gw1] linux -- Python 3.6.15 /home/circleci/.pyenv/versions/3.6.15/bin/python3.6

self = <tests.test_metric_common.LocalMetricTest testMethod=test_load_metric_meteor>
metric_name = 'meteor'

    def test_load_metric(self, metric_name):
        doctest.ELLIPSIS_MARKER = "[...]"
        metric_module = importlib.import_module(datasets.load.prepare_module(os.path.join("metrics", metric_name))[0])
        metric = datasets.load.import_main_class(metric_module.__name__, dataset=False)
        # check parameters
        parameters = inspect.signature(metric._compute).parameters
        self.assertTrue("predictions" in parameters)
        self.assertTrue("references" in parameters)
        self.assertTrue(all([p.kind != p.VAR_KEYWORD for p in parameters.values()]))  # no **kwargs
        # run doctest
        with self.patch_intensive_calls(metric_name, metric_module.__name__):
            with self.use_local_metrics():
>               results = doctest.testmod(metric_module, verbose=True, raise_on_error=True)

tests/test_metric_common.py:75:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../.pyenv/versions/3.6.15/lib/python3.6/doctest.py:1951: in testmod
    runner.run(test)
../.pyenv/versions/3.6.15/lib/python3.6/doctest.py:1839: in run
    r = DocTestRunner.run(self, test, compileflags, out, False)
../.pyenv/versions/3.6.15/lib/python3.6/doctest.py:1476: in run
    return self.__run(test, compileflags, out)
../.pyenv/versions/3.6.15/lib/python3.6/doctest.py:1382: in __run
    exception)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <doctest.DebugRunner object at 0x7f4c26bd3da0>
out = <built-in method write of _io.TextIOWrapper object at 0x7f51a21852d0>
test = <DocTest datasets_modules.datasets.meteor.6201bb45d2c144ea7963680949d20f523d74a741fa0f8a806f836e6caa5245d7.meteor.Mete...ets_modules/datasets/meteor/6201bb45d2c144ea7963680949d20f523d74a741fa0f8a806f836e6caa5245d7/meteor.py:87 (5 examples)>
example = <doctest.Example object at 0x7f4c26bd3eb8>
exc_info = (<class 'TypeError'>, TypeError('"hypothesis" expects pre-tokenized hypothesis (Iterable[str]): It is a guide to action which ensures that the military always obeys the commands of the party',), <traceback object at 0x7f4cd01afec8>)

    def report_unexpected_exception(self, out, test, example, exc_info):
>       raise UnexpectedException(test, example, exc_info)
E       doctest.UnexpectedException: <DocTest datasets_modules.datasets.meteor.6201bb45d2c144ea7963680949d20f523d74a741fa0f8a806f836e6caa5245d7.meteor.Meteor from /tmp/pytest-of-circleci/pytest-0/popen-gw1/cache/modules/datasets_modules/datasets/meteor/6201bb45d2c144ea7963680949d20f523d74a741fa0f8a806f836e6caa5245d7/meteor.py:87 (5 examples)>

../.pyenv/versions/3.6.15/lib/python3.6/doctest.py:1845: UnexpectedException
```


load_dataset('the_pile_openwebtext2') produces ArrowInvalid, value too large to fit in C integer type

Something isn't working

I encountered the same bug using different datasets.
any suggestions?

## Describe the bug
When loading `the_pile_openwebtext2`, we get the error `pyarrow.lib.ArrowInvalid: Value 2111 too large to fit in C integer type`

## Steps to reproduce the bug
```python
import datasets
ds = datasets.load_dataset('the_pile_openwebtext2')
```

## Expected results
Should download the dataset, convert it to an arrow file, and return a working Dataset object.

## Actual results
The download works, but conversion to the arrow file fails as follows:

```
>>> ds = datasets.load_dataset('the_pile_openwebtext2')
Downloading and preparing dataset openwebtext2/plain_text (download: 27.33 GiB, generated: 63.86 GiB
, post-processed: Unknown size, total: 91.19 GiB) to /home/davidbau/.cache/huggingface/datasets/open
webtext2/plain_text/1.0.0/c48ec73ba3483bac673463f48f67e9a4fd8cb49a9d6ec4fb957f0b424b97cf25...
Traceback (most recent call last):
  File "/home/davidbau/.conda/envs/tenv/lib/python3.9/site-packages/datasets/builder.py", line 1133,
 in _prepare_split
    writer.write(example, key)
  File "/home/davidbau/.conda/envs/tenv/lib/python3.9/site-packages/datasets/arrow_writer.py", line
366, in write
    self.write_examples_on_file()
  File "/home/davidbau/.conda/envs/tenv/lib/python3.9/site-packages/datasets/arrow_writer.py", line
311, in write_examples_on_file
    pa_array = pa.array(typed_sequence)
  File "pyarrow/array.pxi", line 222, in pyarrow.lib.array
  File "pyarrow/array.pxi", line 110, in pyarrow.lib._handle_arrow_array_protocol
  File "/home/davidbau/.conda/envs/tenv/lib/python3.9/site-packages/datasets/arrow_writer.py", line
115, in __arrow_array__
    out = pa.array(cast_to_python_objects(self.data, only_1d_for_numpy=True), type=type)
  File "pyarrow/array.pxi", line 305, in pyarrow.lib.array
  File "pyarrow/array.pxi", line 39, in pyarrow.lib._sequence_to_array
  File "pyarrow/error.pxi", line 122, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 84, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Value 2111 too large to fit in C integer type
## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:
```

- Platform: Ubuntu 20.04
- Python version: python 3.9
- PyArrow version: 3.0.0


load_dataset cannot download the data and hangs on forever if cache dir specified

Something isn't working

Issue was environment inconsistency, updating packages did the trick

`conda install -c huggingface -c conda-forge datasets`

> Collecting package metadata (current_repodata.json): done
> Solving environment: |
> The environment is inconsistent, please check the package plan carefully
> The following packages are causing the inconsistency:
>
>   - conda-forge/noarch::datasets==1.12.1=pyhd8ed1ab_1
>   - conda-forge/win-64::multiprocess==0.70.12.2=py38h294d835_0
> done
>
> Package Plan
>
>   environment location: C:\xxx\anaconda3\envs\UnBias-94-1
>
>   added / updated specs:
>     - datasets
>
>
> The following NEW packages will be INSTALLED:
>
>   dill               conda-forge/noarch::dill-0.3.4-pyhd8ed1ab_0
>
> The following packages will be UPDATED:
>
>   ca-certificates    pkgs/main::ca-certificates-2021.9.30-~ --> conda-forge::ca-certificates-2021.10.8-h5b45459_0
>   certifi            pkgs/main::certifi-2021.5.30-py38haa9~ --> conda-forge::certifi-2021.10.8-py38haa244fe_0
>
> The following packages will be SUPERSEDED by a higher-priority channel:
>

## Describe the bug
After updating datasets, a code that ran just fine for ages began to fail. Specifying _datasets.load_dataset_'s _cache_dir_ optional argument on Windows 10 machine results in data download to hang on forever. Same call without cache_dir works just fine. Surprisingly exact same code just runs perfectly fine on Linux docker instance running in cloud.

Unfortunately I updated Windows also at the same time and I can't remember which version of datasets was running in my conda environment prior to the update otherwise I would have tried both to check this out.  :(

## Steps to reproduce the bug
```python
# Sample code to reproduce the bug
```
cache_dir = 'c:/data/datasets'
dataset = load_dataset('wikipedia', '20200501.en', split='train',cache_dir=cache_dir)
```
Note that exact same code without specifying _cache_dir_ argument works perfectly fine.
```
cache_dir = 'c:/data/datasets'
dataset = load_dataset('wikipedia', '20200501.en', split='train')
```

## Expected results
Downloads the dataset and cache is handled in the _cache_dir_ directory
## Actual results
Data download keeps hanging on forever, **NO TRACEBACK**!

## Environment info
- `datasets` version: 1.12.1
- Platform: Windows-10-10.0.19042-SP0
- Python version: 3.8.11
- PyArrow version: 3.0.0


Non-Matching Checksum Error with crd3 dataset

Something isn't working

I got the same error for another dataset (`multi_woz_v22`):

```
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://github.com/budzianowski/multiwoz/raw/master/data/MultiWOZ_2.2/dialog_acts.json', 'https://github.com/budzianowski/multiwoz/raw/master/data/MultiWOZ_2.2/test/dialogues_001.json']
```

I'm seeing the same issue as @RylanSchaeffer:
Python 3.7.11, macOs 11.4
datasets==1.14.0

fails on:
```python
dataset = datasets.load_dataset("multi_woz_v22")
```

## Describe the bug
When I try loading the crd3 dataset (https://huggingface.co/datasets/crd3), an error is thrown.

## Steps to reproduce the bug
```python
dataset = load_dataset('crd3', split='train')
```

## Expected results
I expect no error to be thrown.

## Actual results
A non-matching checksum error is thrown.

```
datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://github.com/RevanthRameshkumar/CRD3/archive/master.zip']
```

## Environment info

- `datasets` version: 1.12.1
- Platform: Linux-4.4.0-173-generic-x86_64-with-Ubuntu-16.04-xenial
- Python version: 3.6.10
- PyArrow version: 5.0.0


TimeoutError during streaming

Something isn't working

## Describe the bug
I got a TimeoutError after streaming for about 10h.

## Steps to reproduce the bug
Very long code but we could do a test of streaming indefinitely data, though error may take a while to appear.

## Expected results
This error was not expected in the code which considers only `ClientError` but not `TimeoutError`.
See [this line](https://github.com/huggingface/datasets/blob/2814fbd0e18150be409f10804670e98d9ecb87d4/src/datasets/utils/streaming_download_manager.py#L129).
Based on the traceback, it looks like the `TimeoutError` was not captured.

## Actual results
```
  File "/home/koush/.pyenv/versions/dev/lib/python3.9/site-packages/fsspec/asyn.py", line 25, in _runner
    result[0] = await coro
  File "/home/koush/.pyenv/versions/dev/lib/python3.9/site-packages/fsspec/implementations/http.py", line 614, in async_fetch_range
    out = await r.read()
  File "/home/koush/.pyenv/versions/dev/lib/python3.9/site-packages/aiohttp/client_reqrep.py", line 1032, in read
    self._body = await self.content.read()
  File "/home/koush/.pyenv/versions/dev/lib/python3.9/site-packages/aiohttp/streams.py", line 370, in read
    block = await self.readany()
  File "/home/koush/.pyenv/versions/dev/lib/python3.9/site-packages/aiohttp/streams.py", line 392, in readany
    await self._wait("readany")
  File "/home/koush/.pyenv/versions/dev/lib/python3.9/site-packages/aiohttp/streams.py", line 306, in _wait
    await waiter
  File "/home/koush/.pyenv/versions/dev/lib/python3.9/site-packages/aiohttp/helpers.py", line 656, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/home/koush/dalle-mini/dev/seq2seq/run_seq2seq_flax.py", line 1027, in <module>
    main()
  File "/home/koush/dalle-mini/dev/seq2seq/run_seq2seq_flax.py", line 991, in main
    for batch in tqdm(
  File "/home/koush/.pyenv/versions/dev/lib/python3.9/site-packages/tqdm/std.py", line 1180, in __iter__
    for obj in iterable:
  File "/home/koush/dalle-mini/dev/seq2seq/run_seq2seq_flax.py", line 376, in data_loader_streaming
    for item in dataset:
  File "/home/koush/datasets/src/datasets/iterable_dataset.py", line 341, in __iter__
    for key, example in self._iter():
  File "/home/koush/datasets/src/datasets/iterable_dataset.py", line 338, in _iter
    yield from ex_iterable
  File "/home/koush/datasets/src/datasets/iterable_dataset.py", line 179, in __iter__
    key_examples_list = [(key, example)] + [
  File "/home/koush/datasets/src/datasets/iterable_dataset.py", line 179, in <listcomp>
    key_examples_list = [(key, example)] + [
  File "/home/koush/datasets/src/datasets/iterable_dataset.py", line 176, in __iter__
    for key, example in iterator:
  File "/home/koush/datasets/src/datasets/iterable_dataset.py", line 225, in __iter__
    for x in self.ex_iterable:
  File "/home/koush/datasets/src/datasets/iterable_dataset.py", line 99, in __iter__
    for key, example in self.generate_examples_fn(**kwargs_with_shuffled_shards):
  File "/home/koush/datasets/src/datasets/iterable_dataset.py", line 287, in wrapper
    for key, table in generate_tables_fn(**kwargs):
  File "/home/koush/datasets/src/datasets/packaged_modules/json/json.py", line 107, in _generate_tables
    batch = f.read(self.config.chunksize)
  File "/home/koush/datasets/src/datasets/utils/streaming_download_manager.py", line 126, in read_with_retries
    out = read(*args, **kwargs)
  File "/home/koush/.pyenv/versions/dev/lib/python3.9/site-packages/fsspec/implementations/http.py", line 572, in read
    return super().read(length)
  File "/home/koush/.pyenv/versions/dev/lib/python3.9/site-packages/fsspec/spec.py", line 1533, in read
    out = self.cache._fetch(self.loc, self.loc + length)
  File "/home/koush/.pyenv/versions/dev/lib/python3.9/site-packages/fsspec/caching.py", line 390, in _fetch
    self.cache = self.fetcher(start, bend)
  File "/home/koush/.pyenv/versions/dev/lib/python3.9/site-packages/fsspec/asyn.py", line 91, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/home/koush/.pyenv/versions/dev/lib/python3.9/site-packages/fsspec/asyn.py", line 69, in sync
    raise FSTimeoutError from return_result
fsspec.exceptions.FSTimeoutError
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.2.dev0
- Platform: Linux-5.4.0-1043-gcp-x86_64-with-glibc2.31
- Python version: 3.9.7
- PyArrow version: 5.0.0


Identify which shard data belongs to

enhancement

New feature or request

Independently of this I think it raises the need to allow multiprocessing during streaming so that we get samples from multiple shards in one batch.

**Is your feature request related to a problem? Please describe.**

I'm training on a large dataset made of multiple sub-datasets.
During training I can observe some jumps in loss which may correspond to different shards.

![image](https://user-images.githubusercontent.com/715491/136668758-521263aa-a9b2-4ad2-8d22-060b6bf86a1c.png)

My suspicion is that either:
* some of the sub-datasets are harder for the model than others
* some of the sub-datasets are not formatted properly

I'd like to identify which shards correspond to those jumps.

**Describe the solution you'd like**

It would be nice to have a key associated to each data sample or data batch containing details on where the data comes from (shard idx + item idx within the shard).
This should be supported both in local and streaming mode.

**Describe alternatives you've considered**
A fix would be for me to add myself details (shard id, sample id) as part of each data sample.
The inconvenient is that it requires users to process/reupload every dataset when they need this feature.


Loading from cache a dataset for LM built from a text classification dataset sometimes errors

bug

Something isn't working

This has been fixed in 1.15, let me know if you still have this issue

## Describe the bug

Yes, I know, that description sucks. So the problem is arising in the course when we build a masked language modeling dataset using the IMDB dataset. To reproduce (or try since it's a bit fickle).

Create a dataset for masled-language modeling from the IMDB dataset.

```python
from datasets import load_dataset
from transformers import Autotokenizer

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-cased)
imdb_dataset = load_dataset("imdb", split="train")

def tokenize_function(examples):
    return tokenizer(examples["text"])

tokenized_dataset = imdb_dataset.map(
    tokenize_function, batched=True, remove_columns=["text", "label"]
)

chunk_size = 128

def group_texts(examples):
    # Concatenate all texts.
    concatenated_examples = {k: sum(examples[k], []) for k in examples.keys()}
    # Compute length of concatenated texts
    total_length = len(concatenated_examples[list(examples.keys())[0]])
    # We drop the last chunk if it's smaller than chunk_size
    total_length = (total_length // chunk_size) * chunk_size
    # Split by chunks of max_len.
    result = {
        k: [t[i : i + chunk_size] for i in range(0, total_length, chunk_size)]
        for k, t in concatenated_examples.items()
    }
    # Create a new labels column
    result["labels"] = result["input_ids"].copy()
    return result

lm_dataset = tokenized_dataset.map(group_texts, batched=True)
```

Until now, all is well. The problem comes when you re-execute that code, more specifically:

```python
tokenized_dataset = imdb_dataset.map(
    tokenize_function, batched=True, remove_columns=["text", "label"]
)
lm_dataset = tokenized_dataset.map(group_texts, batched=True)
```

Try several times if the bug doesn't appear instantly, or just each line at a time, ideally in a notebook/Colab and you should get at some point:

```python
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-40-357a56ee3d53> in <module>
----> 1 lm_dataset = tokenized_dataset.map(group_texts, batched=True)

~/git/datasets/src/datasets/arrow_dataset.py in map(self, function, with_indices, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, num_proc, suffix_template, new_fingerprint, desc)
   1947                 new_fingerprint=new_fingerprint,
   1948                 disable_tqdm=disable_tqdm,
-> 1949                 desc=desc,
   1950             )
   1951         else:

~/git/datasets/src/datasets/arrow_dataset.py in wrapper(*args, **kwargs)
    424         }
    425         # apply actual function
--> 426         out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
    427         datasets: List["Dataset"] = list(out.values()) if isinstance(out, dict) else [out]
    428         # re-apply format to the output

~/git/datasets/src/datasets/fingerprint.py in wrapper(*args, **kwargs)
    404             # Call actual function
    405
--> 406             out = func(self, *args, **kwargs)
    407
    408             # Update fingerprint of in-place transforms + update in-place history of transforms

~/git/datasets/src/datasets/arrow_dataset.py in _map_single(self, function, with_indices, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, new_fingerprint, rank, offset, disable_tqdm, desc, cache_only)
   2138             if os.path.exists(cache_file_name) and load_from_cache_file:
   2139                 logger.warning("Loading cached processed dataset at %s", cache_file_name)
-> 2140                 info = self.info.copy()
   2141                 info.features = features
   2142                 return Dataset.from_file(cache_file_name, info=info, split=self.split)

~/git/datasets/src/datasets/info.py in copy(self)
    278
    279     def copy(self) -> "DatasetInfo":
--> 280         return self.__class__(**{k: copy.deepcopy(v) for k, v in self.__dict__.items()})
    281
    282

~/git/datasets/src/datasets/info.py in __init__(self, description, citation, homepage, license, features, post_processed, supervised_keys, task_templates, builder_name, config_name, version, splits, download_checksums, download_size, post_processing_size, dataset_size, size_in_bytes)

~/git/datasets/src/datasets/info.py in __post_init__(self)
    177                 for idx, template in enumerate(self.task_templates):
    178                     if isinstance(template, TextClassification):
--> 179                         labels = self.features[template.label_column].names
    180                         self.task_templates[idx] = TextClassification(
    181                             text_column=template.text_column, label_column=template.label_column, labels=labels

KeyError: 'label'
```

It seems that when loading the cache, the dataset tries to access some kind of text classification template (which I imagine comes from the original dataset) and to look at a key that has since been removed.


Inconsistent caching behaviour when using `Dataset.map()` with a `new_fingerprint` and `num_proc>1`

Something isn't working

Following the discussion in #3045 if would be nice to have a way to let users have a nice experience with caching even if the function is not hashable.

Currently a workaround is to make the function picklable. This can be done by implementing a callable class instead, that can be pickled using by implementing a custom `__getstate__` method for example.

However it sounds pretty complicated for a simple thing. Maybe one idea would be to have something similar to streamlit: they allow users to register the hashing of their own objects.

See the documentation about their `hash_funcs` here: https://docs.streamlit.io/library/advanced-features/caching#the-hash_funcs-parameter

Here is the example they give:

```python
class FileReference:
    def __init__(self, filename):
        self.filename = filename

def hash_file_reference(file_reference):
    filename = file_reference.filename
    return (filename, os.path.getmtime(filename))

@st.cache(hash_funcs={FileReference: hash_file_reference})
def func(file_reference):
    ...
```

My solution was to generate a custom hash, and use the hash as a `new_fingerprint` argument to the `map()` method to enable caching. This works, but is quite hacky.

@lhoestq, this approach is very neat, this would make the whole caching mechanic more explicit. I don't have so much time to look into this right now, but I might give it a try in the future.

## Describe the bug
Caching does not work when using `Dataset.map()` with:
1. a function that cannot be deterministically fingerprinted
2. `num_proc>1`
3.  using a custom fingerprint set with the argument `new_fingerprint`.

This means that the dataset will be mapped with the function for each and every call, which does not happen if `num_proc==1`. In that case (`num_proc==1`) subsequent calls will load the transformed dataset from the cache, which is the  expected behaviour. The example can easily be translated into a unit test.

I have a fix and will submit a pull request asap.

## Steps to reproduce the bug
```python
import hashlib
import json
import os
from typing import Dict, Any

import numpy as np
from datasets import load_dataset, Dataset

Batch = Dict[str, Any]
filename = 'example.json'

class Transformation():
    """A transformation with a random state that cannot be fingerprinted"""

    def __init__(self):
        self.state = np.random.random()

    def __call__(self, batch: Batch) -> Batch:
        batch['x'] = [np.random.random() for _ in batch['x']]
        return batch

def generate_dataset():
    """generate a simple dataset"""
    rgn = np.random.RandomState(24)
    data = {
        'data': [{'x': float(y), 'y': -float(y)} for y in
                 rgn.random(size=(1000,))]}
    if not os.path.exists(filename):
        with open(filename, 'w') as f:
            f.write(json.dumps(data))

    return filename

def process_dataset_with_cache(num_proc=1, remove_cache=False,
                               cache_expected_to_exist=False):

    # load the generated dataset
    dset: Dataset = next(
        iter(load_dataset('json', data_files=filename, field='data').values()))
    new_fingerprint = hashlib.md5("static-id".encode("utf8")).hexdigest()

    # get the expected cached path
    cache_path = dset._get_cache_file_path(new_fingerprint)
    if remove_cache and os.path.exists(cache_path):
        os.remove(cache_path)

     # check that the cache exists, and print a statement
    # if was actually expected to exist
    cache_exist = os.path.exists(cache_path)
    print(f"> cache file exists={cache_exist}")
    if cache_expected_to_exist and not cache_exist:
        print("=== Cache does not exist! ====")

    # apply the transformation with the new fingerprint
    dset = dset.map(
        Transformation(),
        batched=True,
        num_proc=num_proc,
        new_fingerprint=new_fingerprint,
        desc="mapping dataset with transformation")

generate_dataset()

for num_proc in [1, 2]:
    print(f"# num_proc={num_proc}, first pass")
    # first pass to generate the cache (always create a new cache here)
    process_dataset_with_cache(remove_cache=True,
                               num_proc=num_proc,
                               cache_expected_to_exist=False)
    print(f"# num_proc={num_proc}, second pass")
    # second pass, expects the cache to exist
    process_dataset_with_cache(remove_cache=False,
                               num_proc=num_proc,
                               cache_expected_to_exist=True)

os.remove(filename)

```

## Expected results
In the above python example, with `num_proc=2`, the **cache file should exist in the second call** of `process_dataset_with_cache`  ("=== Cache does not exist! ====" should not be printed).
When the cache is successfully created, `map()` is called only one time.

## Actual results
In the above python example, with `num_proc=2`, the **cache does not exist in the second call** of `process_dataset_with_cache` (this results in printing "=== Cache does not exist! ====").
Because the cache doesn't exist, the `map()` method is executed a second time and the dataset is not loaded from the cache.

## Environment info
- `datasets` version: 1.12.1
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.8.8
- PyArrow version: 5.0.0


Add PASS dataset

dataset request

Requesting to add a new dataset

vision

Vision datasets

## Adding a Dataset
- **Name:** PASS
- **Description:** An ImageNet replacement for self-supervised pretraining without humans
- **Data:** https://www.robots.ox.ac.uk/~vgg/research/pass/ https://github.com/yukimasano/PASS

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).


[save_to_disk] Using `select()` followed by `save_to_disk` saves complete dataset making it hard to create dummy dataset

bug

Something isn't working

Hi,

the `save_to_disk` docstring explains that `flatten_indices` has to be called on a dataset before saving it to save only the shard/slice of the dataset.

That works! Thansk!

Might be worth doing that automatically actually in case the `save_to_disk` is called on a dataset that has an indices mapping :-)

I agree with @patrickvonplaten: this issue is reported recurrently, so better if we implement the `.flatten_indices()` automatically?

That would be great indeed - I don't really see a use case where one would not like to call `.flatten_indices()` before calling `save_to_disk`

## Describe the bug

When only keeping a dummy size of a dataset (say the first 100 samples), and then saving it  to disk to upload it in the following to the hub for easy demo/use - not just the small dataset is saved but the whole dataset with an indices file. The problem with this is that the dataset is still very big.

## Steps to reproduce the bug

E.g. run the following:

```python
from datasets import load_dataset, save_to_disk

nlp = load_dataset("glue", "mnli", split="train")
nlp.save_to_disk("full")

nlp = nlp.select(range(100))
nlp.save_to_disk("dummy")
```

Now one can see that both `"dummy"` and `"full"` have the same size. This shouldn't be the case IMO.

## Expected results

IMO `"dummy"` should be much smaller so that one can easily play around with the dataset on the hub.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->

- `datasets` version: 1.12.2.dev0
- Platform: Linux-5.11.0-34-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyArrow version: 5.0.0


Protect master branch to force contributions via Pull Requests

enhancement

New feature or request

It would be nice to protect the master from direct commits, but still having a way to merge our own PRs when no review is required (for example when updating a dataset_infos.json file, or minor bug fixes - things that happen quite often actually).
Do you know if there's a way ?

This is done. Now the master branch is protected:
- [x] Require a pull request before merging: all commits must be made to a non-protected branch and submitted via a pull request
  - Required number of approvals before merging: 1
- [x] Require linear history: prevent merge commits from being pushed
- [x] These requirements are not enforced for administrators
- [x] Additionally, the master branch is also protected against deletion and force pushes

CC: @lhoestq @julien-c @thomwolf

In order to have a clearer Git history in the master branch, I propose to protect it so that all contributions must be done through a Pull Request and no direct commits to master are allowed.
- The Pull Request allows to give context, discuss any potential issues and improve the quality of the contribution
- The Pull Request will eventually be squashed and merged into master with a single commit that links to the Pull Request page (with all the context/discussions)

Note that we already implemented a protection in the master branch to avoid *merge* commits and ensure a linear history. This proposal goes one step further by avoiding all kind of direct commits and forcing contributions **only** through Pull Requests.

Please note that we can temporarily deactivate this protection if we need to make a direct commit, e.g. at each new version release.

The only way GitHub allows this kind or protection is by requiring a minimal number (at least one) of approvals of the Pull Request. The inconvenient is that the PR creator cannot approve their own PR: another person must approve it before it can be merged into master. To circumvent this, we could eventually disable this protection in the master branch when an urgent commit is needed (e.g. for a hotfix) and there is no other person available at that time to approve the PR.


`load_dataset` does not work with uploaded arrow file

New feature or request

Hi ! This is not a bug, this is simply not implemented.
`save_to_disk` is for on-disk serialization and was not made compatible for the Hub.
That being said, I agree we actually should make it work with the Hub x)

cc @LysandreJik maybe we can solve this at the same time as adding `push_to_hub`

## Describe the bug

I've preprocessed and uploaded a dataset here: https://huggingface.co/datasets/ami-wav2vec2/ami_headset_single_preprocessed . The dataset is in `.arrow` format.

The dataset can correctly be loaded when doing:

```bash
git lfs install
git clone https://huggingface.co/datasets/ami-wav2vec2/ami_headset_single_preprocessed
```

followed by

```python
from datasets import load_from_disk

ds = load_from_disk("./ami_headset_single_preprocessed")
```

However when I try to directly download the dataset as follows:

```python
from datasets import load_dataset

ds = load_dataset("ami-wav2vec2/ami_headset_single_preprocessed")
```

the following error occurs:

```bash
/usr/local/lib/python3.7/dist-packages/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, script_version, use_auth_token, task, streaming, **config_kwargs)
   1115         ignore_verifications=ignore_verifications,
   1116         try_from_hf_gcs=try_from_hf_gcs,
-> 1117         use_auth_token=use_auth_token,
   1118     )
   1119

/usr/local/lib/python3.7/dist-packages/datasets/builder.py in download_and_prepare(self, download_config, download_mode, ignore_verifications, try_from_hf_gcs, dl_manager, base_path, use_auth_token, **download_and_prepare_kwargs)
    635                     if not downloaded_from_gcs:
    636                         self._download_and_prepare(
--> 637                             dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
    638                         )
    639                     # Sync info

/usr/local/lib/python3.7/dist-packages/datasets/builder.py in _download_and_prepare(self, dl_manager, verify_infos, **prepare_split_kwargs)
    724             try:
    725                 # Prepare split will record examples associated to the split
--> 726                 self._prepare_split(split_generator, **prepare_split_kwargs)
    727             except OSError as e:
    728                 raise OSError(

/usr/local/lib/python3.7/dist-packages/datasets/builder.py in _prepare_split(self, split_generator)
   1186                 generator, unit=" tables", leave=False, disable=bool(logging.get_verbosity() == logging.NOTSET)
   1187             ):
-> 1188                 writer.write_table(table)
   1189             num_examples, num_bytes = writer.finalize()
   1190

/usr/local/lib/python3.7/dist-packages/datasets/arrow_writer.py in write_table(self, pa_table, writer_batch_size)
    424         # reorder the arrays if necessary + cast to self._schema
    425         # we can't simply use .cast here because we may need to change the order of the columns
--> 426         pa_table = pa.Table.from_arrays([pa_table[name] for name in self._schema.names], schema=self._schema)
    427         batches: List[pa.RecordBatch] = pa_table.to_batches(max_chunksize=writer_batch_size)
    428         self._num_bytes += sum(batch.nbytes for batch in batches)

/usr/local/lib/python3.7/dist-packages/pyarrow/table.pxi in pyarrow.lib.Table.from_arrays()

/usr/local/lib/python3.7/dist-packages/pyarrow/table.pxi in pyarrow.lib._sanitize_arrays()

/usr/local/lib/python3.7/dist-packages/pyarrow/array.pxi in pyarrow.lib.asarray()

/usr/local/lib/python3.7/dist-packages/pyarrow/table.pxi in pyarrow.lib.ChunkedArray.cast()

/usr/local/lib/python3.7/dist-packages/pyarrow/compute.py in cast(arr, target_type, safe)
    279     else:
    280         options = CastOptions.unsafe(target_type)
--> 281     return call_function("cast", [arr], options)
    282
    283

/usr/local/lib/python3.7/dist-packages/pyarrow/_compute.pyx in pyarrow._compute.call_function()

/usr/local/lib/python3.7/dist-packages/pyarrow/_compute.pyx in pyarrow._compute.Function.call()

/usr/local/lib/python3.7/dist-packages/pyarrow/error.pxi in pyarrow.lib.pyarrow_internal_check_status()

/usr/local/lib/python3.7/dist-packages/pyarrow/error.pxi in pyarrow.lib.check_status()

ArrowNotImplementedError: Unsupported cast from struct<train: struct<name: string, num_bytes: int64, num_examples: int64, dataset_name: string>, validation: struct<name: string, num_bytes: int64, num_examples: int64, dataset_name: string>, test: struct<name: string, num_bytes: int64, num_examples: int64, dataset_name: string>> to list using function cast_list
```

## Expected results

The dataset should be correctly loaded with `load_dataset` IMO.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.2.dev0
- Platform: Linux-5.11.0-34-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyArrow version: 5.0.0


Errors loading dataset using fs = a gcsfs.GCSFileSystem

bug

Something isn't working

## Describe the bug

Cannot load dataset using a `gcsfs.GCSFileSystem`.  I'm not sure if this should be a bug in `gcsfs` or here...

Basically what seems to be happening is that since datasets saves datasets as folders and folders aren't "real objects" in gcs, gcsfs raises a 404 error.  There are workarounds if you use gcsfs directly to download the file, but as is I can't get `load_from_disk` to work.

## Steps to reproduce the bug
```python
from datasets import load_dataset
# load some dataset
dataset = load_dataset("squad", split="train")

# save it to gcs
import gcsfs
fs = gcsfs.GCSFileSystem(project="my-gs-project")
dataset.save_to_disk("gs://my-bucket/squad", fs=fs)

# try to load it from gcs
from datasets import load_from_disk
dataset2 = load_from_disk("my-bucket/squad", fs=fs)
```

## Expected results
`dataset2` would be a copy of `dataset` but loaded from my bucket.

## Actual results
Long traceback but essentially it's a 404 error from gcsfs saying the object `my-bucket/squad` doesn't exist when this is called:
https://github.com/huggingface/datasets/blob/9c81b7d2e6d9feae69a084a3abda265a4ca07fb5/src/datasets/arrow_dataset.py#L977

This is because there is no actual object called `my-bucket/squad`, there are objects called `my-bucket/squad/dataset.arrow`, etc.

Note that *this* works fine, since it's explicitly saying "download all the objects with this prefix":
```python
            fs.download(src_dataset_path + "/*", dataset_path.as_posix(), recursive=True)
```
For example, I can do a workaround this way:
```python
import tempfile

with tempfile.TemporaryDirectory() as temppath:
    fs.download("gs://my-bucket/squad/*", temppath)
    dataset2 = load_from_disk(temppath)
```

It's unclear to me if it's `gcsfs`'s responsibility to say "hey that's folder not a file, I should try to get objects inside of it not the object itself", or if that's `datasets`'s responsibility... I'm leaning towards the latter since you're never loading a dataset from one file using this function/method, only a dataset folder?

Another minor thing that should maybe should be rolled into this bug...

https://github.com/huggingface/datasets/blob/9c81b7d2e6d9feae69a084a3abda265a4ca07fb5/src/datasets/arrow_dataset.py#L968

These fail if you pass in a `gs://` path, e.g.

```python
dataset2 = load_from_disk("gs://my-bucket/squad", fs=fs)
```
Because at this point, `dataset_info_path` is `gs:/my-bucket/squad/dataset_info.json`, gcsfs throws a:
```
Invalid bucket name: 'gs:'
```
error

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: macOS Big Sur 11.6
- Python version: 3.7.12
- PyArrow version: 5.0.0


Error when loading private dataset with "data_files" arg

Something isn't working

We'll do a release tomorrow or on wednesday to make the fix available :)

Thanks for reproting !

## Describe the bug
A clear and concise description of what the bug is.

Private datasets with no loading script can't be loaded using `data_files` parameter.

## Steps to reproduce the bug
```python
from datasets import load_dataset
data_files = {"train": "**/train/*/*.jsonl", "valid": "**/valid/*/*.jsonl"}
dataset = load_dataset('dalle-mini/encoded', data_files=data_files, use_auth_token=True, streaming=True)
```

Same error happens in non-streaming mode.

## Expected results
Files should be loaded (whether in streaming or not).

## Actual results
Error:

```
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/datasets/load.py in prepare_module(path, script_version, download_config, download_mode, dataset, force_local_path, dynamic_modules_path, return_resolved_file_path, return_associated_base_path, data_files, **download_kwargs)
    539                 try:
--> 540                     local_path = cached_path(file_path, download_config=download_config)
    541                 except FileNotFoundError:

8 frames
FileNotFoundError: Couldn't find file at https://huggingface.co/datasets/dalle-mini/encoded/resolve/main/encoded.py

During handling of the above exception, another exception occurred:

HTTPError                                 Traceback (most recent call last)
HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/api/datasets/dalle-mini/encoded?full=true

During handling of the above exception, another exception occurred:

FileNotFoundError                         Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/datasets/load.py in prepare_module(path, script_version, download_config, download_mode, dataset, force_local_path, dynamic_modules_path, return_resolved_file_path, return_associated_base_path, data_files, **download_kwargs)
    547                     except Exception:
    548                         raise FileNotFoundError(
--> 549                             f"Couldn't find a directory or a {resource_type} named '{path}'. "
    550                             f"It doesn't exist locally at {expected_dir_for_combined_path_abs} or remotely on {hf_api.endpoint}/datasets"
    551                         )

FileNotFoundError: Couldn't find a directory or a dataset named 'dalle-mini/encoded'. It doesn't exist locally at /content/dalle-mini/encoded or remotely on https://huggingface.co/datasets
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.12
- PyArrow version: 3.0.0

@lhoestq


Resolve data_files by split name

Awesome @lhoestq I like the proposal and it works great on my JSON community dataset. Here is the [log](https://gist.github.com/vblagoje/714babc325bcbdd5de579fd8e1648892).

From my discussion with @borisdayma it would be more general the files match if their paths contains the split name - not only if the filename contains the split name. For example for a dataset like this:
```
train/
└── data.csv
test/
└── data.csv
```

But IMO the default should be
```
data/
├── train.csv
└── test.csv
```
because it allows people to have other directories if they have different subsets of their data (different configurations, not splits)

I just created a PR for this at https://github.com/huggingface/datasets/pull/3221, let me know what you think :)

This issue is about discussing the default behavior when someone loads a dataset that consists in data files. For example:
```python
load_dataset("lhoestq/demo1")
```
should return two splits "train" and "test" since the dataset repostiory is like
```
data/
├── train.csv
└── test.csv
```
Currently it returns only one split "train" which contains the data of both files

I started playing with this idea on this branch btw: `resolve-data_files-by-split-name`
Basically the idea is that if you named you data files after split names then the default pattern is
```python
{
    "train": ["*train*"],
    "test": ["*test*"],
    "validation": ["*dev*", "valid"],
}
```
otherwise it's
```python
{
    "train": ["*"]
}
```

Let me know what you think !
cc @albertvillanova @LysandreJik @vblagoje


Windows test suite fails

Something isn't working

## Describe the bug

There is an error during installation of tests dependencies for Windows: https://app.circleci.com/pipelines/github/huggingface/datasets/7981/workflows/9b6a0114-2b8e-4069-94e5-e844dbbdba4e/jobs/49206

```
ERROR: Cannot uninstall 'ruamel-yaml'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
```


Support multiple zipped CSV data files

enhancement

New feature or request

@lhoestq I would like to draw your attention to the proposed API by @lewtun, using `data_dir` to pass the ZIP URL.

I'm not totally convinced with this... What do you think?

Maybe we could discuss other approaches...

One brainstorming idea: what about using URL chaining with the hop operator in `data_files`?

`data_dir` is currently exclusively used for manually downloaded data.

Maybe we can have an API that only uses data_files as you are suggesting, using URL chaining ?
```python
from datasets import load_dataset

url = "https://domain.org/filename.zip"
data_files = {"train": "zip://train_filename.csv::" + url, "test": "zip://test_filename.csv::" + url}
dataset = load_dataset("csv", data_files=data_files)
```

URL chaining is used by `fsspec` to get access to files in nested filesystems of any kind. Since `fsspec` is being used by `pandas`, `dask` and also extensively by `datasets` I think it would be nice to use it here too

URL chaining sounds super nice to me! And it's also a nice way to leverage the same concepts we currently have in the docs around `fsspec` :)

As requested by @lewtun, support loading multiple zipped CSV data files.

```python
from datasets import load_dataset

url = "https://domain.org/filename.zip"
data_files = {"train": "train_filename.csv", "test": "test_filename.csv"}
dataset = load_dataset("csv", data_dir=url, data_files=data_files)
```


Improve `get_dataset_infos`?

question

Further information is requested

dataset-viewer

Related to the dataset viewer on huggingface.co

To keeps things simple maybe we should use `load_dataset_builder` in `get_dataset_infos`.
`load_dataset_builder` instantiates a builder and runs the _infos() method in order to give you the most up-to-date infos, even if the dataset_infos.json is outdated or missing.

Using the dedicated function `get_dataset_infos` on a dataset that has no dataset-info.json file returns an empty info:

```
>>> from datasets import get_dataset_infos
>>> get_dataset_infos('wit')
{}
```

While it's totally possible to get it (regenerate it) with:

```
>>> from datasets import load_dataset_builder
>>> builder = load_dataset_builder('wit')
>>> builder.info
DatasetInfo(description='Wikipedia-based Image Text (WIT) Dataset is a large multimodal multilingual dataset. WIT is composed of a curated set\n of 37.6 million entity rich image-text examples with 11.5 million unique images across 108 Wikipedia languages. Its\n size enables WIT to be used as a pretraining dataset for multimodal machine learning models.\n', citation='@article{srinivasan2021wit,\n  title={WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning},\n  author={Srinivasan, Krishna and Raman, Karthik and Chen, Jiecao and Bendersky, Michael and Najork, Marc},\n  journal={arXiv preprint arXiv:2103.01913},\n  year={2021}\n}\n', homepage='https://github.com/google-research-datasets/wit', license='', features={'b64_bytes': Value(dtype='string', id=None), 'embedding': Sequence(feature=Value(dtype='float64', id=None), length=-1, id=None), 'image_url': Value(dtype='string', id=None), 'metadata_url': Value(dtype='string', id=None), 'original_height': Value(dtype='int32', id=None), 'original_width': Value(dtype='int32', id=None), 'mime_type': Value(dtype='string', id=None), 'caption_attribution_description': Value(dtype='string', id=None), 'wit_features': Sequence(feature={'language': Value(dtype='string', id=None), 'page_url': Value(dtype='string', id=None), 'attribution_passes_lang_id': Value(dtype='string', id=None), 'caption_alt_text_description': Value(dtype='string', id=None), 'caption_reference_description': Value(dtype='string', id=None), 'caption_title_and_reference_description': Value(dtype='string', id=None), 'context_page_description': Value(dtype='string', id=None), 'context_section_description': Value(dtype='string', id=None), 'hierarchical_section_title': Value(dtype='string', id=None), 'is_main_image': Value(dtype='string', id=None), 'page_changed_recently': Value(dtype='string', id=None), 'page_title': Value(dtype='string', id=None), 'section_title': Value(dtype='string', id=None)}, length=-1, id=None)}, post_processed=None, supervised_keys=None, task_templates=None, builder_name='wit', config_name='default', version=0.0.0, splits=None, download_checksums=None, download_size=None, post_processing_size=None, dataset_size=None, size_in_bytes=None)
```

Should we test if info is empty, and in that case regenerate it? Or always generate it?


load_dataset_builder should error if "name" does not exist?

question

Further information is requested

Yes I think it should raise an error. Currently it looks like it instantiates a custom configuration with the name given by the user:
https://github.com/huggingface/datasets/blob/ba27ce33bf568374cf23a07669fdd875b5718bc2/src/datasets/builder.py#L391-L397

```
import datasets as ds
builder = ds.load_dataset_builder('sent_comp', name="doesnotexist")
builder.info.config_name
```

returns

```
'doesnotexist'
```

Shouldn't it raise an error instead?

For this dataset, the only valid values for `name` should be: `"default"` or `None` (ie. argument not passed)


Chain filtering is leaking

bug

Something isn't working

### Update:
I wrote a bit cleaner code snippet (without transforming to json) that can expose leaking.
```python
import datasets
import json

items = ['ab', 'c', 'df']

ds = datasets.Dataset.from_dict({'col': items})
print(list(ds))
# > Prints: [{'col': 'ab'}, {'col': 'c'}, {'col': 'df'}]

filtered = ds

# get all items that are starting with a character with ascii code bigger than 'a'
filtered = filtered.filter(lambda x: x['col'][0] > 'a', load_from_cache_file=False)
print(list(filtered))
# > Prints: [{'col': 'c'}, {'col': 'df'}] as expected

# get all items that are shorter than 2
filtered = filtered.filter(lambda x: len(x['col']) < 2, load_from_cache_file=False)
print(list(filtered))
# > Prints: [{'col': 'ab'}] -> this is a leaked item from the first filter
# > Should be: [{'col': 'c'}]
```

Thanks for reporting. I'm looking into it

I just pushed a fix ! We'll do a new release soon.
In the meantime feel free to install `datasets` from source to play with it

Thanks, I'm already using it from your branch!

## Describe the bug
As there's no support for lists within dataset fields, I convert my lists to json-string format. However, the bug described is occurring even when the data format is 'string'.
These samples show that filtering behavior diverges from what's expected when chaining filterings.
On sample 2 the second filtering leads to "leaking" of data that should've been filtered on the first filtering into the results.

## Steps to reproduce the bug
Sample 1:
```python
import datasets
import json

items = [[1, 2], [3], [4]]
jsoned_items = map(json.dumps, [[1, 2], [3], [4]])
ds = datasets.Dataset.from_dict({'a': jsoned_items})
print(list(ds))
# > Prints: [{'a': '[1, 2]'}, {'a': '[3]'}, {'a': '[4]'}] as expected

filtered = ds

# get all lists that are shorter than 2
filtered = filtered.filter(lambda x: len(json.loads(x['a'])) < 2, load_from_cache_file=False)
print(list(filtered))
# > Prints: [{'a': '[3]'}, {'a': '[4]'}] as expected

# get all lists, which have a value bigger than 3 on its zero index
filtered = filtered.filter(lambda x: json.loads(x['a'])[0] > 3, load_from_cache_file=False)
print(list(filtered))
# > Should be: [{'a': [4]}]
# > Prints: [{'a': [3]}]
```
Sample 2:
```python
import datasets
import json

items = [[1, 2], [3], [4]]
jsoned_items = map(json.dumps, [[1, 2], [3], [4]])
ds = datasets.Dataset.from_dict({'a': jsoned_items})
print(list(ds))
# > Prints: [{'a': '[1, 2]'}, {'a': '[3]'}, {'a': '[4]'}]

filtered = ds

# get all lists, which have a value bigger than 3 on its zero index
filtered = filtered.filter(lambda x: json.loads(x['a'])[0] > 3, load_from_cache_file=False)
print(list(filtered))
# > Prints: [{'a': '[4]'}] as expected

# get all lists that are shorter than 2
filtered = filtered.filter(lambda x: len(json.loads(x['a'])) < 2, load_from_cache_file=False)
print(list(filtered))
# > Prints: [{'a': '[1, 2]'}]
# > Should be: [{'a': '[4]'}] (remain intact)
```

## Expected results
Expected and actual results are attached to the code snippets.

## Actual results
Expected and actual results are attached to the code snippets.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: Windows-10-10.0.19042-SP0
- Python version: 3.9.7
- PyArrow version: 5.0.0


DatasetDict.filter and Dataset.filter crashes with any "fn_kwargs" argument

Something isn't working

Hi @DrMatters, thanks for reporting.

This issue was fixed 14 days ago: #2950.

Currently, the fix is only in the master branch and will be made available in our next library release.

In the meantime, you can incorporate the fix by installing datasets from the master branch:
```shell
pip install -U git+ssh://git@github.com/huggingface/datasets.git@master#egg=datasest
```
or
```shell
pip install -U git+https://github.com/huggingface/datasets.git@master#egg=datasets
```

## Describe the bug
The ".filter" method of DatasetDict or Dataset objects fails when passing any "fn_kwargs" argument

## Steps to reproduce the bug
```python
import datasets

example_dataset = datasets.Dataset.from_dict({"a": {1, 2, 3, 4}})

def filter_value(example, value):
    return example['a'] == value

filtered = example_dataset.filter(filter_value, fn_kwargs={'value': 3})
```

## Expected results
`filtered` is a dataset containing {"a": {3}}

## Actual results

> Traceback (most recent call last):
>   File "C:\Users\qsemi\Documents\git\nlp_experiments\gpt_celebrity\src\test_faulty_filter.py", line 8, in <module>
>     filtered = example_dataset.filter(filter_value, fn_kwargs={'value': 3})
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\arrow_dataset.py", line 185, in wrapper
>     out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\fingerprint.py", line 398, in wrapper
>     out = func(self, *args, **kwargs)
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\arrow_dataset.py", line 2169, in filter
>     indices = self.map(
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\arrow_dataset.py", line 1686, in map
>     return self._map_single(
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\arrow_dataset.py", line 185, in wrapper
>     out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\fingerprint.py", line 398, in wrapper
>     out = func(self, *args, **kwargs)
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\arrow_dataset.py", line 2048, in _map_single
>     batch = apply_function_on_filtered_inputs(
>   File "C:\Users\qsemi\miniconda3\envs\main\lib\site-packages\datasets\arrow_dataset.py", line 1939, in apply_function_on_filtered_inputs
>     function(*fn_args, effective_indices, **fn_kwargs) if with_indices else function(*fn_args, **fn_kwargs)
> TypeError: get_indices_from_mask_function() got an unexpected keyword argument 'value'

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: Windows-10-10.0.19042-SP0
- Python version: 3.9.7
- PyArrow version: 5.0.0


cannot shuffle dataset loaded from disk

Something isn't working

## Describe the bug
dataset loaded from disk cannot be shuffled.

## Steps to reproduce the bug
```
my_dataset = load_from_disk('s3://my_file/validate', fs=s3)
sample = my_dataset.select(range(100)).shuffle(seed=1234)
```

## Actual results
```
sample = my_dataset .select(range(100)).shuffle(seed=1234)
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 185, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/site-packages/datasets/fingerprint.py", line 398, in wrapper
    out = func(self, *args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 2494, in shuffle
    new_fingerprint=new_fingerprint,
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 185, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/site-packages/datasets/fingerprint.py", line 398, in wrapper
    out = func(self, *args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 2303, in select
    tmp_file = tempfile.NamedTemporaryFile("wb", dir=os.path.dirname(indices_cache_file_name), delete=False)
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/tempfile.py", line 547, in NamedTemporaryFile
    (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
  File "/home/ubuntu/anaconda3/envs/pytorch_p37/lib/python3.7/tempfile.py", line 258, in _mkstemp_inner
    fd = _os.open(file, flags, 0o600)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpnnu5uhnx/my_file/validate/tmpy76d70g4'
```
## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1

- Python version: 3.7
- PyArrow version: 5.0.0


Dataset has incorrect labels

Hi @marshmellow77, thanks for reporting.

That issue is fixed since `datasets` version 1.9.0 (see 16bc665f2753677c765011ef79c84e55486d4347).

Please, update `datasets` with: `pip install -U datasets`

Thanks. Please note that the dataset explorer (https://huggingface.co/datasets/viewer/?dataset=turkish_product_reviews) still shows the incorrect state. The sentiment for the first few customer reviews is actually negative and should be labelled with "0", see screenshot:

![Capture1](https://user-images.githubusercontent.com/63367770/135637150-93d9b09b-f1dd-4701-97a5-5cb2672ec0c7.PNG)

Thanks @marshmellow77, good catch! I'm transferring this issue to https://github.com/huggingface/datasets-viewer.

The dataset https://huggingface.co/datasets/turkish_product_reviews has incorrect labels - all reviews are labelled with "1" (positive sentiment). None of the reviews is labelled with "0". See screenshot attached:

![Capture](https://user-images.githubusercontent.com/63367770/135617428-14ce0b27-5208-4e66-a3ee-71542e3257b4.PNG)


Can't download `trivia_qa/unfiltered`

Something isn't working

wooo that was fast! thank you @lhoestq !
it is able to process now, though it's ignoring all files and ending up with 0 examples now haha :/

For subset "unfiltered":
```python
>>> load_dataset("trivia_qa", "unfiltered")
Downloading and preparing dataset trivia_qa/unfiltered (download: 3.07 GiB, generated: 27.23 GiB, post-processed: Unknown size, total: 30.30 GiB) to /gpfsscratch/rech/six/commun/datasets/trivia_qa/unfiltered/1.1.0/910043a609bb2bdf62b4874f68e0c24fb648cf81e40a358f4bd54c919d72c9ab...
100%|███████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 1354.53it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 40.60it/s]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/load.py", line 1198, in load_dataset
    use_auth_token=use_auth_token,
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/builder.py", line 647, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/builder.py", line 748, in _download_and_prepare
    verify_splits(self.info.splits, split_dict)
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/utils/info_utils.py", line 74, in verify_splits
    raise NonMatchingSplitsSizesError(str(bad_splits))
datasets.utils.info_utils.NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='test', num_bytes=2906575347, num_examples=10832, dataset_name='trivia_qa'), 'recorded': SplitInfo(name='test', num_bytes=0, num_examples=0, dataset_name='trivia_qa')}, {'expected': SplitInfo(name='validation', num_bytes=3038966234, num_examples=11313, dataset_name='trivia_qa'), 'recorded': SplitInfo(name='validation', num_bytes=0, num_examples=0, dataset_name='trivia_qa')}]
```
For subset "rc":
```python
>>> load_dataset("trivia_qa", "rc")
Downloading and preparing dataset trivia_qa/rc (download: 2.48 GiB, generated: 14.92 GiB, post-processed: Unknown size, total: 17.40 GiB) to /gpfsscratch/rech/six/commun/datasets/trivia_qa/rc/1.1.0/910043a609bb2bdf62b4874f68e0c24fb648cf81e40a358f4bd54c919d72c9ab...
100%|███████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 3806.08it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 51.57it/s]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/load.py", line 1198, in load_dataset
    use_auth_token=use_auth_token,
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/builder.py", line 647, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/builder.py", line 748, in _download_and_prepare
    verify_splits(self.info.splits, split_dict)
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/utils/info_utils.py", line 74, in verify_splits
    raise NonMatchingSplitsSizesError(str(bad_splits))
datasets.utils.info_utils.NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='test', num_bytes=1577814583, num_examples=17210, dataset_name='trivia_qa'), 'recorded': SplitInfo(name='test', num_bytes=0, num_examples=0, dataset_name='trivia_qa')}, {'expected': SplitInfo(name='train', num_bytes=12750976012, num_examples=138384, dataset_name='trivia_qa'), 'recorded': SplitInfo(name='train', num_bytes=0, num_examples=0, dataset_name='trivia_qa')}, {'expected': SplitInfo(name='validation', num_bytes=1688535379, num_examples=18669, dataset_name='trivia_qa'), 'recorded': SplitInfo(name='validation', num_bytes=0, num_examples=0, dataset_name='trivia_qa')}]
```

Could you look into that when you get a chance?
I wonder if it's not something they changed on the file to download? i couldn't find any information

@VictorSanh have you tried passing `download_mode="force_redownload"`?
```python
ds = load_dataset("trivia_qa", "unfiltered", download_mode="force_redownload")
```

I aggressively rmed caches, especially rming the `datasets/downloads/extracted/c3d265fa20d99a147a76e4f5e...` solved the issue.
thank you both!

## Describe the bug

For some reason, I can't download `trivia_qa/unfilted`. A file seems to be missing... I am able to see it fine though the viewer tough...

## Steps to reproduce the bug
```python
>>> from datasets import load_dataset
>>> load_dataset("trivia_qa", "unfiltered")
Downloading and preparing dataset trivia_qa/unfiltered (download: 3.07 GiB, generated: 27.23 GiB, post-processed: Unknown size, total: 30.30 GiB) to /gpfsscratch/rech/six/commun/datasets/trivia_qa/unfiltered/1.1.0/9977a5d6f72acfd92f587de052403e8138b43bb0d1ce595016c3baf7e14deba6...
Traceback (most recent call last):
  File "/gpfswork/rech/six/commun/modules/datasets_modules/datasets/trivia_qa/9977a5d6f72acfd92f587de052403e8138b43bb0d1ce595016c3baf7e14deba6/trivia_qa.py", line 251, in _add_context
    with open(os.path.join(file_dir, fname), encoding="utf-8") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/gpfsscratch/rech/six/commun/datasets/downloads/extracted/9fcb7eddc6afd46fd074af3c5128931dfe4b548f933c925a23847faf4c1995ad/evidence/wikipedia/Peanuts.txt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/load.py", line 852, in load_dataset
    use_auth_token=use_auth_token,
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/builder.py", line 616, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/builder.py", line 693, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/builder.py", line 1107, in _prepare_split
    disable=bool(logging.get_verbosity() == logging.NOTSET),
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/tqdm/std.py", line 1133, in __iter__
    for obj in iterable:
  File "/gpfswork/rech/six/commun/modules/datasets_modules/datasets/trivia_qa/9977a5d6f72acfd92f587de052403e8138b43bb0d1ce595016c3baf7e14deba6/trivia_qa.py", line 303, in _generate_examples
    example = parse_example(article)
  File "/gpfswork/rech/six/commun/modules/datasets_modules/datasets/trivia_qa/9977a5d6f72acfd92f587de052403e8138b43bb0d1ce595016c3baf7e14deba6/trivia_qa.py", line 274, in parse_example
    _add_context(article.get("EntityPages", []), "WikiContext", wiki_dir),
  File "/gpfswork/rech/six/commun/modules/datasets_modules/datasets/trivia_qa/9977a5d6f72acfd92f587de052403e8138b43bb0d1ce595016c3baf7e14deba6/trivia_qa.py", line 253, in _add_context
    except (IOError, datasets.Value("errors").NotFoundError):
  File "<string>", line 5, in __init__
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/features.py", line 265, in __post_init__
    self.pa_type = string_to_arrow(self.dtype)
  File "/gpfswork/rech/six/commun/conda/victor/lib/python3.7/site-packages/datasets/features.py", line 134, in string_to_arrow
    f"Neither {datasets_dtype} nor {datasets_dtype + '_'} seems to be a pyarrow data type. "
ValueError: Neither errors nor errors_ seems to be a pyarrow data type. Please make sure to use a correct data type, see: https://arrow.apache.org/docs/python/api/datatypes.html#factory-functions
```

## Expected results
I am able to load another subset (`rc`), but unable to load.
I am not sure why the try/except doesn't catch it...
https://github.com/huggingface/datasets/blob/9675a5a1e7b99a86f9c250f6ea5fa5d1e6d5cc7d/datasets/trivia_qa/trivia_qa.py#L253

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.11.0
- Platform: Linux-4.18.0-147.51.2.el8_1.x86_64-x86_64-with-redhat-8.1-Ootpa
- Python version: 3.7.10
- PyArrow version: 3.0.0


add docmentation for the `Unix style pattern` matching feature that can be leverage for `data_files` into `load_dataset`

enhancement

New feature or request

Unless I'm mistaken, it seems that in the new documentation it is no longer mentioned that you can use Unix style pattern matching in the  `data_files` argument of the `load_dataset` method.

This feature was mentioned [here](https://huggingface.co/docs/datasets/loading_datasets.html#from-a-community-dataset-on-the-hugging-face-hub) in the previous documentation.

I'd love to hear your opinion @lhoestq , @albertvillanova and @stevhliu


bug

Something isn't working

Hi ! Could you check the length of the `self.dataset` object (i.e. the Dataset object passed to the data loader) ? It looks like the dataset is empty.
Not sure why the SWA optimizer would cause this though.

Any updates on this?
The same error occurred to me too when running `cardiffnlp/twitter-roberta-base-sentiment` on a custom dataset. This happened when I tried to do `model = torch.nn.DataParallel(model, device_ids=[0, 1, 2, 3])` without using sagemaker distribution.
Python: 3.6.13
datasets: 1.6.2

Hi @ruisi-su, do you have this issue while using SWA as well, or just data parallel ?

If you have a code example to reproduce this issue it would also be helpful

@lhoestq I had this issue without SWA. I followed [this](https://github.com/huggingface/notebooks/blob/master/sagemaker/03_distributed_training_data_parallelism/sagemaker-notebook.ipynb) notebook to utilize multiple gpus on the [roberta-base](https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment) model. This tutorial could only work if I am on `ml.p3.16xlarge`, which I don't have access to. So I tried using just `model = torch.nn.DataParallel(model, device_ids=[0, 1, 2, 3]` before calling `trainer.fit()`. But maybe this is not the right way to do distributed training. I can provide a code example if that will be more helpful.

It might be an issue with old versions of `datasets`, can you try updating `datasets` ?

## Describe the bug
A clear and concise description of what the bug is.

Hi. I am trying to implement stochastic weighted averaging optimizer with transformer library as described here https://pytorch.org/blog/pytorch-1.6-now-includes-stochastic-weight-averaging/ , for this I am using a run_clm.py codes which is working fine before adding SWA optimizer, the moment I modify the model with `swa_model = AveragedModel(model)` in this script, I am getting the below error,  since I am NOT touching the dataloader part, I am confused why this is occurring, I very much appreciate your opinion on this @lhoestq

## Steps to reproduce the bug
```
Traceback (most recent call last):
  File "run_clm.py", line 723, in <module>
    main()
  File "run_clm.py", line 669, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/transformers/trainer.py", line 1258, in train
    for step, inputs in enumerate(epoch_iterator):
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 435, in __next__
    data = self._next_data()
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 475, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 1530, in __getitem__
    format_kwargs=self._format_kwargs,
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 1517, in _getitem
    pa_subtable = query_table(self._data, key, indices=self._indices if self._indices is not None else None)
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/datasets/formatting/formatting.py", line 368, in query_table
    _check_valid_index_key(key, size)
  File "/user/dara/libs/anaconda3/envs/success/lib/python3.7/site-packages/datasets/formatting/formatting.py", line 311, in _check_valid_index_key
    raise IndexError(f"Invalid key: {key} is out of bounds for size {size}")
IndexError: Invalid key: 14 is out of bounds for size 0
```

## Expected results
not getting the index error

## Actual results
Please see the above

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: datasets                  1.12.1
- Platform: linux
- Python version: 3.7.11
- PyArrow version:  5.0.0


Something isn't working

Hi @NielsRogge, thanks for reporting!

In `datasets`, we were handling N-dimensional arrays only when passed as an instance of `np.array`, not when passed as a list of `np.array`s.

I'm fixing it.

## Describe the bug

For the ViT and LayoutLMv2 demo notebooks in my [Transformers-Tutorials repo](https://github.com/NielsRogge/Transformers-Tutorials), people reported an ArrowInvalid issue after applying the following function to a Dataset:

```
def preprocess_data(examples):
  images = [Image.open(path).convert("RGB") for path in examples['image_path']]
  words = examples['words']
  boxes = examples['bboxes']
  word_labels = examples['ner_tags']

  encoded_inputs = processor(images, words, boxes=boxes, word_labels=word_labels,
                             padding="max_length", truncation=True)

  return encoded_inputs
```

```
Full trace:

---------------------------------------------------------------------------
ArrowInvalid                              Traceback (most recent call last)
<ipython-input-8-0fc3efc6f0c2> in <module>()
     27
     28 train_dataset = datasets['train'].map(preprocess_data, batched=True, remove_columns=datasets['train'].column_names,
---> 29                                       features=features)
     30 test_dataset = datasets['test'].map(preprocess_data, batched=True, remove_columns=datasets['test'].column_names,
     31                                       features=features)

13 frames
/usr/local/lib/python3.7/dist-packages/datasets/arrow_dataset.py in map(self, function, with_indices, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, num_proc, suffix_template, new_fingerprint, desc)
   1701                 new_fingerprint=new_fingerprint,
   1702                 disable_tqdm=disable_tqdm,
-> 1703                 desc=desc,
   1704             )
   1705         else:

/usr/local/lib/python3.7/dist-packages/datasets/arrow_dataset.py in wrapper(*args, **kwargs)
    183         }
    184         # apply actual function
--> 185         out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
    186         datasets: List["Dataset"] = list(out.values()) if isinstance(out, dict) else [out]
    187         # re-apply format to the output

/usr/local/lib/python3.7/dist-packages/datasets/fingerprint.py in wrapper(*args, **kwargs)
    396             # Call actual function
    397
--> 398             out = func(self, *args, **kwargs)
    399
    400             # Update fingerprint of in-place transforms + update in-place history of transforms

/usr/local/lib/python3.7/dist-packages/datasets/arrow_dataset.py in _map_single(self, function, with_indices, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, new_fingerprint, rank, offset, disable_tqdm, desc, cache_only)
   2063                                 writer.write_table(batch)
   2064                             else:
-> 2065                                 writer.write_batch(batch)
   2066                 if update_data and writer is not None:
   2067                     writer.finalize()  # close_stream=bool(buf_writer is None))  # We only close if we are writing in a file

/usr/local/lib/python3.7/dist-packages/datasets/arrow_writer.py in write_batch(self, batch_examples, writer_batch_size)
    409             typed_sequence = OptimizedTypedSequence(batch_examples[col], type=col_type, try_type=col_try_type, col=col)
    410             typed_sequence_examples[col] = typed_sequence
--> 411         pa_table = pa.Table.from_pydict(typed_sequence_examples)
    412         self.write_table(pa_table, writer_batch_size)
    413

/usr/local/lib/python3.7/dist-packages/pyarrow/table.pxi in pyarrow.lib.Table.from_pydict()

/usr/local/lib/python3.7/dist-packages/pyarrow/array.pxi in pyarrow.lib.asarray()

/usr/local/lib/python3.7/dist-packages/pyarrow/array.pxi in pyarrow.lib.array()

/usr/local/lib/python3.7/dist-packages/pyarrow/array.pxi in pyarrow.lib._handle_arrow_array_protocol()

/usr/local/lib/python3.7/dist-packages/datasets/arrow_writer.py in __arrow_array__(self, type)
    106                     storage = numpy_to_pyarrow_listarray(self.data, type=type.value_type)
    107                 else:
--> 108                     storage = pa.array(self.data, type.storage_dtype)
    109                 out = pa.ExtensionArray.from_storage(type, storage)
    110             elif isinstance(self.data, np.ndarray):

/usr/local/lib/python3.7/dist-packages/pyarrow/array.pxi in pyarrow.lib.array()

/usr/local/lib/python3.7/dist-packages/pyarrow/array.pxi in pyarrow.lib._sequence_to_array()

/usr/local/lib/python3.7/dist-packages/pyarrow/error.pxi in pyarrow.lib.pyarrow_internal_check_status()

/usr/local/lib/python3.7/dist-packages/pyarrow/error.pxi in pyarrow.lib.check_status()

ArrowInvalid: Can only convert 1-dimensional array values
```
It can be fixed by adding the following line:

```diff
def preprocess_data(examples):
  images = [Image.open(path).convert("RGB") for path in examples['image_path']]
  words = examples['words']
  boxes = examples['bboxes']
  word_labels = examples['ner_tags']

  encoded_inputs = processor(images, words, boxes=boxes, word_labels=word_labels,
                             padding="max_length", truncation=True)
+ encoded_inputs["image"] = np.array(encoded_inputs["image"])

  return encoded_inputs
```

However, would be great if this can be fixed within Datasets itself.


Exceeded maximum rows when reading large files

Something isn't working

Hi @zijwang, thanks for reporting this issue.

You did not mention which `datasets` version you are using, but looking at the code in the stack trace, it seems you are using an old version.

Could you please update `datasets` (`pip install -U datasets`) and check if the problem persists?

## Describe the bug
A clear and concise description of what the bug is.
When using `load_dataset` with json files, if the files are too large, there will be "Exceeded maximum rows" error.

## Steps to reproduce the bug
```python
dataset = load_dataset('json', data_files=data_files) # data files have 3M rows in a single file
```

## Expected results
No error

## Actual results
```
~/anaconda3/envs/python/lib/python3.9/site-packages/datasets/packaged_modules/json/json.py in _generate_tables(self, files)
    134                                 with open(file, encoding="utf-8") as f:
--> 135                                     dataset = json.load(f)
    136                             except json.JSONDecodeError:

~/anaconda3/envs/python/lib/python3.9/json/__init__.py in load(fp, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    292     """
--> 293     return loads(fp.read(),
    294         cls=cls, object_hook=object_hook,

~/anaconda3/envs/python/lib/python3.9/json/__init__.py in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    345             parse_constant is None and object_pairs_hook is None and not kw):
--> 346         return _default_decoder.decode(s)
    347     if cls is None:

~/anaconda3/envs/python/lib/python3.9/json/decoder.py in decode(self, s, _w)
    339         if end != len(s):
--> 340             raise JSONDecodeError("Extra data", s, end)
    341         return obj

JSONDecodeError: Extra data: line 2 column 1 (char 20321)

During handling of the above exception, another exception occurred:

ArrowInvalid                              Traceback (most recent call last)
<ipython-input-20-ab3718a6482f> in <module>
----> 1 dataset = load_dataset('json', data_files=data_files)

~/anaconda3/envs/python/lib/python3.9/site-packages/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, script_version, use_auth_token, task, streaming, **config_kwargs)
    841
    842     # Download and prepare data
--> 843     builder_instance.download_and_prepare(
    844         download_config=download_config,
    845         download_mode=download_mode,

~/anaconda3/envs/python/lib/python3.9/site-packages/datasets/builder.py in download_and_prepare(self, download_config, download_mode, ignore_verifications, try_from_hf_gcs, dl_manager, base_path, use_auth_token, **download_and_prepare_kwargs)
    606                             logger.warning("HF google storage unreachable. Downloading and preparing it from source")
    607                     if not downloaded_from_gcs:
--> 608                         self._download_and_prepare(
    609                             dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
    610                         )

~/anaconda3/envs/python/lib/python3.9/site-packages/datasets/builder.py in _download_and_prepare(self, dl_manager, verify_infos, **prepare_split_kwargs)
    684             try:
    685                 # Prepare split will record examples associated to the split
--> 686                 self._prepare_split(split_generator, **prepare_split_kwargs)
    687             except OSError as e:
    688                 raise OSError(

~/anaconda3/envs/python/lib/python3.9/site-packages/datasets/builder.py in _prepare_split(self, split_generator)
   1153         generator = self._generate_tables(**split_generator.gen_kwargs)
   1154         with ArrowWriter(features=self.info.features, path=fpath) as writer:
-> 1155             for key, table in utils.tqdm(
   1156                 generator, unit=" tables", leave=False, disable=bool(logging.get_verbosity() == logging.NOTSET)
   1157             ):

~/anaconda3/envs/python/lib/python3.9/site-packages/datasets/packaged_modules/json/json.py in _generate_tables(self, files)
    135                                     dataset = json.load(f)
    136                             except json.JSONDecodeError:
--> 137                                 raise e
    138                             raise ValueError(
    139                                 f"Not able to read records in the JSON file at {file}. "

~/anaconda3/envs/python/lib/python3.9/site-packages/datasets/packaged_modules/json/json.py in _generate_tables(self, files)
    114                             while True:
    115                                 try:
--> 116                                     pa_table = paj.read_json(
    117                                         BytesIO(batch), read_options=paj.ReadOptions(block_size=block_size)
    118                                     )

~/anaconda3/envs/python/lib/python3.9/site-packages/pyarrow/_json.pyx in pyarrow._json.read_json()

~/anaconda3/envs/python/lib/python3.9/site-packages/pyarrow/error.pxi in pyarrow.lib.pyarrow_internal_check_status()

~/anaconda3/envs/python/lib/python3.9/site-packages/pyarrow/error.pxi in pyarrow.lib.check_status()

ArrowInvalid: Exceeded maximum rows
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:
- Platform: Linux
- Python version: 3.9
- PyArrow version: 4.0.1


OpenSLR 25: ASR data for Amharic, Swahili and Wolof

dataset request

Requesting to add a new dataset

Whoever handles this just needs to:

- [ ] fork the HuggingFace Datasets repo
- [ ] update the [existing dataset script](https://github.com/huggingface/datasets/blob/master/datasets/openslr/openslr.py) to add SLR25. Lots of copypasting from other sections of the script should make that easy.
Amharic URL: https://www.openslr.org/resources/25/data_readspeech_am.tar.bz2.
Swahili URL: https://www.openslr.org/resources/25/data_broadcastnews_sw.tar.bz2,
Wolof URL: https://www.openslr.org/resources/25/data_readspeech_wo.tar.bz2
- [ ] update the [data card](https://github.com/huggingface/datasets/blob/master/datasets/openslr/README.md) to include information about SLR25. There's lots of other examples to draw from.
- [ ] add the appropriate language tags to the data card as well. https://www.w3.org/International/questions/qa-choosing-language-tags, or just use `sw`, `am`, and `wo` for consistency.
- [ ] make a pull request to merge your changes back into HuggingFace's repo

... also the example in "use in datasets library" should be updated. It currently says
![image](https://user-images.githubusercontent.com/4109253/135115980-8583a44a-cae6-4121-b699-00667020849f.png)
But you actually have to specify a subset, e.g.
```python
dataset = load_dataset("openslr", "SLR32")
```

## Adding a Dataset
- **Name:** *SLR25*
- **Description:** *Subset 25 from OpenSLR. Other subsets have been added to https://huggingface.co/datasets/openslr, 25 covers Amharic, Swahili and Wolof data*
- **Paper:** *https://www.openslr.org/25/ has citations for each of the three subsubsets. *
- **Data:** *Currently the three links to the .tar.bz2 files can be found a thttps://www.openslr.org/25/*
- **Motivation:** *Increase ASR data for underrepresented African languages. Also, other subsets of OpenSLR speech recognition have been uploaded, so this would be easy.*

https://github.com/huggingface/datasets/blob/master/datasets/openslr/openslr.py already has been created for various other OpenSLR subsets, this should be relatively straightforward to do.


ValueError when computing f1 metric with average None

bug

Something isn't working

Hi @asofiaoliveira, thanks for reporting.

I'm fixing it.

## Describe the bug
When I try to compute the f1 score for each class in a multiclass classification problem, I get a ValueError. The same happens with recall and precision. I traced the error to the `.item()` in these scripts, which is probably there for the other averages. E.g. from f1.py:
```python
return {
            "f1": f1_score(
                references,
                predictions,
                labels=labels,
                pos_label=pos_label,
                average=average,
                sample_weight=sample_weight,
            ).item(),
        }
```
Since the result is an array with more than one item, the `.item()` throws the error. I didn't submit a PR because this might be needed for the other averages, I'm not very familiar with the library

## Steps to reproduce the bug
```python
from datasets import load_metric
metric = load_metric("f1")
metric.add_batch(predictions=[2,34,1,34,1,2,3], references=[23,52,1,3,523,5,8])
metric.compute(average=None)
```

## Expected results
`array([0.66666667, 0.        , 0.        , 0.        , 0.        ,
       0.        , 0.        , 0.        , 0.        ])`

## Actual results
ValueError: can only convert an array of size 1 to a Python scalar

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.9.5
- PyArrow version: 5.0.0


Run CI tests against non-production server

Hey @albertvillanova could you provide more context, including extracts from the discussion we had ?

Let's ping @Pierrci @julien-c and @n1t0 for their opinion about that

@julien-c increased the huggingface.co production workers in order to see if it solve [the 502 you had this morning](https://app.circleci.com/pipelines/github/huggingface/datasets/7843/workflows/fc83fa32-18f5-4dc3-9e2f-ba277ae1af74)

For the decision process: be aware that moon-staging does not have persistent repos (they are deleted regularly). as a consequence, **if the moon-staging solution is validated**, you should consider a way to keep the repository that are loaded in tests. These are the ones I found : https://github.com/huggingface/datasets/blob/d488db2f64f312f88f72bbc57a09b7eddb329182/tests/test_load.py and https://github.com/huggingface/datasets/blob/40773111c3e7db8a992fa1c48af32d900a1018d6/tests/test_streaming_download_manager.

Currently, the CI test suite performs requests to the HF production server.

As discussed with @elishowk, we should refactor our tests to use the HF staging server instead, like `huggingface_hub` and `transformers`.


Impossible to load compressed csv

Something isn't working

Hi @Valahaar, thanks for reporting and for your investigation about the source cause.

You are right and that commit prevents `pandas` from inferring the compression. On the other hand, @lhoestq did that change to support loading that dataset in streaming mode.

I'm fixing it.

## Describe the bug
It is not possible to load from a compressed csv anymore.

## Steps to reproduce the bug
```python
load_dataset('csv', data_files=['/path/to/csv.bz2'])
```

## Problem and possible solution
This used to work, but the commit that broke it is [this one](https://github.com/huggingface/datasets/commit/ad489d4597381fc2d12c77841642cbeaecf7a2e0#diff-6f60f8d0552b75be8b3bfd09994480fd60dcd4e7eb08d02f721218c3acdd2782).

`pandas` usually gets the compression information from the filename itself (which was previously directly passed). Now, since it gets a file descriptor, it might be good to auto-infer the compression or let the user pass the `compression` kwarg to `load_dataset` (or maybe warn the user if the file ends with a commonly known compression scheme?).

## Environment info
- `datasets` version: 1.10.0 (and over)
- Platform: Linux-5.8.0-45-generic-x86_64-with-glibc2.17
- Python version: 3.8.10
- PyArrow version: 3.0.0


Can't load dataset

Something isn't working

Hi @mskovalova,

Some datasets have multiple configurations. Therefore, in order to load them, you have to specify both the *dataset name* and the *configuration name*.

In the error message you got, you have a usage example:
- To load the 'wikitext-103-raw-v1' configuration of the 'wikitext' dataset, you should use:
  ```python
  load_dataset('wikitext', 'wikitext-103-raw-v1')
  ```

In your case, if you would like to load the 'wikitext-2-v1' configuration of the 'wikitext' dataset, please use:
```python
raw_datasets = load_dataset("wikitext", "wikitext-2-v1")
```

I'm trying to load a wikitext dataset

```
from datasets import load_dataset
raw_datasets = load_dataset("wikitext")
```

ValueError: Config name is missing.
Please pick one among the available configs: ['wikitext-103-raw-v1', 'wikitext-2-raw-v1', 'wikitext-103-v1', 'wikitext-2-v1']
Example of usage:
	`load_dataset('wikitext', 'wikitext-103-raw-v1')`.

If I try
```
from datasets import load_dataset
raw_datasets = load_dataset("wikitext-2-v1")
```

FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/huggingface/datasets/1.12.1/datasets/wikitext-2-v1/wikitext-2-v1.py

#### Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic (colab)
- Python version: 3.7.12
- PyArrow version: 3.0.0


OSError: Not enough disk space.

Something isn't working

Maybe we can change the disk space calculating API from `shutil.disk_usage` to  `os.statvfs` in UNIX-like system, which can provide correct results.
```
statvfs = os.statvfs('path')
avail_space_bytes = statvfs.f_frsize * statvfs.f_bavail
```

Hi @qqaatw, thanks for reporting.

Could you please try:
```python
dataset = load_dataset("natural_questions", cache_dir=os.path.abspath(args.dataset_cache_dir))
```

@albertvillanova it works! Thanks for your suggestion. Is that a bug of `DownloadConfig`?

`DownloadConfig` only sets the location to download the files. On the other hand, `cache_dir` sets the location for both downloading and caching the data. You can find more information here: https://huggingface.co/docs/datasets/loading_datasets.html#cache-directory

## Describe the bug
I'm trying to download `natural_questions` dataset from the Internet, and I've specified the cache_dir which locates in a mounted disk and has enough disk space. However, even though the space is enough, the disk space checking function still reports the space of root `/` disk having no enough space.

The file system structure is like below. The root `/` has `115G` disk space available, and the `sda1` is mounted to `/mnt`, which has `1.2T` disk space available:
```
/
/mnt/sda1/path/to/args.dataset_cache_dir
```

## Steps to reproduce the bug
```python
dataset_config = DownloadConfig(
    cache_dir=os.path.abspath(args.dataset_cache_dir),
    resume_download=True,
)
dataset = load_dataset("natural_questions", download_config=dataset_config)
```

## Expected results

Can download the dataset without an error.

## Actual results

The following error raised:
```
OSError: Not enough disk space. Needed: 134.92 GiB (download: 41.97 GiB, generated: 92.95 GiB, post-processed: Unknown size)
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.9.0
- Platform: Ubuntu 18.04
- Python version: 3.8.10
- PyArrow version: