"File name too long" error for file locks
## Describe the bug

Getting the following error when calling `load_dataset("gar1t/test")`:

```
OSError: [Errno 36] File name too long: '<user>/.cache/huggingface/datasets/_home_garrett_.cache_huggingface_datasets_csv_test-7c856aea083a7043_0.0.0_9144e0a4e8435090117cea53e6c7537173ef2304525df4a077c435d8ee7828ff.incomplete.lock'
```

## Steps to reproduce the bug

Where the user cache dir (e.g. `~/.cache`) is on a file system that limits filenames to 255 chars (e.g. ext4):

```python
from datasets import load_dataset
load_dataset("gar1t/test")
```

## Expected results

Expect the function to return without an error.

## Actual results

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<python_venv>/lib/python3.9/site-packages/datasets/load.py", line 1112, in load_dataset
    builder_instance.download_and_prepare(
  File "<python_venv>/lib/python3.9/site-packages/datasets/builder.py", line 644, in download_and_prepare
    self._save_info()
  File "<python_venv>/lib/python3.9/site-packages/datasets/builder.py", line 765, in _save_info
    with FileLock(lock_path):
  File "<python_venv>/lib/python3.9/site-packages/datasets/utils/filelock.py", line 323, in __enter__
    self.acquire()
  File "<python_venv>/lib/python3.9/site-packages/datasets/utils/filelock.py", line 272, in acquire
    self._acquire()
  File "<python_venv>/lib/python3.9/site-packages/datasets/utils/filelock.py", line 403, in _acquire
    fd = os.open(self._lock_file, open_mode)
OSError: [Errno 36] File name too long: '<user>/.cache/huggingface/datasets/_home_garrett_.cache_huggingface_datasets_csv_test-7c856aea083a7043_0.0.0_9144e0a4e8435090117cea53e6c7537173ef2304525df4a077c435d8ee7828ff.incomplete.lock'
```

## Environment info

- `datasets` version: 1.12.1
- Platform: Linux-5.11.0-27-generic-x86_64-with-glibc2.31
- Python version: 3.9.7
- PyArrow version: 5.0.0

Yes, you're right! I need to get you more info here. Either there's something going with the name itself that the file system doesn't like (an encoding that blows up the name length??) or perhaps there's something with the path that's causing the entire string to  be used as a name. I haven't seen this on any system before and the Internet's not forthcoming with any info.


Unwanted progress bars when accessing examples
When accessing examples from a dataset formatted for pytorch, some progress bars appear when accessing examples:
```python
In [1]: import datasets as ds

In [2]: d = ds.Dataset.from_dict({"a": [0, 1, 2]}).with_format("torch")

In [3]: d[0]
100%|████████████████████████████████| 1/1 [00:00<00:00, 3172.70it/s]
Out[3]: {'a': tensor(0)}
```

This is because the pytorch formatter calls `map_nested` that uses progress bars

cc @sgugger
doing a patch release now :)


`Can not decode content-encoding: gzip` when loading `scitldr` dataset with streaming
## Describe the bug

Trying to load the `"FullText"` config of the `"scitldr"` dataset with `streaming=True` raises an error from `aiohttp`:
```python
ClientPayloadError: 400, message='Can not decode content-encoding: gzip'
```

cc @lhoestq

## Steps to reproduce the bug
```python
from datasets import load_dataset

iter_dset = iter(
    load_dataset("scitldr", name="FullText", split="test", streaming=True)
)

next(iter_dset)
```

## Expected results
Returns the first sample of the dataset

## Actual results
Calling `__next__` crashes with the following Traceback:

```python
----> 1 next(dset_iter)

~\miniconda3\envs\datasets\lib\site-packages\datasets\iterable_dataset.py in __iter__(self)
    339
    340     def __iter__(self):
--> 341         for key, example in self._iter():
    342             if self.features:
    343                 # we encode the example for ClassLabel feature types for example

~\miniconda3\envs\datasets\lib\site-packages\datasets\iterable_dataset.py in _iter(self)
    336         else:
    337             ex_iterable = self._ex_iterable
--> 338         yield from ex_iterable
    339
    340     def __iter__(self):

~\miniconda3\envs\datasets\lib\site-packages\datasets\iterable_dataset.py in __iter__(self)
     76
     77     def __iter__(self):
---> 78         for key, example in self.generate_examples_fn(**self.kwargs):
     79             yield key, example
     80

~\.cache\huggingface\modules\datasets_modules\datasets\scitldr\72d6e2195786c57e1d343066fb2cc4f93ea39c5e381e53e6ae7c44bbfd1f05ef\scitldr.py in _generate_examples(self, filepath, split)
    162
    163         with open(filepath, encoding="utf-8") as f:
--> 164             for id_, row in enumerate(f):
    165                 data = json.loads(row)
    166                 if self.config.name == "AIC":

~\miniconda3\envs\datasets\lib\site-packages\fsspec\implementations\http.py in read(self, length)
    496         else:
    497             length = min(self.size - self.loc, length)
--> 498         return super().read(length)
    499
    500     async def async_fetch_all(self):

~\miniconda3\envs\datasets\lib\site-packages\fsspec\spec.py in read(self, length)
   1481             # don't even bother calling fetch
   1482             return b""
-> 1483         out = self.cache._fetch(self.loc, self.loc + length)
   1484         self.loc += len(out)
   1485         return out

~\miniconda3\envs\datasets\lib\site-packages\fsspec\caching.py in _fetch(self, start, end)
    378         elif start < self.start:
    379             if self.end - end > self.blocksize:
--> 380                 self.cache = self.fetcher(start, bend)
    381                 self.start = start
    382             else:

~\miniconda3\envs\datasets\lib\site-packages\fsspec\asyn.py in wrapper(*args, **kwargs)
     86     def wrapper(*args, **kwargs):
     87         self = obj or args[0]
---> 88         return sync(self.loop, func, *args, **kwargs)
     89
     90     return wrapper

~\miniconda3\envs\datasets\lib\site-packages\fsspec\asyn.py in sync(loop, func, timeout, *args, **kwargs)
     67         raise FSTimeoutError
     68     if isinstance(result[0], BaseException):
---> 69         raise result[0]
     70     return result[0]
     71

~\miniconda3\envs\datasets\lib\site-packages\fsspec\asyn.py in _runner(event, coro, result, timeout)
     23         coro = asyncio.wait_for(coro, timeout=timeout)
     24     try:
---> 25         result[0] = await coro
     26     except Exception as ex:
     27         result[0] = ex

~\miniconda3\envs\datasets\lib\site-packages\fsspec\implementations\http.py in async_fetch_range(self, start, end)
    538             if r.status == 206:
    539                 # partial content, as expected
--> 540                 out = await r.read()
    541             elif "Content-Length" in r.headers:
    542                 cl = int(r.headers["Content-Length"])

~\miniconda3\envs\datasets\lib\site-packages\aiohttp\client_reqrep.py in read(self)
   1030         if self._body is None:
   1031             try:
-> 1032                 self._body = await self.content.read()
   1033                 for trace in self._traces:
   1034                     await trace.send_response_chunk_received(

~\miniconda3\envs\datasets\lib\site-packages\aiohttp\streams.py in read(self, n)
    342     async def read(self, n: int = -1) -> bytes:
    343         if self._exception is not None:
--> 344             raise self._exception
    345
    346         # migration problem; with DataQueue you have to catch

ClientPayloadError: 400, message='Can not decode content-encoding: gzip'
```

## Environment info

- `datasets` version: 1.12.0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.8.5
- PyArrow version: 2.0.0
- aiohttp version: 3.7.4.post0

Hi @SBrandeis, thanks for reporting! ^^

I think this is an issue with `fsspec`: https://github.com/intake/filesystem_spec/issues/389

I will ask them if they are planning to fix it...


`Can not decode content-encoding: gzip` when loading `scitldr` dataset with streaming
## Describe the bug

Trying to load the `"FullText"` config of the `"scitldr"` dataset with `streaming=True` raises an error from `aiohttp`:
```python
ClientPayloadError: 400, message='Can not decode content-encoding: gzip'
```

cc @lhoestq

## Steps to reproduce the bug
```python
from datasets import load_dataset

iter_dset = iter(
    load_dataset("scitldr", name="FullText", split="test", streaming=True)
)

next(iter_dset)
```

## Expected results
Returns the first sample of the dataset

## Actual results
Calling `__next__` crashes with the following Traceback:

```python
----> 1 next(dset_iter)

~\miniconda3\envs\datasets\lib\site-packages\datasets\iterable_dataset.py in __iter__(self)
    339
    340     def __iter__(self):
--> 341         for key, example in self._iter():
    342             if self.features:
    343                 # we encode the example for ClassLabel feature types for example

~\miniconda3\envs\datasets\lib\site-packages\datasets\iterable_dataset.py in _iter(self)
    336         else:
    337             ex_iterable = self._ex_iterable
--> 338         yield from ex_iterable
    339
    340     def __iter__(self):

~\miniconda3\envs\datasets\lib\site-packages\datasets\iterable_dataset.py in __iter__(self)
     76
     77     def __iter__(self):
---> 78         for key, example in self.generate_examples_fn(**self.kwargs):
     79             yield key, example
     80

~\.cache\huggingface\modules\datasets_modules\datasets\scitldr\72d6e2195786c57e1d343066fb2cc4f93ea39c5e381e53e6ae7c44bbfd1f05ef\scitldr.py in _generate_examples(self, filepath, split)
    162
    163         with open(filepath, encoding="utf-8") as f:
--> 164             for id_, row in enumerate(f):
    165                 data = json.loads(row)
    166                 if self.config.name == "AIC":

~\miniconda3\envs\datasets\lib\site-packages\fsspec\implementations\http.py in read(self, length)
    496         else:
    497             length = min(self.size - self.loc, length)
--> 498         return super().read(length)
    499
    500     async def async_fetch_all(self):

~\miniconda3\envs\datasets\lib\site-packages\fsspec\spec.py in read(self, length)
   1481             # don't even bother calling fetch
   1482             return b""
-> 1483         out = self.cache._fetch(self.loc, self.loc + length)
   1484         self.loc += len(out)
   1485         return out

~\miniconda3\envs\datasets\lib\site-packages\fsspec\caching.py in _fetch(self, start, end)
    378         elif start < self.start:
    379             if self.end - end > self.blocksize:
--> 380                 self.cache = self.fetcher(start, bend)
    381                 self.start = start
    382             else:

~\miniconda3\envs\datasets\lib\site-packages\fsspec\asyn.py in wrapper(*args, **kwargs)
     86     def wrapper(*args, **kwargs):
     87         self = obj or args[0]
---> 88         return sync(self.loop, func, *args, **kwargs)
     89
     90     return wrapper

~\miniconda3\envs\datasets\lib\site-packages\fsspec\asyn.py in sync(loop, func, timeout, *args, **kwargs)
     67         raise FSTimeoutError
     68     if isinstance(result[0], BaseException):
---> 69         raise result[0]
     70     return result[0]
     71

~\miniconda3\envs\datasets\lib\site-packages\fsspec\asyn.py in _runner(event, coro, result, timeout)
     23         coro = asyncio.wait_for(coro, timeout=timeout)
     24     try:
---> 25         result[0] = await coro
     26     except Exception as ex:
     27         result[0] = ex

~\miniconda3\envs\datasets\lib\site-packages\fsspec\implementations\http.py in async_fetch_range(self, start, end)
    538             if r.status == 206:
    539                 # partial content, as expected
--> 540                 out = await r.read()
    541             elif "Content-Length" in r.headers:
    542                 cl = int(r.headers["Content-Length"])

~\miniconda3\envs\datasets\lib\site-packages\aiohttp\client_reqrep.py in read(self)
   1030         if self._body is None:
   1031             try:
-> 1032                 self._body = await self.content.read()
   1033                 for trace in self._traces:
   1034                     await trace.send_response_chunk_received(

~\miniconda3\envs\datasets\lib\site-packages\aiohttp\streams.py in read(self, n)
    342     async def read(self, n: int = -1) -> bytes:
    343         if self._exception is not None:
--> 344             raise self._exception
    345
    346         # migration problem; with DataQueue you have to catch

ClientPayloadError: 400, message='Can not decode content-encoding: gzip'
```

## Environment info

- `datasets` version: 1.12.0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.8.5
- PyArrow version: 2.0.0
- aiohttp version: 3.7.4.post0

Code to reproduce the bug: `ClientPayloadError: 400, message='Can not decode content-encoding: gzip'`
```python
In [1]: import fsspec

In [2]: import json

In [3]: with fsspec.open('https://raw.githubusercontent.com/allenai/scitldr/master/SciTLDR-Data/SciTLDR-FullText/test.jsonl', encoding="utf-8") as f:
   ...:     for row in f:
   ...:         data = json.loads(row)
   ...:
---------------------------------------------------------------------------
ClientPayloadError                        Traceback (most recent call last)
```


`Can not decode content-encoding: gzip` when loading `scitldr` dataset with streaming
## Describe the bug

Trying to load the `"FullText"` config of the `"scitldr"` dataset with `streaming=True` raises an error from `aiohttp`:
```python
ClientPayloadError: 400, message='Can not decode content-encoding: gzip'
```

cc @lhoestq

## Steps to reproduce the bug
```python
from datasets import load_dataset

iter_dset = iter(
    load_dataset("scitldr", name="FullText", split="test", streaming=True)
)

next(iter_dset)
```

## Expected results
Returns the first sample of the dataset

## Actual results
Calling `__next__` crashes with the following Traceback:

```python
----> 1 next(dset_iter)

~\miniconda3\envs\datasets\lib\site-packages\datasets\iterable_dataset.py in __iter__(self)
    339
    340     def __iter__(self):
--> 341         for key, example in self._iter():
    342             if self.features:
    343                 # we encode the example for ClassLabel feature types for example

~\miniconda3\envs\datasets\lib\site-packages\datasets\iterable_dataset.py in _iter(self)
    336         else:
    337             ex_iterable = self._ex_iterable
--> 338         yield from ex_iterable
    339
    340     def __iter__(self):

~\miniconda3\envs\datasets\lib\site-packages\datasets\iterable_dataset.py in __iter__(self)
     76
     77     def __iter__(self):
---> 78         for key, example in self.generate_examples_fn(**self.kwargs):
     79             yield key, example
     80

~\.cache\huggingface\modules\datasets_modules\datasets\scitldr\72d6e2195786c57e1d343066fb2cc4f93ea39c5e381e53e6ae7c44bbfd1f05ef\scitldr.py in _generate_examples(self, filepath, split)
    162
    163         with open(filepath, encoding="utf-8") as f:
--> 164             for id_, row in enumerate(f):
    165                 data = json.loads(row)
    166                 if self.config.name == "AIC":

~\miniconda3\envs\datasets\lib\site-packages\fsspec\implementations\http.py in read(self, length)
    496         else:
    497             length = min(self.size - self.loc, length)
--> 498         return super().read(length)
    499
    500     async def async_fetch_all(self):

~\miniconda3\envs\datasets\lib\site-packages\fsspec\spec.py in read(self, length)
   1481             # don't even bother calling fetch
   1482             return b""
-> 1483         out = self.cache._fetch(self.loc, self.loc + length)
   1484         self.loc += len(out)
   1485         return out

~\miniconda3\envs\datasets\lib\site-packages\fsspec\caching.py in _fetch(self, start, end)
    378         elif start < self.start:
    379             if self.end - end > self.blocksize:
--> 380                 self.cache = self.fetcher(start, bend)
    381                 self.start = start
    382             else:

~\miniconda3\envs\datasets\lib\site-packages\fsspec\asyn.py in wrapper(*args, **kwargs)
     86     def wrapper(*args, **kwargs):
     87         self = obj or args[0]
---> 88         return sync(self.loop, func, *args, **kwargs)
     89
     90     return wrapper

~\miniconda3\envs\datasets\lib\site-packages\fsspec\asyn.py in sync(loop, func, timeout, *args, **kwargs)
     67         raise FSTimeoutError
     68     if isinstance(result[0], BaseException):
---> 69         raise result[0]
     70     return result[0]
     71

~\miniconda3\envs\datasets\lib\site-packages\fsspec\asyn.py in _runner(event, coro, result, timeout)
     23         coro = asyncio.wait_for(coro, timeout=timeout)
     24     try:
---> 25         result[0] = await coro
     26     except Exception as ex:
     27         result[0] = ex

~\miniconda3\envs\datasets\lib\site-packages\fsspec\implementations\http.py in async_fetch_range(self, start, end)
    538             if r.status == 206:
    539                 # partial content, as expected
--> 540                 out = await r.read()
    541             elif "Content-Length" in r.headers:
    542                 cl = int(r.headers["Content-Length"])

~\miniconda3\envs\datasets\lib\site-packages\aiohttp\client_reqrep.py in read(self)
   1030         if self._body is None:
   1031             try:
-> 1032                 self._body = await self.content.read()
   1033                 for trace in self._traces:
   1034                     await trace.send_response_chunk_received(

~\miniconda3\envs\datasets\lib\site-packages\aiohttp\streams.py in read(self, n)
    342     async def read(self, n: int = -1) -> bytes:
    343         if self._exception is not None:
--> 344             raise self._exception
    345
    346         # migration problem; with DataQueue you have to catch

ClientPayloadError: 400, message='Can not decode content-encoding: gzip'
```

## Environment info

- `datasets` version: 1.12.0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.8.5
- PyArrow version: 2.0.0
- aiohttp version: 3.7.4.post0

Thanks for investigating @albertvillanova ! 🤗


windows download abnormal

windows download abnormal
## Describe the bug
The script clearly exists (accessible from the browser), but the script download fails on windows. Then I tried it again and it can be downloaded normally on linux. why??
## Steps to reproduce the bug
```python3.7 + windows
![image](https://user-images.githubusercontent.com/52347799/133436174-4303f847-55d5-434f-a749-08da3bb9b654.png)

# Sample code to reproduce the bug
```

## Expected results
It can be downloaded normally.

## Actual results
it cann't

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:1.11.0
- Platform:windows
- Python version:3.7
- PyArrow version:

Hi ! Is there some kind of proxy that is configured in your browser that gives you access to internet ? If it's the case it could explain why it doesn't work in the code, since the proxy wouldn't be used


windows download abnormal

windows download abnormal
## Describe the bug
The script clearly exists (accessible from the browser), but the script download fails on windows. Then I tried it again and it can be downloaded normally on linux. why??
## Steps to reproduce the bug
```python3.7 + windows
![image](https://user-images.githubusercontent.com/52347799/133436174-4303f847-55d5-434f-a749-08da3bb9b654.png)

# Sample code to reproduce the bug
```

## Expected results
It can be downloaded normally.

## Actual results
it cann't

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:1.11.0
- Platform:windows
- Python version:3.7
- PyArrow version:

It is indeed an agency problem, thank you very, very much


windows download abnormal

windows download abnormal
## Describe the bug
The script clearly exists (accessible from the browser), but the script download fails on windows. Then I tried it again and it can be downloaded normally on linux. why??
## Steps to reproduce the bug
```python3.7 + windows
![image](https://user-images.githubusercontent.com/52347799/133436174-4303f847-55d5-434f-a749-08da3bb9b654.png)

# Sample code to reproduce the bug
```

## Expected results
It can be downloaded normally.

## Actual results
it cann't

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:1.11.0
- Platform:windows
- Python version:3.7
- PyArrow version:

Let me know if you have other questions :)

Closing this issue now


Having a dependency defining fsspec entrypoint raises an AttributeError when importing datasets
## Describe the bug
In one of my project, I defined a custom fsspec filesystem with an entrypoint.
My guess is that by doing so, a variable named `spec` is created in the module `fsspec` (created by entering a for loop as there are entrypoints defined, see the loop in question [here](https://github.com/intake/filesystem_spec/blob/0589358d8a029ed6b60d031018f52be2eb721291/fsspec/__init__.py#L55)).
So that `fsspec.spec`, that was previously referring to the `spec` submodule, is now referring to that `spec` variable.
This make the import of datasets failing as it is using that `fsspec.spec`.

## Steps to reproduce the bug
I could reproduce the bug with a dummy poetry project.

Here is the pyproject.toml:
```toml
[tool.poetry]
name = "debug-datasets"
version = "0.1.0"
description = ""
authors = ["Pierre Godard"]

[tool.poetry.dependencies]
python = "^3.8"
datasets = "^1.11.0"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry.plugins."fsspec.specs"]
"file2" = "fsspec.implementations.local.LocalFileSystem"
```

The only other file being a `debug_datasets/__init__.py` empty file.

The overall structure of the project is as follows:
```
.
├── pyproject.toml
└── debug_datasets
    └── __init__.py
```

Then, within the project folder run:

```
poetry install
poetry run python
```

And in the python interpreter, try to import `datasets`:

```
import datasets
```

## Expected results
The import should run successfully.

## Actual results

Here is the trace of the error I get:

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/godarpi/.cache/pypoetry/virtualenvs/debug-datasets-JuFzTKL--py3.8/lib/python3.8/site-packages/datasets/__init__.py", line 33, in <module>
    from .arrow_dataset import Dataset, concatenate_datasets
  File "/home/godarpi/.cache/pypoetry/virtualenvs/debug-datasets-JuFzTKL--py3.8/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 48, in <module>
    from .filesystems import extract_path_from_uri, is_remote_filesystem
  File "/home/godarpi/.cache/pypoetry/virtualenvs/debug-datasets-JuFzTKL--py3.8/lib/python3.8/site-packages/datasets/filesystems/__init__.py", line 30, in <module>
    def is_remote_filesystem(fs: fsspec.spec.AbstractFileSystem) -> bool:
AttributeError: 'EntryPoint' object has no attribute 'AbstractFileSystem'
```

## Suggested fix

`datasets/filesystems/__init__.py`, line 30, replace:
```
    def is_remote_filesystem(fs: fsspec.spec.AbstractFileSystem) -> bool:
```
by:
```
    def is_remote_filesystem(fs: fsspec.AbstractFileSystem) -> bool:
```

I will come up with a PR soon if this effectively solves the issue.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.11.0
- Platform: WSL2 (Ubuntu 20.04.1 LTS)
- Python version: 3.8.5
- PyArrow version: 5.0.0
- `fsspec` version: 2021.8.1

Closed by #2915.


timit_asr dataset only includes one text phrase
## Describe the bug
The dataset 'timit_asr' only includes one text phrase. It only includes the transcription "Would such an act of refusal be useful?" multiple times rather than different phrases.

## Steps to reproduce the bug
Note: I am following the tutorial https://huggingface.co/blog/fine-tune-wav2vec2-english

1. Install the dataset and other packages
```python
!pip install datasets>=1.5.0
!pip install transformers==4.4.0
!pip install soundfile
!pip install jiwer
```
2. Load the dataset
```python
from datasets import load_dataset, load_metric

timit = load_dataset("timit_asr")
```
3. Remove columns that we don't want
```python
timit = timit.remove_columns(["phonetic_detail", "word_detail", "dialect_region", "id", "sentence_type", "speaker_id"])
```
4. Write a short function to display some random samples of the dataset.
```python
from datasets import ClassLabel
import random
import pandas as pd
from IPython.display import display, HTML

def show_random_elements(dataset, num_examples=10):
    assert num_examples <= len(dataset), "Can't pick more elements than there are in the dataset."
    picks = []
    for _ in range(num_examples):
        pick = random.randint(0, len(dataset)-1)
        while pick in picks:
            pick = random.randint(0, len(dataset)-1)
        picks.append(pick)

    df = pd.DataFrame(dataset[picks])
    display(HTML(df.to_html()))

show_random_elements(timit["train"].remove_columns(["file"]))
```

## Expected results
10 random different transcription phrases.

## Actual results
10 of the same transcription phrase "Would such an act of refusal be useful?"

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.4.1
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.5
- PyArrow version: not listed

Hi @margotwagner,
This bug was fixed in #1995. Upgrading the datasets should work (min v1.8.0 ideally)


timit_asr dataset only includes one text phrase
## Describe the bug
The dataset 'timit_asr' only includes one text phrase. It only includes the transcription "Would such an act of refusal be useful?" multiple times rather than different phrases.

## Steps to reproduce the bug
Note: I am following the tutorial https://huggingface.co/blog/fine-tune-wav2vec2-english

1. Install the dataset and other packages
```python
!pip install datasets>=1.5.0
!pip install transformers==4.4.0
!pip install soundfile
!pip install jiwer
```
2. Load the dataset
```python
from datasets import load_dataset, load_metric

timit = load_dataset("timit_asr")
```
3. Remove columns that we don't want
```python
timit = timit.remove_columns(["phonetic_detail", "word_detail", "dialect_region", "id", "sentence_type", "speaker_id"])
```
4. Write a short function to display some random samples of the dataset.
```python
from datasets import ClassLabel
import random
import pandas as pd
from IPython.display import display, HTML

def show_random_elements(dataset, num_examples=10):
    assert num_examples <= len(dataset), "Can't pick more elements than there are in the dataset."
    picks = []
    for _ in range(num_examples):
        pick = random.randint(0, len(dataset)-1)
        while pick in picks:
            pick = random.randint(0, len(dataset)-1)
        picks.append(pick)

    df = pd.DataFrame(dataset[picks])
    display(HTML(df.to_html()))

show_random_elements(timit["train"].remove_columns(["file"]))
```

## Expected results
10 random different transcription phrases.

## Actual results
10 of the same transcription phrase "Would such an act of refusal be useful?"

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.4.1
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.5
- PyArrow version: not listed

Hi @margotwagner,

Yes, as @bhavitvyamalik has commented, this bug was fixed in `datasets` version 1.5.0. You need to update it, as your current version is 1.4.1:
> Environment info
> - `datasets` version: 1.4.1


FORCE_REDOWNLOAD does not work

FORCE_REDOWNLOAD does not work
## Describe the bug
With GenerateMode.FORCE_REDOWNLOAD, the documentation says
    +------------------------------------+-----------+---------+
    |                                    | Downloads | Dataset |
    +====================================+===========+=========+
    | `REUSE_DATASET_IF_EXISTS` (default)| Reuse     | Reuse   |
    +------------------------------------+-----------+---------+
    | `REUSE_CACHE_IF_EXISTS`            | Reuse     | Fresh   |
    +------------------------------------+-----------+---------+
    | `FORCE_REDOWNLOAD`                 | Fresh     | Fresh   |
    +------------------------------------+-----------+---------+

However, the old dataset is loaded even when FORCE_REDOWNLOAD is chosen.

## Steps to reproduce the bug
```python

import pandas as pd
from datasets import load_dataset, GenerateMode
pd.DataFrame(range(5), columns=['numbers']).to_csv('/tmp/test.tsv.gz', index=False)
ee = load_dataset('csv', data_files=['/tmp/test.tsv.gz'], delimiter='\t', split='train', download_mode=GenerateMode.FORCE_REDOWNLOAD)
print(ee)
pd.DataFrame(range(10), columns=['numerals']).to_csv('/tmp/test.tsv.gz', index=False)
ee = load_dataset('csv', data_files=['/tmp/test.tsv.gz'], delimiter='\t', split='train', download_mode=GenerateMode.FORCE_REDOWNLOAD)
print(ee)

```

## Expected results
Dataset({
    features: ['numbers'],
    num_rows: 5
})
Dataset({
    features: ['numerals'],
    num_rows: 10
})

## Actual results
Dataset({
    features: ['numbers'],
    num_rows: 5
})
Dataset({
    features: ['numbers'],
    num_rows: 5
})

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.8.0
- Platform: Linux-4.14.181-108.257.amzn1.x86_64-x86_64-with-glibc2.10
- Python version: 3.7.10
- PyArrow version: 3.0.0

Hi ! Thanks for reporting. The error seems to happen only if you use compressed files.

The second dataset is prepared in another dataset cache directory than the first - which is normal, since the source file is different. However, it doesn't uncompress the new data file because it finds the old uncompressed data in the extraction cache directory.

If we fix the extraction cache mechanism to uncompress a local file if it changed then it should fix the issue.
Currently the extraction cache mechanism only takes into account the path of the compressed file, which is an issue.


Add WIT Dataset

Add WIT Dataset
## Adding a Dataset
- **Name:** *WIT*
- **Description:** *Wikipedia-based Image Text Dataset*
- **Paper:** *[WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning
](https://arxiv.org/abs/2103.01913)*
- **Data:** *https://github.com/google-research-datasets/wit*
- **Motivation:**  (excerpt from their Github README.md)

> - The largest multimodal dataset (publicly available at the time of this writing) by the number of image-text examples.
> - A massively multilingual dataset (first of its kind) with coverage for over 100+ languages.
> - A collection of diverse set of concepts and real world entities.
> - Brings forth challenging real-world test sets.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

@hassiahk is working on it  #2810


Add WIT Dataset

Add WIT Dataset
## Adding a Dataset
- **Name:** *WIT*
- **Description:** *Wikipedia-based Image Text Dataset*
- **Paper:** *[WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning
](https://arxiv.org/abs/2103.01913)*
- **Data:** *https://github.com/google-research-datasets/wit*
- **Motivation:**  (excerpt from their Github README.md)

> - The largest multimodal dataset (publicly available at the time of this writing) by the number of image-text examples.
> - A massively multilingual dataset (first of its kind) with coverage for over 100+ languages.
> - A collection of diverse set of concepts and real world entities.
> - Brings forth challenging real-world test sets.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

WikiMedia is now hosting the pixel values directly which should make it a lot easier!
The files can be found here:
https://techblog.wikimedia.org/2021/09/09/the-wikipedia-image-caption-matching-challenge-and-a-huge-release-of-image-data-for-research/
https://analytics.wikimedia.org/published/datasets/one-off/caption_competition/training/image_pixels/


Add WIT Dataset

Add WIT Dataset
## Adding a Dataset
- **Name:** *WIT*
- **Description:** *Wikipedia-based Image Text Dataset*
- **Paper:** *[WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning
](https://arxiv.org/abs/2103.01913)*
- **Data:** *https://github.com/google-research-datasets/wit*
- **Motivation:**  (excerpt from their Github README.md)

> - The largest multimodal dataset (publicly available at the time of this writing) by the number of image-text examples.
> - A massively multilingual dataset (first of its kind) with coverage for over 100+ languages.
> - A collection of diverse set of concepts and real world entities.
> - Brings forth challenging real-world test sets.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

> @hassiahk is working on it #2810

Thank you @bhavitvyamalik! Added this issue so we could track progress 😄 . Just linked the PR as well for visibility.


Incompatibility with pytest

Incompatibility with pytest
## Describe the bug

pytest complains about xpathopen / path.open("w")

## Steps to reproduce the bug

Create a test file, `test.py`:

```python
import datasets as ds
def load_dataset():
    ds.load_dataset("counter", split="train", streaming=True)
```

And launch it with pytest:

```bash
python -m pytest test.py
```

## Expected results

It should give something like:

```
collected 1 item

test.py .                                                                                [100%]

======= 1 passed in 3.15s =======
```

## Actual results

```
============================================================================================================================= test session starts ==============================================================================================================================
platform linux -- Python 3.8.11, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /home/slesage/hf/datasets-preview-backend, configfile: pyproject.toml
plugins: anyio-3.3.1
collected 1 item

tests/queries/test_rows.py .                                                                                [100%]Traceback (most recent call last):
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/pytest/__main__.py", line 5, in <module>
    raise SystemExit(pytest.console_main())
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/_pytest/config/__init__.py", line 185, in console_main
    code = main()
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/_pytest/config/__init__.py", line 162, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/_pytest/main.py", line 316, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/_pytest/main.py", line 304, in wrap_session
    config.hook.pytest_sessionfinish(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/_pytest/terminal.py", line 803, in pytest_sessionfinish
    outcome.get_result()
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/_pytest/cacheprovider.py", line 428, in pytest_sessionfinish
    config.cache.set("cache/nodeids", sorted(self.cached_nodeids))
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/_pytest/cacheprovider.py", line 188, in set
    f = path.open("w")
TypeError: xpathopen() takes 1 positional argument but 2 were given
```

## Environment info

- `datasets` version: 1.12.0
- Platform: Linux-5.11.0-1017-aws-x86_64-with-glibc2.29
- Python version: 3.8.11
- PyArrow version: 4.0.1

Sorry, my bad... When implementing `xpathopen`, I just considered the use case in the COUNTER dataset... I'm fixing it!


Error when encoding a dataset with None objects with a Sequence feature
There is an error when encoding a dataset with None objects with a Sequence feature

To reproduce:
```python
from datasets import Dataset, Features, Value, Sequence
data = {"a": [[0], None]}
features = Features({"a": Sequence(Value("int32"))})
dataset = Dataset.from_dict(data, features=features)
```
raises

```python
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-24-40add67f8751> in <module>
      2 data = {"a": [[0], None]}
      3 features = Features({"a": Sequence(Value("int32"))})
----> 4 dataset = Dataset.from_dict(data, features=features)
[...]
~/datasets/features.py in encode_nested_example(schema, obj)
    888         if isinstance(obj, str):  # don't interpret a string as a list
    889             raise ValueError("Got a string but expected a list instead: '{}'".format(obj))
--> 890         return [encode_nested_example(schema.feature, o) for o in obj]
    891     # Object with special encoding:
    892     # ClassLabel will convert from string to int, TranslationVariableLanguages does some checks

TypeError: 'NoneType' object is not iterable
```

Instead, if should run without error, as if the `features` were not passed
This has been fixed by https://github.com/huggingface/datasets/pull/2900
We're doing a new release 1.12 today to make the fix available :)


v1.11.1 release date

v1.11.1 release date
Hello, i need to use latest features in one of my packages but there have been no new datasets release since 2 months ago.

When do you plan to publush v1.11.1 release?
Hi ! Probably 1.12 on monday :)


v1.11.1 release date

v1.11.1 release date
Hello, i need to use latest features in one of my packages but there have been no new datasets release since 2 months ago.

When do you plan to publush v1.11.1 release?
@albertvillanova i think this issue is still valid and should not be closed till `>1.11.0` is published :)


Adding an Elastic Search index to a Dataset
## Describe the bug
When trying to index documents from the squad dataset, the connection to ElasticSearch seems to break:

Reusing dataset squad (/Users/andreasmotz/.cache/huggingface/datasets/squad/plain_text/1.0.0/d6ec3ceb99ca480ce37cdd35555d6cb2511d223b9150cce08a837ef62ffea453)
 90%|████████████████████████████████████████████▉     | 9501/10570 [00:01<00:00, 6335.61docs/s]

No error is thrown, but the indexing breaks ~90%.

## Steps to reproduce the bug
```python
# Sample code to reproduce the bug
from datasets import load_dataset
from elasticsearch import Elasticsearch
es = Elasticsearch()
squad = load_dataset('squad', split='validation')
index_name = "corpus"
es_config = {
    "settings": {
        "number_of_shards": 1,
        "analysis": {"analyzer": {"stop_standard": {"type": "standard", " stopwords": "_english_"}}},
    },
    "mappings": {
        "properties": {
            "idx" : {"type" : "keyword"},
            "title" : {"type" : "keyword"},
            "text": {
                "type": "text",
                "analyzer": "standard",
                "similarity": "BM25"
            },
        }
    },
}
class IndexBuilder:
    """
    Elastic search indexing of a corpus
    """
    def __init__(
        self,
        *args,
        #corpus : None,
        dataset : squad,
        index_name = str,
        query = str,
        config = dict,
        **kwargs,
    ):
        #instantiate HuggingFace dataset
        self.dataset = dataset
        #instantiate ElasticSearch config
        self.config = config
        self.es = Elasticsearch()
        self.index_name = index_name
        self.query = query
    def elastic_index(self):
        print(self.es.info)
        self.es.indices.delete(index=self.index_name, ignore=[400, 404])
        search_index = self.dataset.add_elasticsearch_index(column='context', host='localhost', port='9200', es_index_name=self.index_name, es_index_config=self.config)
        return search_index
    def exact_match_method(self, index):
        scores, retrieved_examples = index.get_nearest_examples('context', query=self.query, k=1)
        return scores, retrieved_examples
if __name__ == "__main__":
    print(type(squad))
    Index = IndexBuilder(dataset=squad, index_name='corpus_index', query='Where was Chopin born?', config=es_config)
    search_index = Index.elastic_index()
    scores, examples = Index.exact_match_method(search_index)
    print(scores, examples)
    for name in squad.column_names:
        print(type(squad[name]))
```

## Environment info
We run the code in Poetry. This might be the issue, since the script runs successfully in our local environment.

Poetry:
- Python version: 3.8
- PyArrow: 4.0.1
- Elasticsearch: 7.13.4
- datasets: 1.10.2

Local:
- Python version: 3.8
- PyArrow: 3.0.0
- Elasticsearch: 7.7.1
- datasets: 1.7.0

Hi, is this bug deterministic in your poetry env ? I mean, does it always stop at 90% or is it random ?

Also, can you try using another version of Elasticsearch ? Maybe there's an issue with the one of you poetry env


`load_dataset('docred')` results in a `NonMatchingChecksumError`
## Describe the bug
I get consistent `NonMatchingChecksumError: Checksums didn't match for dataset source files` errors when trying to execute `datasets.load_dataset('docred')`.

## Steps to reproduce the bug
It is quasi only this code:
```python
import datasets
data = datasets.load_dataset('docred')
```

## Expected results
The DocRED dataset should be loaded without any problems.

## Actual results
```
NonMatchingChecksumError                  Traceback (most recent call last)
<ipython-input-4-b1b83f25a16c> in <module>
----> 1 d = datasets.load_dataset('docred')

~/anaconda3/lib/python3.8/site-packages/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, script_version, use_auth_token, task, streaming, **config_kwargs)
    845
    846     # Download and prepare data
--> 847     builder_instance.download_and_prepare(
    848         download_config=download_config,
    849         download_mode=download_mode,

~/anaconda3/lib/python3.8/site-packages/datasets/builder.py in download_and_prepare(self, download_config, download_mode, ignore_verifications, try_from_hf_gcs, dl_manager, base_path, use_auth_token, **download_and_prepare_kwargs)
    613                             logger.warning("HF google storage unreachable. Downloading and preparing it from source")
    614                     if not downloaded_from_gcs:
--> 615                         self._download_and_prepare(
    616                             dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
    617                         )

~/anaconda3/lib/python3.8/site-packages/datasets/builder.py in _download_and_prepare(self, dl_manager, verify_infos, **prepare_split_kwargs)
    673         # Checksums verification
    674         if verify_infos:
--> 675             verify_checksums(
    676                 self.info.download_checksums, dl_manager.get_recorded_sizes_checksums(), "dataset source files"
    677             )

~/anaconda3/lib/python3.8/site-packages/datasets/utils/info_utils.py in verify_checksums(expected_checksums, recorded_checksums, verification_name)
     38     if len(bad_urls) > 0:
     39         error_msg = "Checksums didn't match" + for_verification_name + ":\n"
---> 40         raise NonMatchingChecksumError(error_msg + str(bad_urls))
     41     logger.info("All the checksums matched successfully" + for_verification_name)
     42

NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/uc?export=download&id=1fDmfUUo5G7gfaoqWWvK81u08m71TK2g7']
```

## Environment info
- `datasets` version: 1.11.0
- Platform: Linux-5.11.0-7633-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyArrow version: 5.0.0

This error also happened on my Windows-partition, after freshly installing python 3.9 and `datasets`.

## Remarks

- I have already called `rm -rf /home/<user>/.cache/huggingface`, i.e., I have tried clearing the cache.
- The problem does not exist for other datasets, i.e., it seems to be DocRED-specific.
Hi @tmpr, thanks for reporting.

Two weeks ago (23th Aug), the host of the source `docred` dataset updated one of the files (`dev.json`): you can see it [here](https://drive.google.com/drive/folders/1c5-0YwnoJx8NS6CV2f-NoTHR__BdkNqw).

Therefore, the checksum needs to be updated.

Normally, in the meantime, you could avoid the error by passing `ignore_verifications=True` to `load_dataset`. However, as the old link points to a non-existing file, the link must be updated too.

I'm fixing all this.


In v1.4.1, all TIMIT train transcripts are "Would such an act of refusal be useful?"
## Describe the bug
Using version 1.4.1 of `datasets`, TIMIT transcripts are all the same.

## Steps to reproduce the bug
I was following this tutorial
- https://huggingface.co/blog/fine-tune-wav2vec2-english

But here's a distilled repro:
```python
!pip install datasets==1.4.1
from datasets import load_dataset
timit = load_dataset("timit_asr", cache_dir="./temp")
unique_transcripts = set(timit["train"]["text"])
print(unique_transcripts)
assert len(unique_transcripts) > 1
```
## Expected results
Expected the correct TIMIT data. Or an error saying that this version of `datasets` can't produce it.

## Actual results
Every train transcript was "Would such an act of refusal be useful?" Every test transcript was "The bungalow was pleasantly situated near the shore."

## Environment info
- `datasets` version: 1.4.1
- Platform: Darwin-18.7.0-x86_64-i386-64bit
- Python version: 3.7.9
- PyTorch version (GPU?): 1.9.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: tried both
- Using distributed or parallel set-up in script?: no
-

Hi @rcgale, thanks for reporting.

Please note that this bug was fixed on `datasets` version 1.5.0: https://github.com/huggingface/datasets/commit/a23c73e526e1c30263834164f16f1fdf76722c8c#diff-f12a7a42d4673bb6c2ca5a40c92c29eb4fe3475908c84fd4ce4fad5dc2514878

If you update `datasets` version, that should work.

On the other hand, would it be possible for @patrickvonplaten to update the [blog post](https://huggingface.co/blog/fine-tune-wav2vec2-english) with the correct version of `datasets`?


In v1.4.1, all TIMIT train transcripts are "Would such an act of refusal be useful?"
## Describe the bug
Using version 1.4.1 of `datasets`, TIMIT transcripts are all the same.

## Steps to reproduce the bug
I was following this tutorial
- https://huggingface.co/blog/fine-tune-wav2vec2-english

But here's a distilled repro:
```python
!pip install datasets==1.4.1
from datasets import load_dataset
timit = load_dataset("timit_asr", cache_dir="./temp")
unique_transcripts = set(timit["train"]["text"])
print(unique_transcripts)
assert len(unique_transcripts) > 1
```
## Expected results
Expected the correct TIMIT data. Or an error saying that this version of `datasets` can't produce it.

## Actual results
Every train transcript was "Would such an act of refusal be useful?" Every test transcript was "The bungalow was pleasantly situated near the shore."

## Environment info
- `datasets` version: 1.4.1
- Platform: Darwin-18.7.0-x86_64-i386-64bit
- Python version: 3.7.9
- PyTorch version (GPU?): 1.9.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: tried both
- Using distributed or parallel set-up in script?: no
-

I just proposed a change in the blog post.

I had assumed there was a data format change that broke a previous version of the code, since presumably @patrickvonplaten tested the tutorial with the version they explicitly referenced. But that fix you linked suggests a problem in the code, which surprised me.

I still wonder, though, is there a way for downloads to be invalidated server-side? If the client can announce its version during a download request, perhaps the server could reject known incompatibilities? It would save much valuable time if `datasets` raised an informative error on a known problem ("Error: the requested data set requires `datasets>=1.5.0`."). This kind of API versioning is a prudent move anyhow, as there will surely come a time when you'll need to make a breaking change to data.


In v1.4.1, all TIMIT train transcripts are "Would such an act of refusal be useful?"
## Describe the bug
Using version 1.4.1 of `datasets`, TIMIT transcripts are all the same.

## Steps to reproduce the bug
I was following this tutorial
- https://huggingface.co/blog/fine-tune-wav2vec2-english

But here's a distilled repro:
```python
!pip install datasets==1.4.1
from datasets import load_dataset
timit = load_dataset("timit_asr", cache_dir="./temp")
unique_transcripts = set(timit["train"]["text"])
print(unique_transcripts)
assert len(unique_transcripts) > 1
```
## Expected results
Expected the correct TIMIT data. Or an error saying that this version of `datasets` can't produce it.

## Actual results
Every train transcript was "Would such an act of refusal be useful?" Every test transcript was "The bungalow was pleasantly situated near the shore."

## Environment info
- `datasets` version: 1.4.1
- Platform: Darwin-18.7.0-x86_64-i386-64bit
- Python version: 3.7.9
- PyTorch version (GPU?): 1.9.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: tried both
- Using distributed or parallel set-up in script?: no
-

Also, thank you for a quick and helpful reply!


datasets.config.PYARROW_VERSION has no attribute 'major'
In the test_dataset_common.py script, line 288-289

```
if datasets.config.PYARROW_VERSION.major < 3:
   packaged_datasets = [pd for pd in packaged_datasets if pd["dataset_name"] != "parquet"]
```

which throws the error below. `datasets.config.PYARROW_VERSION` itself return the string '4.0.1'. I have tested this on both datasets.__version_=='1.11.0' and '1.9.0'. I am using Mac OS.

```
import datasets
datasets.config.PYARROW_VERSION.major
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/var/folders/1f/0wqmlgp90qjd5mpj53fnjq440000gn/T/ipykernel_73361/2547517336.py in <module>
      1 import datasets
----> 2 datasets.config.PYARROW_VERSION.major

AttributeError: 'str' object has no attribute 'major'
```

## Environment info
- `datasets` version: 1.11.0
- Platform: Darwin-20.6.0-x86_64-i386-64bit
- Python version: 3.7.11
- PyArrow version: 4.0.1

I have changed line 288 to `if int(datasets.config.PYARROW_VERSION.split(".")[0]) < 3:` just to get around it.


datasets.config.PYARROW_VERSION has no attribute 'major'
In the test_dataset_common.py script, line 288-289

```
if datasets.config.PYARROW_VERSION.major < 3:
   packaged_datasets = [pd for pd in packaged_datasets if pd["dataset_name"] != "parquet"]
```

which throws the error below. `datasets.config.PYARROW_VERSION` itself return the string '4.0.1'. I have tested this on both datasets.__version_=='1.11.0' and '1.9.0'. I am using Mac OS.

```
import datasets
datasets.config.PYARROW_VERSION.major
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/var/folders/1f/0wqmlgp90qjd5mpj53fnjq440000gn/T/ipykernel_73361/2547517336.py in <module>
      1 import datasets
----> 2 datasets.config.PYARROW_VERSION.major

AttributeError: 'str' object has no attribute 'major'
```

## Environment info
- `datasets` version: 1.11.0
- Platform: Darwin-20.6.0-x86_64-i386-64bit
- Python version: 3.7.11
- PyArrow version: 4.0.1

Hi @bwang482,

I'm sorry but I'm not able to reproduce your bug.

Please note that in our current master branch, we made a commit (d03223d4d64b89e76b48b00602aba5aa2f817f1e) that simultaneously modified:
- test_dataset_common.py: https://github.com/huggingface/datasets/commit/d03223d4d64b89e76b48b00602aba5aa2f817f1e#diff-a1bc225bd9a5bade373d1f140e24d09cbbdc97971c2f73bb627daaa803ada002L289 that introduces the usage of `datasets.config.PYARROW_VERSION.major`
- but also changed config.py: https://github.com/huggingface/datasets/commit/d03223d4d64b89e76b48b00602aba5aa2f817f1e#diff-e021fcfc41811fb970fab889b8d245e68382bca8208e63eaafc9a396a336f8f2L40, so that `datasets.config.PYARROW_VERSION.major` exists


datasets.config.PYARROW_VERSION has no attribute 'major'
In the test_dataset_common.py script, line 288-289

```
if datasets.config.PYARROW_VERSION.major < 3:
   packaged_datasets = [pd for pd in packaged_datasets if pd["dataset_name"] != "parquet"]
```

which throws the error below. `datasets.config.PYARROW_VERSION` itself return the string '4.0.1'. I have tested this on both datasets.__version_=='1.11.0' and '1.9.0'. I am using Mac OS.

```
import datasets
datasets.config.PYARROW_VERSION.major
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/var/folders/1f/0wqmlgp90qjd5mpj53fnjq440000gn/T/ipykernel_73361/2547517336.py in <module>
      1 import datasets
----> 2 datasets.config.PYARROW_VERSION.major

AttributeError: 'str' object has no attribute 'major'
```

## Environment info
- `datasets` version: 1.11.0
- Platform: Darwin-20.6.0-x86_64-i386-64bit
- Python version: 3.7.11
- PyArrow version: 4.0.1

Sorted. Thanks!


datasets.config.PYARROW_VERSION has no attribute 'major'
In the test_dataset_common.py script, line 288-289

```
if datasets.config.PYARROW_VERSION.major < 3:
   packaged_datasets = [pd for pd in packaged_datasets if pd["dataset_name"] != "parquet"]
```

which throws the error below. `datasets.config.PYARROW_VERSION` itself return the string '4.0.1'. I have tested this on both datasets.__version_=='1.11.0' and '1.9.0'. I am using Mac OS.

```
import datasets
datasets.config.PYARROW_VERSION.major
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/var/folders/1f/0wqmlgp90qjd5mpj53fnjq440000gn/T/ipykernel_73361/2547517336.py in <module>
      1 import datasets
----> 2 datasets.config.PYARROW_VERSION.major

AttributeError: 'str' object has no attribute 'major'
```

## Environment info
- `datasets` version: 1.11.0
- Platform: Darwin-20.6.0-x86_64-i386-64bit
- Python version: 3.7.11
- PyArrow version: 4.0.1

Reopening this. Although the `test_dataset_common.py` script works fine now.

Has this got something to do with my pull request not passing `ci/circleci: run_dataset_script_tests_pyarrow` tests?

https://github.com/huggingface/datasets/pull/2873


datasets.config.PYARROW_VERSION has no attribute 'major'
In the test_dataset_common.py script, line 288-289

```
if datasets.config.PYARROW_VERSION.major < 3:
   packaged_datasets = [pd for pd in packaged_datasets if pd["dataset_name"] != "parquet"]
```

which throws the error below. `datasets.config.PYARROW_VERSION` itself return the string '4.0.1'. I have tested this on both datasets.__version_=='1.11.0' and '1.9.0'. I am using Mac OS.

```
import datasets
datasets.config.PYARROW_VERSION.major
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/var/folders/1f/0wqmlgp90qjd5mpj53fnjq440000gn/T/ipykernel_73361/2547517336.py in <module>
      1 import datasets
----> 2 datasets.config.PYARROW_VERSION.major

AttributeError: 'str' object has no attribute 'major'
```

## Environment info
- `datasets` version: 1.11.0
- Platform: Darwin-20.6.0-x86_64-i386-64bit
- Python version: 3.7.11
- PyArrow version: 4.0.1

Hi @bwang482,

If you click on `Details` (on the right of your non passing CI test names: `ci/circleci: run_dataset_script_tests_pyarrow`), you can have more information about the non-passing tests.

For example, for ["ci/circleci: run_dataset_script_tests_pyarrow_1" details](https://circleci.com/gh/huggingface/datasets/46324?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link), you can see that the only non-passing test has to do with the dataset card (missing information in the `README.md` file): `test_changed_dataset_card`
```
=========================== short test summary info ============================
FAILED tests/test_dataset_cards.py::test_changed_dataset_card[swedish_medical_ner]
= 1 failed, 3214 passed, 2874 skipped, 2 xfailed, 1 xpassed, 15 warnings in 175.59s (0:02:55) =
```

Therefore, your PR non-passing test has nothing to do with this issue.


TypeError: 'NoneType' object is not callable
## Describe the bug

TypeError: 'NoneType' object is not callable
## Steps to reproduce the bug
```python
from datasets import load_dataset, load_metric
dataset = datasets.load_dataset("glue", 'cola')
```

## Expected results
A clear and concise description of the expected results.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.11.0
- Platform:
- Python version: 3.7
- PyArrow version:

Hi, @Chenfei-Kang.

I'm sorry, but I'm not able to reproduce your bug:
```python
from datasets import load_dataset

ds = load_dataset("glue", 'cola')
ds
```
```
DatasetDict({
    train: Dataset({
        features: ['sentence', 'label', 'idx'],
        num_rows: 8551
    })
    validation: Dataset({
        features: ['sentence', 'label', 'idx'],
        num_rows: 1043
    })
    test: Dataset({
        features: ['sentence', 'label', 'idx'],
        num_rows: 1063
    })
})
```

Could you please give more details and environment info (platform, PyArrow version)?


TypeError: 'NoneType' object is not callable
## Describe the bug

TypeError: 'NoneType' object is not callable
## Steps to reproduce the bug
```python
from datasets import load_dataset, load_metric
dataset = datasets.load_dataset("glue", 'cola')
```

## Expected results
A clear and concise description of the expected results.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.11.0
- Platform:
- Python version: 3.7
- PyArrow version:

> Hi, @Chenfei-Kang.
>
> I'm sorry, but I'm not able to reproduce your bug:
>
> ```python
> from datasets import load_dataset
>
> ds = load_dataset("glue", 'cola')
> ds
> ```
>
> ```
> DatasetDict({
>     train: Dataset({
>         features: ['sentence', 'label', 'idx'],
>         num_rows: 8551
>     })
>     validation: Dataset({
>         features: ['sentence', 'label', 'idx'],
>         num_rows: 1043
>     })
>     test: Dataset({
>         features: ['sentence', 'label', 'idx'],
>         num_rows: 1063
>     })
> })
> ```
>
> Could you please give more details and environment info (platform, PyArrow version)?

Sorry to reply you so late.
platform: pycharm 2021 + anaconda with python 3.7
PyArrow version: 5.0.0
huggingface-hub: 0.0.16
datasets: 1.9.0


TypeError: 'NoneType' object is not callable
## Describe the bug

TypeError: 'NoneType' object is not callable
## Steps to reproduce the bug
```python
from datasets import load_dataset, load_metric
dataset = datasets.load_dataset("glue", 'cola')
```

## Expected results
A clear and concise description of the expected results.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.11.0
- Platform:
- Python version: 3.7
- PyArrow version:

- For the platform, we need to know the operating system of your machine. Could you please run the command `datasets-cli env` and copy-and-paste its output below?
- In relation with the error, you just gave us the error type and message (`TypeError: 'NoneType' object is not callable`). Could you please copy-paste the complete stack trace, so that we know exactly which part of the code threw the error?


TypeError: 'NoneType' object is not callable
## Describe the bug

TypeError: 'NoneType' object is not callable
## Steps to reproduce the bug
```python
from datasets import load_dataset, load_metric
dataset = datasets.load_dataset("glue", 'cola')
```

## Expected results
A clear and concise description of the expected results.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.11.0
- Platform:
- Python version: 3.7
- PyArrow version:

> * For the platform, we need to know the operating system of your machine. Could you please run the command `datasets-cli env` and copy-and-paste its output below?
> * In relation with the error, you just gave us the error type and message (`TypeError: 'NoneType' object is not callable`). Could you please copy-paste the complete stack trace, so that we know exactly which part of the code threw the error?

1. For the platform, here are the output:
        - datasets` version: 1.11.0
        - Platform: Windows-10-10.0.19041-SP0
        - Python version: 3.7.10
        - PyArrow version: 5.0.0
2. For the code and error：
     ```python
     from datasets import load_dataset, load_metric
     dataset = load_dataset("glue", "cola")
    ```
    ```python
    Traceback (most recent call last):
    ....
    ....
    File "my_file.py", line 2, in <module>
    dataset = load_dataset("glue", "cola")
    File "My environments\lib\site-packages\datasets\load.py", line 830, in load_dataset
    **config_kwargs,
     File "My environments\lib\site-packages\datasets\load.py", line 710, in load_dataset_builder
    **config_kwargs,
    TypeError: 'NoneType' object is not callable
    ```
   Thank you!


TypeError: 'NoneType' object is not callable
## Describe the bug

TypeError: 'NoneType' object is not callable
## Steps to reproduce the bug
```python
from datasets import load_dataset, load_metric
dataset = datasets.load_dataset("glue", 'cola')
```

## Expected results
A clear and concise description of the expected results.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.11.0
- Platform:
- Python version: 3.7
- PyArrow version:

For that environment, I am sorry but I can't reproduce the bug: I can load the dataset without any problem.


TypeError: 'NoneType' object is not callable
## Describe the bug

TypeError: 'NoneType' object is not callable
## Steps to reproduce the bug
```python
from datasets import load_dataset, load_metric
dataset = datasets.load_dataset("glue", 'cola')
```

## Expected results
A clear and concise description of the expected results.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.11.0
- Platform:
- Python version: 3.7
- PyArrow version:

One naive question: do you have internet access from the machine where you execute the code?


TypeError: 'NoneType' object is not callable
## Describe the bug

TypeError: 'NoneType' object is not callable
## Steps to reproduce the bug
```python
from datasets import load_dataset, load_metric
dataset = datasets.load_dataset("glue", 'cola')
```

## Expected results
A clear and concise description of the expected results.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.11.0
- Platform:
- Python version: 3.7
- PyArrow version:

> For that environment, I am sorry but I can't reproduce the bug: I can load the dataset without any problem.

But I can download other task dataset such as `dataset = load_dataset('squad')`. I don't know what went wrong.  Thank you so much!


"counter" dataset raises an error in normal mode, but not in streaming mode
## Describe the bug

`counter` dataset raises an error on `load_dataset()`, but simply returns an empty iterator in streaming mode.

## Steps to reproduce the bug

```python
>>> import datasets as ds
>>> a = ds.load_dataset('counter', split="train", streaming=False)
Using custom data configuration default
Downloading and preparing dataset counter/default (download: 1.29 MiB, generated: 2.48 MiB, post-processed: Unknown size, total: 3.77 MiB) to /home/slesage/.cache/huggingface/datasets/counter/default/1.0.0/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9...
Traceback (most recent call last):
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 726, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 1124, in _prepare_split
    for key, record in utils.tqdm(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/tqdm/std.py", line 1185, in __iter__
    for obj in iterable:
  File "/home/slesage/.cache/huggingface/modules/datasets_modules/datasets/counter/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9/counter.py", line 161, in _generate_examples
    with derived_file.open(encoding="utf-8") as f:
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1222, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1078, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/load.py", line 1112, in load_dataset
    builder_instance.download_and_prepare(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 636, in download_and_prepare
    self._download_and_prepare(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 728, in _download_and_prepare
    raise OSError(
OSError: Cannot find data file.
Original error:
[Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'
```

```python
>>> import datasets as ds
>>> b = ds.load_dataset('counter', split="train", streaming=True)
Using custom data configuration default
>>> list(b)
[]
```

## Expected results

An exception should be raised in streaming mode

## Actual results

No exception is raised in streaming mode: there is no way to tell if something has broken or if the dataset is simply empty.

## Environment info

- `datasets` version: 1.11.1.dev0
- Platform: Linux-5.11.0-1016-aws-x86_64-with-glibc2.29
- Python version: 3.8.11
- PyArrow version: 4.0.1

Hi @severo, thanks for reporting.

Just note that currently not all canonical datasets support streaming mode: this is one case!

All datasets that use `pathlib` joins (using `/`) instead of `os.path.join` (as in this dataset) do not support streaming mode yet.


"counter" dataset raises an error in normal mode, but not in streaming mode
## Describe the bug

`counter` dataset raises an error on `load_dataset()`, but simply returns an empty iterator in streaming mode.

## Steps to reproduce the bug

```python
>>> import datasets as ds
>>> a = ds.load_dataset('counter', split="train", streaming=False)
Using custom data configuration default
Downloading and preparing dataset counter/default (download: 1.29 MiB, generated: 2.48 MiB, post-processed: Unknown size, total: 3.77 MiB) to /home/slesage/.cache/huggingface/datasets/counter/default/1.0.0/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9...
Traceback (most recent call last):
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 726, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 1124, in _prepare_split
    for key, record in utils.tqdm(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/tqdm/std.py", line 1185, in __iter__
    for obj in iterable:
  File "/home/slesage/.cache/huggingface/modules/datasets_modules/datasets/counter/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9/counter.py", line 161, in _generate_examples
    with derived_file.open(encoding="utf-8") as f:
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1222, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1078, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/load.py", line 1112, in load_dataset
    builder_instance.download_and_prepare(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 636, in download_and_prepare
    self._download_and_prepare(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 728, in _download_and_prepare
    raise OSError(
OSError: Cannot find data file.
Original error:
[Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'
```

```python
>>> import datasets as ds
>>> b = ds.load_dataset('counter', split="train", streaming=True)
Using custom data configuration default
>>> list(b)
[]
```

## Expected results

An exception should be raised in streaming mode

## Actual results

No exception is raised in streaming mode: there is no way to tell if something has broken or if the dataset is simply empty.

## Environment info

- `datasets` version: 1.11.1.dev0
- Platform: Linux-5.11.0-1016-aws-x86_64-with-glibc2.29
- Python version: 3.8.11
- PyArrow version: 4.0.1

OK. Do you think it's possible to detect this, and raise an exception (maybe `NotImplementedError`, or a specific `StreamingError`)?


"counter" dataset raises an error in normal mode, but not in streaming mode
## Describe the bug

`counter` dataset raises an error on `load_dataset()`, but simply returns an empty iterator in streaming mode.

## Steps to reproduce the bug

```python
>>> import datasets as ds
>>> a = ds.load_dataset('counter', split="train", streaming=False)
Using custom data configuration default
Downloading and preparing dataset counter/default (download: 1.29 MiB, generated: 2.48 MiB, post-processed: Unknown size, total: 3.77 MiB) to /home/slesage/.cache/huggingface/datasets/counter/default/1.0.0/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9...
Traceback (most recent call last):
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 726, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 1124, in _prepare_split
    for key, record in utils.tqdm(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/tqdm/std.py", line 1185, in __iter__
    for obj in iterable:
  File "/home/slesage/.cache/huggingface/modules/datasets_modules/datasets/counter/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9/counter.py", line 161, in _generate_examples
    with derived_file.open(encoding="utf-8") as f:
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1222, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1078, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/load.py", line 1112, in load_dataset
    builder_instance.download_and_prepare(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 636, in download_and_prepare
    self._download_and_prepare(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 728, in _download_and_prepare
    raise OSError(
OSError: Cannot find data file.
Original error:
[Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'
```

```python
>>> import datasets as ds
>>> b = ds.load_dataset('counter', split="train", streaming=True)
Using custom data configuration default
>>> list(b)
[]
```

## Expected results

An exception should be raised in streaming mode

## Actual results

No exception is raised in streaming mode: there is no way to tell if something has broken or if the dataset is simply empty.

## Environment info

- `datasets` version: 1.11.1.dev0
- Platform: Linux-5.11.0-1016-aws-x86_64-with-glibc2.29
- Python version: 3.8.11
- PyArrow version: 4.0.1

We should definitely support datasets using `pathlib` in streaming mode...

For non-supported datasets in streaming mode, we have already a request of raising an error/warning: see #2654.


"counter" dataset raises an error in normal mode, but not in streaming mode
## Describe the bug

`counter` dataset raises an error on `load_dataset()`, but simply returns an empty iterator in streaming mode.

## Steps to reproduce the bug

```python
>>> import datasets as ds
>>> a = ds.load_dataset('counter', split="train", streaming=False)
Using custom data configuration default
Downloading and preparing dataset counter/default (download: 1.29 MiB, generated: 2.48 MiB, post-processed: Unknown size, total: 3.77 MiB) to /home/slesage/.cache/huggingface/datasets/counter/default/1.0.0/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9...
Traceback (most recent call last):
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 726, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 1124, in _prepare_split
    for key, record in utils.tqdm(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/tqdm/std.py", line 1185, in __iter__
    for obj in iterable:
  File "/home/slesage/.cache/huggingface/modules/datasets_modules/datasets/counter/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9/counter.py", line 161, in _generate_examples
    with derived_file.open(encoding="utf-8") as f:
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1222, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1078, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/load.py", line 1112, in load_dataset
    builder_instance.download_and_prepare(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 636, in download_and_prepare
    self._download_and_prepare(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 728, in _download_and_prepare
    raise OSError(
OSError: Cannot find data file.
Original error:
[Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'
```

```python
>>> import datasets as ds
>>> b = ds.load_dataset('counter', split="train", streaming=True)
Using custom data configuration default
>>> list(b)
[]
```

## Expected results

An exception should be raised in streaming mode

## Actual results

No exception is raised in streaming mode: there is no way to tell if something has broken or if the dataset is simply empty.

## Environment info

- `datasets` version: 1.11.1.dev0
- Platform: Linux-5.11.0-1016-aws-x86_64-with-glibc2.29
- Python version: 3.8.11
- PyArrow version: 4.0.1

Hi @severo, please note that "counter" dataset will be streamable (at least until it arrives at the missing file, error already in normal mode) once these PRs are merged:
- #2874
- #2876
- #2880

I have tested it. 😉


"counter" dataset raises an error in normal mode, but not in streaming mode
## Describe the bug

`counter` dataset raises an error on `load_dataset()`, but simply returns an empty iterator in streaming mode.

## Steps to reproduce the bug

```python
>>> import datasets as ds
>>> a = ds.load_dataset('counter', split="train", streaming=False)
Using custom data configuration default
Downloading and preparing dataset counter/default (download: 1.29 MiB, generated: 2.48 MiB, post-processed: Unknown size, total: 3.77 MiB) to /home/slesage/.cache/huggingface/datasets/counter/default/1.0.0/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9...
Traceback (most recent call last):
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 726, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 1124, in _prepare_split
    for key, record in utils.tqdm(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/tqdm/std.py", line 1185, in __iter__
    for obj in iterable:
  File "/home/slesage/.cache/huggingface/modules/datasets_modules/datasets/counter/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9/counter.py", line 161, in _generate_examples
    with derived_file.open(encoding="utf-8") as f:
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1222, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1078, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/load.py", line 1112, in load_dataset
    builder_instance.download_and_prepare(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 636, in download_and_prepare
    self._download_and_prepare(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 728, in _download_and_prepare
    raise OSError(
OSError: Cannot find data file.
Original error:
[Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'
```

```python
>>> import datasets as ds
>>> b = ds.load_dataset('counter', split="train", streaming=True)
Using custom data configuration default
>>> list(b)
[]
```

## Expected results

An exception should be raised in streaming mode

## Actual results

No exception is raised in streaming mode: there is no way to tell if something has broken or if the dataset is simply empty.

## Environment info

- `datasets` version: 1.11.1.dev0
- Platform: Linux-5.11.0-1016-aws-x86_64-with-glibc2.29
- Python version: 3.8.11
- PyArrow version: 4.0.1

Now (on master), we get:

```
import datasets as ds
ds.load_dataset('counter', split="train", streaming=False)
```

```
Using custom data configuration default
Downloading and preparing dataset counter/default (download: 1.29 MiB, generated: 2.48 MiB, post-processed: Unknown size, total: 3.77 MiB) to /home/slesage/.cache/huggingface/datasets/counter/default/1.0.0/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9...
Traceback (most recent call last):
  File "/home/slesage/hf/datasets/src/datasets/builder.py", line 726, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File "/home/slesage/hf/datasets/src/datasets/builder.py", line 1124, in _prepare_split
    for key, record in utils.tqdm(
  File "/home/slesage/hf/datasets/.venv/lib/python3.8/site-packages/tqdm/std.py", line 1185, in __iter__
    for obj in iterable:
  File "/home/slesage/.cache/huggingface/modules/datasets_modules/datasets/counter/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9/counter.py", line 161, in _generate_examples
    with derived_file.open(encoding="utf-8") as f:
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1222, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1078, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets/src/datasets/load.py", line 1112, in load_dataset
    builder_instance.download_and_prepare(
  File "/home/slesage/hf/datasets/src/datasets/builder.py", line 636, in download_and_prepare
    self._download_and_prepare(
  File "/home/slesage/hf/datasets/src/datasets/builder.py", line 728, in _download_and_prepare
    raise OSError(
OSError: Cannot find data file.
Original error:
[Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'
```

The error is now the same with or without streaming. I close the issue, thanks @albertvillanova and @lhoestq!


"counter" dataset raises an error in normal mode, but not in streaming mode
## Describe the bug

`counter` dataset raises an error on `load_dataset()`, but simply returns an empty iterator in streaming mode.

## Steps to reproduce the bug

```python
>>> import datasets as ds
>>> a = ds.load_dataset('counter', split="train", streaming=False)
Using custom data configuration default
Downloading and preparing dataset counter/default (download: 1.29 MiB, generated: 2.48 MiB, post-processed: Unknown size, total: 3.77 MiB) to /home/slesage/.cache/huggingface/datasets/counter/default/1.0.0/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9...
Traceback (most recent call last):
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 726, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 1124, in _prepare_split
    for key, record in utils.tqdm(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/tqdm/std.py", line 1185, in __iter__
    for obj in iterable:
  File "/home/slesage/.cache/huggingface/modules/datasets_modules/datasets/counter/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9/counter.py", line 161, in _generate_examples
    with derived_file.open(encoding="utf-8") as f:
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1222, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1078, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/load.py", line 1112, in load_dataset
    builder_instance.download_and_prepare(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 636, in download_and_prepare
    self._download_and_prepare(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 728, in _download_and_prepare
    raise OSError(
OSError: Cannot find data file.
Original error:
[Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'
```

```python
>>> import datasets as ds
>>> b = ds.load_dataset('counter', split="train", streaming=True)
Using custom data configuration default
>>> list(b)
[]
```

## Expected results

An exception should be raised in streaming mode

## Actual results

No exception is raised in streaming mode: there is no way to tell if something has broken or if the dataset is simply empty.

## Environment info

- `datasets` version: 1.11.1.dev0
- Platform: Linux-5.11.0-1016-aws-x86_64-with-glibc2.29
- Python version: 3.8.11
- PyArrow version: 4.0.1

Note that we might want to open an issue to fix the "counter" dataset by itself, but I let it up to you.


"counter" dataset raises an error in normal mode, but not in streaming mode
## Describe the bug

`counter` dataset raises an error on `load_dataset()`, but simply returns an empty iterator in streaming mode.

## Steps to reproduce the bug

```python
>>> import datasets as ds
>>> a = ds.load_dataset('counter', split="train", streaming=False)
Using custom data configuration default
Downloading and preparing dataset counter/default (download: 1.29 MiB, generated: 2.48 MiB, post-processed: Unknown size, total: 3.77 MiB) to /home/slesage/.cache/huggingface/datasets/counter/default/1.0.0/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9...
Traceback (most recent call last):
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 726, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 1124, in _prepare_split
    for key, record in utils.tqdm(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/tqdm/std.py", line 1185, in __iter__
    for obj in iterable:
  File "/home/slesage/.cache/huggingface/modules/datasets_modules/datasets/counter/9f84962fa0f35bec5a34fe0bdff8681838d497008c457f7856c48654476ec0e9/counter.py", line 161, in _generate_examples
    with derived_file.open(encoding="utf-8") as f:
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1222, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/home/slesage/.pyenv/versions/3.8.11/lib/python3.8/pathlib.py", line 1078, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/load.py", line 1112, in load_dataset
    builder_instance.download_and_prepare(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 636, in download_and_prepare
    self._download_and_prepare(
  File "/home/slesage/hf/datasets-preview-backend/.venv/lib/python3.8/site-packages/datasets/builder.py", line 728, in _download_and_prepare
    raise OSError(
OSError: Cannot find data file.
Original error:
[Errno 2] No such file or directory: '/home/slesage/.cache/huggingface/datasets/downloads/extracted/b57aa6db5601a738e57b95c1fd8cced54ff28fc540efcdaf0f6c4f1bb5dfe211/COUNTER/0032p.xml'
```

```python
>>> import datasets as ds
>>> b = ds.load_dataset('counter', split="train", streaming=True)
Using custom data configuration default
>>> list(b)
[]
```

## Expected results

An exception should be raised in streaming mode

## Actual results

No exception is raised in streaming mode: there is no way to tell if something has broken or if the dataset is simply empty.

## Environment info

- `datasets` version: 1.11.1.dev0
- Platform: Linux-5.11.0-1016-aws-x86_64-with-glibc2.29
- Python version: 3.8.11
- PyArrow version: 4.0.1

Fixed here: https://github.com/huggingface/datasets/pull/2894. Thanks @albertvillanova


Cannot download TOTTO dataset

Cannot download TOTTO dataset
Error: Couldn't find file at https://storage.googleapis.com/totto/totto_data.zip

`datasets version: 1.11.0`
# How to reproduce:

```py
from datasets import load_dataset
dataset = load_dataset('totto')
```

Hola @mrm8488, thanks for reporting.

Apparently, the data source host changed their URL one week ago: https://github.com/google-research-datasets/ToTTo/commit/cebeb430ec2a97747e704d16a9354f7d9073ff8f

I'm fixing it.


Loading allenai/c4 in streaming mode does too many HEAD requests
This does 60,000+ HEAD requests to get all the ETags of all the data files:
```python
from datasets import load_dataset
load_dataset("allenai/c4", streaming=True)
```
It makes loading the dataset completely impractical.

The ETags are used to compute the config id (it must depend on the data files being used).
Instead of using the ETags, we could simply use the commit hash of the dataset repository on the hub, as well and the glob pattern used to resolve the files (here it's `*` by default, to load all the files of the repository)
https://github.com/huggingface/datasets/blob/6c766f9115d686182d76b1b937cb27e099c45d68/src/datasets/builder.py#L179-L186


Negative timezone

Negative timezone
## Describe the bug
The load_dataset method do not accept a parquet file with a negative timezone, as it has the following regex:
```
"^(s|ms|us|ns),\s*tz=([a-zA-Z0-9/_+:]*)$"
```
So a valid timestap ```timestamp[us, tz=-03:00]``` returns an error when loading parquet files.

## Steps to reproduce the bug
```python
# Where the timestamp column has a tz of -03:00
datasets = load_dataset('parquet', data_files={'train': train_files, 'validation': validation_files,
                                        'test': test_files}, cache_dir="./cache_teste/")
```

## Expected results
The -03:00 is a valid tz so the regex should accept this without raising an error.

## Actual results
As this regex disaproves a valid tz it raises the following error:
```python
raise ValueError(
                f"{datasets_dtype} is not a validly formatted string representation of a pyarrow timestamp."
                f"Examples include timestamp[us] or timestamp[us, tz=America/New_York]"
                f"See: https://arrow.apache.org/docs/python/generated/pyarrow.timestamp.html#pyarrow.timestamp"
            )
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.11.0
- Platform: Ubuntu 20.04
- Python version: 3.8
- PyArrow version: 5.0.0

Fixed by #2847.


always requiring the username in the dataset name when there is one
Me and now another person have been bitten by the `datasets`'s non-strictness on requiring a dataset creator's username when it's due.

So both of us started with `stas/openwebtext-10k`, somewhere along the lines lost `stas/` and continued using `openwebtext-10k` and it all was good until we published the software and things broke, since there is no `openwebtext-10k`

So this feature request is asking to tighten the checking and not allow dataset loading if it was downloaded with the user prefix, but then attempted to be used w/o it.

The same in code:

```
# first run
python -c "from datasets import load_dataset; load_dataset('stas/openwebtext-10k')"
# now run immediately
python -c "from datasets import load_dataset; load_dataset('openwebtext-10k')"
# the second command should fail, but it doesn't fail now.
```

Please let me know if I explained myself clearly.

Thank you!
From what I can understand, you want the saved arrow file directory to have username as well instead of just dataset name if it was downloaded with the user prefix?


always requiring the username in the dataset name when there is one
Me and now another person have been bitten by the `datasets`'s non-strictness on requiring a dataset creator's username when it's due.

So both of us started with `stas/openwebtext-10k`, somewhere along the lines lost `stas/` and continued using `openwebtext-10k` and it all was good until we published the software and things broke, since there is no `openwebtext-10k`

So this feature request is asking to tighten the checking and not allow dataset loading if it was downloaded with the user prefix, but then attempted to be used w/o it.

The same in code:

```
# first run
python -c "from datasets import load_dataset; load_dataset('stas/openwebtext-10k')"
# now run immediately
python -c "from datasets import load_dataset; load_dataset('openwebtext-10k')"
# the second command should fail, but it doesn't fail now.
```

Please let me know if I explained myself clearly.

Thank you!
I don't think the user cares of how this is done, but the 2nd command should fail, IMHO, as its dataset name is invalid:
```
# first run
python -c "from datasets import load_dataset; load_dataset('stas/openwebtext-10k')"
# now run immediately
python -c "from datasets import load_dataset; load_dataset('openwebtext-10k')"
# the second command should fail, but it doesn't fail now.
```

Moreover, if someone were to create `openwebtext-10k` w/o the prefix, they will now get the wrong dataset, if they previously downloaded `stas/openwebtext-10k`.

And if there are 2 users with the same dataset name `foo/ds` and `bar/ds` - currently this won't work to get the correct dataset.

So really there 3 unrelated issues hiding in the current behavior.