KRadim/github-issues-minimal huggingface dataset:
`SystemError 15` thrown in `Dataset.__del__` when using `Dataset.map()` with `num_proc>1`
State: closed
Labels: name: bug

## Describe the bug
I use `datasets.map` to preprocess some data in my application. The error `SystemError 15` is thrown at the end of the execution of `Dataset.map()` (only with `num_proc>1`. Traceback included bellow. 
 
The exception is raised only when the code runs within a specific context. Despite ~10h spent investigating this issue, I have failed to isolate the bug, so let me describe my setup. 

In my project, `Dataset` is wrapped into a `LightningDataModule` and the data is preprocessed when calling `LightningDataModule.setup()`. Calling `.setup()` in an isolated script works fine (even when wrapped with `hydra.main()`). However, when calling `.setup()` within the experiment script (depends on `pytorch_lightning`), the script crashes and `SystemError 15`.

I could avoid throwing this error by modifying ` Dataset.__del__()` (see bellow), but I believe this only moves the problem somewhere else. I am completely stuck with this issue, any hint would be welcome. 

```python
class Dataset()
    ...
    def __del__(self):
        if hasattr(self, "_data"):
            _ = self._data # <- ugly trick that allows avoiding the issue.
            del self._data
        if hasattr(self, "_indices"):
            del self._indices
```

## Steps to reproduce the bug
```python
# Unfortunately I couldn't isolate the bug.
```

## Expected results
Calling `Dataset.map()` without throwing an exception. Or at least raising a more detailed exception/traceback.

## Actual results
```
Exception ignored in: <function Dataset.__del__ at 0x7f7cec179160>███████████████████████████████████████████████████| 5/5 [00:05<00:00,  1.17ba/s]
Traceback (most recent call last):
  File ".../python3.8/site-packages/datasets/arrow_dataset.py", line 906, in __del__
    del self._data
  File ".../python3.8/site-packages/ray/worker.py", line 1033, in sigterm_handler
    sys.exit(signum)
SystemExit: 15

```

## Environment info

Tested on 2 environments:

**Environment 1.**
- `datasets` version: 1.14.0
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.8.8
- PyArrow version: 6.0.0

**Environment 2.**
- `datasets` version: 1.14.0
- Platform: Linux-4.18.0-305.19.1.el8_4.x86_64-x86_64-with-glibc2.28
- Python version: 3.9.7
- PyArrow version: 6.0.0
Raise exceptions instead of using assertions for control flow
State: closed
Labels: name: good first issue

Motivated by https://github.com/huggingface/transformers/issues/12789 in Transformers, one welcoming change would be replacing assertions with proper exceptions. The only type of assertions we should keep are those used as sanity checks.

Currently, there is a total of 87 files with the `assert` statements (located under `datasets` and `src/datasets`), so when working on this, to manage the PR size, only modify 4-5 files at most before submitting a PR.
OpenSLR/83 is empty
State: closed
Labels: name: bug

## Describe the bug
As the summary says, openslr / SLR83 / train is empty.

The dataset returned after loading indicates there are **zero** rows. The correct number should be **17877**.
## Steps to reproduce the bug
```python
import datasets


datasets.load_dataset('openslr', 'SLR83')
```

## Expected results
```
DatasetDict({
    train: Dataset({
        features: ['path', 'audio', 'sentence'],
        num_rows: 17877
    })
})
```
## Actual results
```
DatasetDict({
    train: Dataset({
        features: ['path', 'audio', 'sentence'],
        num_rows: 0
    })
})
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.14.1.dev0 (master HEAD)
- Platform: Ubuntu 20.04
- Python version: 3.7.10
- PyArrow version: 3.0.0
bookcorpusopen no longer works
State: closed
Labels: name: bug

## Describe the bug

When using the latest version of datasets (1.14.0), I cannot use the `bookcorpusopen` dataset. The process blocks always around `9924 examples [00:06, 1439.61 examples/s]` when preparing the dataset. I also noticed that after half an hour the process is automatically killed because of the RAM usage (the machine has 1TB of RAM...).

This did not happen with 1.4.1.
I tried also `rm -rf ~/.cache/huggingface` but did not help.
Changing python version between 3.7, 3.8 and 3.9 did not help too.

## Steps to reproduce the bug
```python
import datasets
d = datasets.load_dataset('bookcorpusopen')
```

## Expected results
A clear and concise description of the expected results.

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.14.0
- Platform: Linux-5.4.0-1054-aws-x86_64-with-glibc2.27
- Python version: 3.9.7
- PyArrow version: 4.0.1
Add raw data files to the Hub with GitHub LFS for canonical dataset
State: closed

I'm interested in sharing the CaseHOLD dataset (https://arxiv.org/abs/2104.08671) as a canonical dataset on the HuggingFace Hub and would like to add the raw data files to the Hub with GitHub LFS, since it seems like a more sustainable long term storage solution, compared to other storage solutions available to my team. From what I can tell, this option is not immediately supported if one follows the sharing steps detailed here: [https://huggingface.co/docs/datasets/share_dataset.html#sharing-a-canonical-dataset](https://huggingface.co/docs/datasets/share_dataset.html#sharing-a-canonical-dataset), since GitHub LFS is not supported for public forks. Is there a way to request this? Thanks!
Add Image feature
State: closed

Adds the Image feature. This feature is heavily inspired by the recently added Audio feature (#2324). Currently, this PR is pretty simple.

Some considerations that need further discussion:
* I've decided to use `Pillow`/`PIL` as the image decoding library. Another candidate I considered is `torchvision`, mostly because of its `accimage` backend, which should be faster for loading `jpeg` images than `Pillow`. However, `torchvision`'s io module only supports png and jpeg images, has `torch` as a hard dependency, and requires magic to work with image bytes ( `torch.ByteTensor(torch.ByteStorage.from_buffer(image_bytes)))`).
* Currently, I'm converting `PIL`'s `Image` type to `np.ndarray`. The vision models in Transformers such as ViT prefer the raw `Image` type and not the decoded tensors, so there is a small overhead due to [this conversion](https://github.com/huggingface/transformers/blob/3e8761ab8077e3bb243fe2f78b2a682bd2257cf1/src/transformers/image_utils.py#L62-L73). IMO this is justified to keep this part aligned with the Audio feature, which also returns `np.ndarray`. What do you think?
* Still have to work on the channel decoding logic:
    * PyTorch prefers the channel-first ordering (C, H, W); TF and Flax the channel-last ordering (H, W, C). One cool feature would be adjusting the channel order based on the selected formatter (`torch`, `tf`, `jax`). 
    * By default, `Image.open` returns images of shape (H, W, C). However, ViT's feature extractor expects the format (C, H, W) if the image is passed as an array (explained [here](https://huggingface.co/transformers/model_doc/vit.html#transformers.ViTFeatureExtractor.__call__)), so I'm more inclined to the format (C, H, W). Which one do you prefer, (C, H, W) or (H, W, C)?
* Are there any options you'd like to see? (the user could change those via `cast_column`, such as `sampling_rate` in the Audio feature)


TODOs:
* [x] tests
* in subsequent PRs:
  * docs - a section in the docs, which gives some additional info on the Image and Audio feature and compares them to 
  `ArrayND`  
  * streaming (waiting for #3129 and #3133 to get merged first)
  * update the image tasks and the datasets to use the new feature
  * Image/Audio formatting

[Colab Notebook](https://colab.research.google.com/drive/1mIrTnqTVkWLJWoBzT1ABSe-LFelIep1c?usp=sharing) where you can play with this feature.

I'm also adding a link to the [Image](https://github.com/tensorflow/datasets/blob/7ac7d506488d46038a5854961d068926b3f93c7f/tensorflow_datasets/core/features/image_feature.py#L155) feature in TFDS because one of our goals is to parse TFDS scripts eventually, so our Image feature has to (at least) support all the formats theirs does.
Feel free to cc anyone who might be interested.

P.S. Please ignore the changes in the `datasets/**/*.py` files 😄.
`datasets-cli test` should work with datasets without scripts
State: open
Labels: name: enhancement

It would be really useful to be able to run `datasets-cli test`for datasets that don't have scripts attached  to them (whether the datasets are private or not).

I wasn't able to run the script for a private test dataset that I had created on the hub (https://huggingface.co/datasets/huggingface/DataMeasurementsTest/tree/main) -- although @lhoestq came to save the day!
Fix string encoding for Value type
State: closed

Some metrics have `string` features but currently it fails if users pass integers instead. Indeed feature encoding that handles the conversion of the user's objects to the right python type is missing a case for `string`, while it already works as expected for integers, floats and booleans

Here is an example code that didn't work previously, but that works with this fix:
```python
import datasets

# Note that 'id' is an integer while the SQuAD metric uses strings
predictions = [{'prediction_text': '1976', 'id': 5}]
references = [{'answers': {'answer_start': [97], 'text': ['1976']}, 'id': 5}] 

squad_metric = datasets.load_metric("squad") 
squad_metric.add_batch(predictions=predictions, references=references) 
results = squad_metric.compute()
# {'exact_match': 100.0, 'f1': 100.0}
```

cc @sgugger @philschmid
Illegal instruction (core dumped) at datasets import
State: closed
Labels: name: bug

## Describe the bug
I install datasets using conda and when I import datasets I get: "Illegal instruction (core dumped)"

## Steps to reproduce the bug

```
conda create --prefix path/to/env
conda activate path/to/env
conda install -c huggingface -c conda-forge datasets
# exits with output "Illegal instruction (core dumped)"
python -m datasets
```

## Environment info

When I run "datasets-cli env", I also get "Illegal instruction (core dumped)"

If I run the following commands:

```
conda create --prefix path/to/another/new/env
conda activate path/to/another/new/env
conda install -c huggingface transformers
transformers-cli env
```

Then I get:

- `transformers` version: 4.11.3
- Platform: Linux-5.4.0-67-generic-x86_64-with-glibc2.17
- Python version: 3.8.12
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

Let me know what additional information you need in order to debug this issue. Thanks in advance!
Sacrebleu unexpected behaviour/requirement for data format
State: closed
Labels: name: bug

## Describe the bug
When comparing with the original `sacrebleu` implementation, the `datasets` implementation does some strange things that I do not quite understand. This issue was triggered when I was trying to implement TER and found the datasets implementation of BLEU [here](https://github.com/huggingface/datasets/pull/3153).

In the below snippet, the original sacrebleu snippet works just fine whereas the datasets implementation throws an error.

## Steps to reproduce the bug
```python
import sacrebleu
import datasets

refs = [
    ['The dog bit the man.', 'It was not unexpected.', 'The man bit him first.'],
    ['The dog had bit the man.', 'No one was surprised.', 'The man had bitten the dog.'],
]

hyps = ['The dog bit the man.', "It wasn't surprising.", 'The man had just bitten him.']

expected_bleu = 48.530827

ds_bleu = datasets.load_metric("sacrebleu")

bleu_score_sb = sacrebleu.corpus_bleu(hyps, refs).score
print(bleu_score_sb, expected_bleu)
# works: 48.5308...
bleu_score_ds = ds_bleu.compute(predictions=hyps, references=refs)["score"]
print(bleu_score_ds, expected_bleu)
# ValueError: Predictions and/or references don't match the expected format.
```
This seems to be related to how datasets forces the features format here:

https://github.com/huggingface/datasets/blob/87c71b9c29a40958973004910f97e4892559dfed/metrics/sacrebleu/sacrebleu.py#L94-L99

and then manipulates the references during the compute stage here

https://github.com/huggingface/datasets/blob/87c71b9c29a40958973004910f97e4892559dfed/metrics/sacrebleu/sacrebleu.py#L119-L122

I do not quite understand why that is required since sacrebleu handles argument parsing quite well [by itself](https://github.com/mjpost/sacrebleu/blob/2787185dd0f8d224c72ee5a831d163c2ac711a47/sacrebleu/metrics/base.py#L229). 
## Actual results
Traceback (most recent call last):
  File "C:\Users\bramv\AppData\Roaming\JetBrains\PyCharm2020.3\scratches\scratch_23.py", line 23, in <module>
    bleu_score_ds = ds_bleu.compute(predictions=hyps, references=refs)["score"]
  File "C:\dev\python\datasets\src\datasets\metric.py", line 392, in compute
    self.add_batch(predictions=predictions, references=references)
  File "C:\dev\python\datasets\src\datasets\metric.py", line 439, in add_batch
    raise ValueError(
ValueError: Predictions and/or references don't match the expected format.
Expected format: {'predictions': Value(dtype='string', id='sequence'), 'references': Sequence(feature=Value(dtype='string', id='sequence'), length=-1, id='references')},
Input predictions: ['The dog bit the man.', "It wasn't surprising.", 'The man had just bitten him.'],
Input references: [['The dog bit the man.', 'It was not unexpected.', 'The man bit him first.'], ['The dog had bit the man.', 'No one was surprised.', 'The man had bitten the dog.']]

## Environment info
- `datasets` version: 1.14.1.dev0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.9.2
- PyArrow version: 4.0.1
Add TER (as implemented in sacrebleu)
State: closed

Implements TER (Translation Edit Rate) as per its implementation in sacrebleu. Sacrebleu for BLEU scores is already implemented in `datasets` so I thought this would be a nice addition.

I started from the sacrebleu implementation, as the two metrics have a lot in common.

Verified with sacrebleu's [testing suite](https://github.com/mjpost/sacrebleu/blob/078c440168c6adc89ba75fe6d63f0d922d42bcfe/test/test_ter.py) that this indeed works as intended.

```python
import datasets


test_cases = [
    (['aaaa bbbb cccc dddd'], ['aaaa bbbb cccc dddd'], 0),  # perfect match
    (['dddd eeee ffff'], ['aaaa bbbb cccc'], 1),  # no overlap
    ([''], ['a'], 1),  # corner case, empty hypothesis
    (['d e f g h a b c'], ['a b c d e f g h'], 1 / 8),  # a single shift fixes MT
    (
        [
            'wählen Sie " Bild neu berechnen , " um beim Ändern der Bildgröße Pixel hinzuzufügen oder zu entfernen , damit das Bild ungefähr dieselbe Größe aufweist wie die andere Größe .',
            'wenn Sie alle Aufgaben im aktuellen Dokument aktualisieren möchten , wählen Sie im Menü des Aufgabenbedienfelds die Option " Alle Aufgaben aktualisieren . "',
            'klicken Sie auf der Registerkarte " Optionen " auf die Schaltfläche " Benutzerdefiniert " und geben Sie Werte für " Fehlerkorrektur-Level " und " Y / X-Verhältnis " ein .',
            'Sie können beispielsweise ein Dokument erstellen , das ein Auto über die Bühne enthält .',
            'wählen Sie im Dialogfeld " Neu aus Vorlage " eine Vorlage aus und klicken Sie auf " Neu . "',
        ],
        [
            'wählen Sie " Bild neu berechnen , " um beim Ändern der Bildgröße Pixel hinzuzufügen oder zu entfernen , damit die Darstellung des Bildes in einer anderen Größe beibehalten wird .',
            'wenn Sie alle Aufgaben im aktuellen Dokument aktualisieren möchten , wählen Sie im Menü des Aufgabenbedienfelds die Option " Alle Aufgaben aktualisieren . "',
            'klicken Sie auf der Registerkarte " Optionen " auf die Schaltfläche " Benutzerdefiniert " und geben Sie für " Fehlerkorrektur-Level " und " Y / X-Verhältnis " niedrigere Werte ein .',
            'Sie können beispielsweise ein Dokument erstellen , das ein Auto enthalt , das sich über die Bühne bewegt .',
            'wählen Sie im Dialogfeld " Neu aus Vorlage " eine Vorlage aus und klicken Sie auf " Neu . "',
        ],
        0.136  # realistic example from WMT dev data (2019)
    ),
]

ter = datasets.load_metric(r"path\to\datasets\metrics\ter")

predictions = ["hello there general kenobi", "foo bar foobar"]
references = [["hello there general kenobi", "hello there !"], ["foo bar foobar", "foo bar foobar"]]
print(ter.compute(predictions=predictions, references=references))

for hyp, ref, score in test_cases:
    # Note the reference transformation which is different from scarebleu's input format
    results = ter.compute(predictions=hyp, references=[[r] for r in ref])
    assert 100*score == results["score"], f"expected {100*score}, got {results['score']}"
```
Re-add faiss to windows testing suite
State: closed

In recent versions, `faiss-cpu` seems to be available for Windows as well. See the [PyPi page](https://pypi.org/project/faiss-cpu/#files) to confirm. We can therefore included it for Windows in the setup file.

At first tests didn't pass due to problems with permissions as caused by `NamedTemporaryFile` on Windows. This built-in library is notoriously poor in playing nice on Windows. The required change isn't pretty, but it works. First set `delete=False` to not automatically try to delete the file on `exit`. Then, manually delete the file with `unlink`. It's weird, I know, but it works.

```python
with tempfile.NamedTemporaryFile(delete=False) as tmp_file:
    # do stuff
os.unlink(tmp_file.name)
```

closes #3150
Faiss _is_ available on Windows
State: closed

In the setup file, I find the following:

https://github.com/huggingface/datasets/blob/87c71b9c29a40958973004910f97e4892559dfed/setup.py#L171

However, FAISS does install perfectly fine on Windows on my system. You can also confirm this on the [PyPi page](https://pypi.org/project/faiss-cpu/#files), where Windows wheels are available. Maybe this was true for older versions? For current versions, this can be removed I think.

(This isn't really a bug but didn't know how else to tag.)

If you agree I can do a quick PR and remove that line.
Add CMU Hinglish DoG Dataset for MT
State: closed

Address part of #2841 

Added the CMU Hinglish DoG Dataset as in GLUECoS. Added it as a seperate dataset as unlike other tasks of GLUE CoS this can't be evaluated for a BERT like model. 
Consists of parallel dataset between Hinglish (Hindi-English) and English, can be used for Machine Translation between the two. 

The data processing part is inspired from the GLUECoS repo [here](https://github.com/microsoft/GLUECoS/blob/7fdc51653e37a32aee17505c47b7d1da364fa77e/Data/Preprocess_Scripts/preprocess_mt_en_hi.py)
The dummy data part is not working properly, it shows 
``` UnboundLocalError: local variable 'generator_splits' referenced before assignment ``` 
when I run without ``--auto_generate``.

Please let me know how I can fix that.
Thanks
Streaming with num_workers != 0
State: closed
Labels: name: bug

## Describe the bug
When using dataset streaming with pytorch DataLoader, the setting num_workers to anything other than 0 causes the code to freeze forever before yielding the first batch.

The code owner is likely @lhoestq 

## Steps to reproduce the bug

For your convenience, we've prepped a colab notebook that reproduces the bug
https://colab.research.google.com/drive/1Mgl0oTZSNIE3UeGl_oX9wPCOIxRg19h1?usp=sharing
```python
!pip install datasets==1.14.0

should_freeze_forever = True
# ^-- set this to True in order to freeze forever, set to False in order to work normally

import torch
from datasets import load_dataset

data = load_dataset("oscar", "unshuffled_deduplicated_bn", split="train", streaming=True)
data = data.map(lambda x: {"text": x["text"], "orig": f"oscar[{x['id']}]"}, batched=True)
data = data.shuffle(100, seed=1337)

data = data.with_format("torch")
loader = torch.utils.data.DataLoader(data, batch_size=2, num_workers=2 if should_freeze_forever else 0)

# v-- the code should freeze forever at this line
for i, row in enumerate(loader):
  print(row)
  if i > 10: break
print("DONE!")
```

## Expected results
The code should not freeze forever with num_workers=2

## Actual results
The code freezes forever with num_workers=2

## Environment info
- `datasets` version: 1.14.0 (also found in previous versions)
- Platform: google colab (also locally)
- Python version: 3.7, (also 3.8)
- PyArrow version: 3.0.0
CLI test command throws NonMatchingSplitsSizesError when saving infos
State: closed
Labels: name: bug

When trying to generate a datset JSON metadata, a `NonMatchingSplitsSizesError` is thrown:
```
$ datasets-cli test datasets/arabic_billion_words --save_infos --all_configs
Testing builder 'Alittihad' (1/10)
Downloading and preparing dataset arabic_billion_words/Alittihad (download: 332.13 MiB, generated: Unknown size, post-processed: Unknown size, total: 332.13 MiB) to .cache\arabic_billion_words\Alittihad\1.1.0\8175ff1c9714c6d5d15b1141b6042e5edf048276bb81a9c14e35e149a7a62ae4...
Traceback (most recent call last):
  File "path\huggingface\datasets\.venv\Scripts\datasets-cli-script.py", line 33, in <module>
    sys.exit(load_entry_point('datasets', 'console_scripts', 'datasets-cli')())
  File "path\huggingface\datasets\src\datasets\commands\datasets_cli.py", line 33, in main
    service.run()
  File "path\huggingface\datasets\src\datasets\commands\test.py", line 144, in run
    builder.download_and_prepare(
  File "path\huggingface\datasets\src\datasets\builder.py", line 607, in download_and_prepare
    self._download_and_prepare(
  File "path\huggingface\datasets\src\datasets\builder.py", line 709, in _download_and_prepare
    verify_splits(self.info.splits, split_dict)
  File "path\huggingface\datasets\src\datasets\utils\info_utils.py", line 74, in verify_splits
    raise NonMatchingSplitsSizesError(str(bad_splits))
datasets.utils.info_utils.NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='train', num_bytes=0, num_examples=0, dataset_name='arabic_billion_words'), 'recorded': SplitInfo(name='train', num_bytes=1601790302, num_examples=349342, dataset_name='arabic_billion_words')}]
```

This is due because a previous run generated a wrong `dataset_info.json`.

This error can be avoided by passing `--ignore_verifications`, but I think this should be assumed when passing `--save_infos`.
[when Image type will exist] provide a way to get the data as binary + filename
State: closed
Labels: name: enhancement, name: dataset-viewer

**Is your feature request related to a problem? Please describe.**

When a dataset cell contains a value of type Image (be it from a remote URL, an Array2D/3D, or any other way to represent images), I want to be able to write the image to the disk, with the correct filename, and optionally to know its mimetype, in order to serve it on the web.

Note: this issue would apply exactly the same for the `Audio` type.

**Describe the solution you'd like**

If a "cell" has the type `Image`, provide a way to get the binary content of the file, and the filename, eg as:

```python
  filename: str
  data: bytes
```

**Describe alternatives you've considered**

A way to write the cell to the disk (passing a local directory), and then return the pathname, filename, and mimetype.
Infer the features if missing
State: closed
Labels: name: enhancement, name: dataset-viewer

**Is your feature request related to a problem? Please describe.**

Some datasets, in particular community datasets, have no info file, thus no features.

**Describe the solution you'd like**

If a dataset has no features, the first loaded data (5-10 rows) could be used to infer the type.

Related: `datasets` would provide a way to load the data, and get the rows AND the features as the result.

**Describe alternatives you've considered**

The HF hub could also provide some UI to help the dataset maintainers to explicit the types of their rows, or automatically infer them as an initial proposal.
Provide a way to check if the features (in info) match with the data of a split
State: open
Labels: name: enhancement, name: dataset-viewer

**Is your feature request related to a problem? Please describe.**

I understand that currently the data loaded has not always the type described in the info features

**Describe the solution you'd like**

Provide a way to check if the rows have the type described by info features

**Describe alternatives you've considered**

Always check it, and raise an error when loading the data if their type doesn't match the features.
Provide a way to write a streamed dataset to the disk
State: open
Labels: name: enhancement, name: dataset-viewer

**Is your feature request related to a problem? Please describe.**

The streaming mode allows to get the 100 first rows of a dataset very quickly. But it does not cache the answer, so a posterior call to get the same 100 rows will send a request to the server again and again.

**Describe the solution you'd like**

Provide a way to write the streamed rows of a dataset on the disk, and to load from it later.

**Describe alternatives you've considered**

Provide a third mode: `lazy`, which would use the local cache for the data that have already been fetched previously, and use streaming to get the rest of the requested data.
Fix file/directory deletion on Windows  
State: open
Labels: name: bug

Currently, on Windows, some attempts to delete a dataset file/directory will fail with the `PerimissionError`. 

Examples:
- download a dataset, then force redownload it in the same session while keeping a reference to the downloaded dataset
```python
from datasets import load_dataset
dset = load_dataset("sst", split="train")
dset = load_dataset("sst", split="train", download_mode="force_redownload")
```
- try to clean up the cache files while keeping a reference to those files (via the mapped dataset):
```python
from datasets import load_dataset
dset = load_dataset("sst", split="train")
dset_mapped = dset.map(lambda _: {"dummy_col": 1})
dset.cleanup_cache_files()
```
We should fix those.
More fine-grained taxonomy of error types
State: open
Labels: name: enhancement, name: dataset-viewer

**Is your feature request related to a problem? Please describe.**

Exceptions like `FileNotFoundError` can be raised by different parts of the code, and it's hard to detect which one did

**Describe the solution you'd like**

Give a specific exception type for every group of similar errors

**Describe alternatives you've considered**

Rely on the error message, using regex
Fix script of Arabic Billion Words dataset to return all data
State: closed

The script has a bug and only parses and generates a portion of the entire dataset.

This PR fixes the loading script so that is properly parses the entire dataset. 

Current implementation generates the same number of examples as reported in the [original paper](https://arxiv.org/abs/1611.04033) for all configurations except for one:
- For "Youm7" we generate more examples (1172136) than the ones reported by the paper (1025027)

|                | Number of examples | Number of examples according to the source |
|:---------------|-------------------:|-----:|
| Alittihad      |             349342 |349342 |
| Almasryalyoum  |             291723 |291723 |
| Almustaqbal    |             446873 |446873 |
| Alqabas        |             817274 |817274 |
| Echoroukonline |             139732 |139732 |
| Ryiadh         |             858188 | 858188 |
| Sabanews       |              92149 |92149 |
| SaudiYoum      |             888068 |888068 |
| Techreen       |             314597 |314597 |
| Youm7          |            1172136 |1025027 |

Fix #3126.
Make inspect.get_dataset_config_names always return a non-empty list of configs
State: closed
Labels: name: enhancement, name: dataset-viewer

**Is your feature request related to a problem? Please describe.**

Currently, some datasets have a configuration, while others don't. It would be simpler for the user to always have configuration names to refer to

**Describe the solution you'd like**

In that sense inspect.get_dataset_config_names should always return at least one configuration name, be it `default` or `Check___region_1` (for community datasets like `Check/region_1`).

https://github.com/huggingface/datasets/blob/c5747a5e1dde2670b7f2ca6e79e2ffd99dff85af/src/datasets/inspect.py#L161
Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.11.0/metrics/rouge/rouge.py
State: closed
Labels: name: bug

datasets version: 1.12.1

`metric = datasets.load_metric('rouge')`

The error:

> ConnectionError                           Traceback (most recent call last)
> <ipython-input-3-dd10a0c5212f> in <module>
> ----> 1 metric = datasets.load_metric('rouge')
> 
> /usr/local/lib/python3.6/dist-packages/datasets/load.py in load_metric(path, config_name, process_id, num_process, cache_dir, experiment_id, keep_in_memory, download_config, download_mode, script_version, **metric_init_kwargs)
>     613         download_config=download_config,
>     614         download_mode=download_mode,
> --> 615         dataset=False,
>     616     )
>     617     metric_cls = import_main_class(module_path, dataset=False)
> 
> /usr/local/lib/python3.6/dist-packages/datasets/load.py in prepare_module(path, script_version, download_config, download_mode, dataset, force_local_path, dynamic_modules_path, return_resolved_file_path, **download_kwargs)
>     328                 file_path = hf_github_url(path=path, name=name, dataset=dataset, version=script_version)
>     329                 try:
> --> 330                     local_path = cached_path(file_path, download_config=download_config)
>     331                 except FileNotFoundError:
>     332                     if script_version is not None:
> 
> /usr/local/lib/python3.6/dist-packages/datasets/utils/file_utils.py in cached_path(url_or_filename, download_config, **download_kwargs)
>     296             use_etag=download_config.use_etag,
>     297             max_retries=download_config.max_retries,
> --> 298             use_auth_token=download_config.use_auth_token,
>     299         )
>     300     elif os.path.exists(url_or_filename):
> 
> /usr/local/lib/python3.6/dist-packages/datasets/utils/file_utils.py in get_from_cache(url, cache_dir, force_download, proxies, etag_timeout, resume_download, user_agent, local_files_only, use_etag, max_retries, use_auth_token)
>     603             raise FileNotFoundError("Couldn't find file at {}".format(url))
>     604         _raise_if_offline_mode_is_enabled(f"Tried to reach {url}")
> --> 605         raise ConnectionError("Couldn't reach {}".format(url))
>     606
>     607     # Try a second time
> 
> ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.11.0/metrics/rouge/rouge.py


Is there any remedy to solve the connection issue ?
Add ADE20k
State: closed
Labels: name: dataset request, name: vision

## Adding a Dataset
- **Name:** ADE20k (actually it's called the MIT Scene Parsing Benchmark, it's actually a subset of ADE20k but a lot of authors still call it ADE20k)
- **Description:** A semantic segmentation dataset, consisting of 150 classes.
- **Paper:** http://people.csail.mit.edu/bzhou/publication/scene-parse-camera-ready.pdf
- **Data:** http://sceneparsing.csail.mit.edu/
- **Motivation:** I am currently adding Transformer-based semantic segmentation models that achieve SOTA on this dataset. It would be great to directly access this dataset using HuggingFace Datasets, in order to make example scripts in HuggingFace Transformers.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).
datasets-cli: convertion of a tfds dataset to a huggingface one.
State: open

### Discussed in https://github.com/huggingface/datasets/discussions/3079

<div type='discussions-op-text'>

<sup>Originally posted by **vitalyshalumov** October 14, 2021</sup>
I'm trying to convert a tfds dataset to a huggingface one.

I've tried:

1.       datasets-cli convert  --tfds_path ~/tensorflow_datasets/mnist/3.0.1/ --datasets_directory ~/.cache/huggingface/datasets/mnist/3.0.1/

2.       datasets-cli convert  --tfds_path ~/tensorflow_datasets/mnist/3.0.1/ --datasets_directory ~/.cache/huggingface/datasets/


and other permutations.
The script appears to be running and finishing without an error but when looking in the  huggingface/datasets/ folder nothing is created.


</div>
"arabic_billion_words" dataset does not create the full dataset
State: closed
Labels: name: bug

## Describe the bug
When running: 
raw_dataset = load_dataset('arabic_billion_words','Alittihad')
the correct dataset file is pulled from the url.
But, the generated dataset includes just a small portion of the data included in the file.
This is true for all other portions of the "arabic_billion_words" dataset ('Almasryalyoum',.....)

## Steps to reproduce the bug
```python
# Sample code to reproduce the bug
raw_dataset = load_dataset('arabic_billion_words','Alittihad')

#The screen message
Downloading and preparing dataset arabic_billion_words/Alittihad (download: 332.13 MiB, generated: 20.62 MiB, post-processed: Unknown size, total: 352.74 MiB) 

## Expected results
over 100K sentences

## Actual results
only 11K sentences

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.14.0
- Platform: Linux-5.8.0-63-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyArrow version: 4.0.1
More efficient nested features encoding
State: closed

Nested encoding of features wastes a lot of time on operations which are effectively doing nothing when lists are used.
For example, if in the input we have a list of integers, `encoded_nested_example` will iterate over it and apply `encoded_nested_example` on every element even though it just return the int as is.

A similar issue is handled at an earlier stage when casting pytorch/tensorflow/pandas objects to python lists/numpy arrays:
https://github.com/huggingface/datasets/blob/c98c23c4260edadab00f997d1a5d66b7f2e93ce9/src/datasets/features/features.py#L149-L156
https://github.com/huggingface/datasets/blob/c98c23c4260edadab00f997d1a5d66b7f2e93ce9/src/datasets/features/features.py#L212-L228

In this pull request I suggest to use the same approach in `encoded_nested_example`.
In my setup there was a major speedup with this change: loading the data was at least x4 faster.
Segmentation fault when loading datasets from file
State: closed
Labels: name: bug

## Describe the bug
Custom dataset loading sometimes segfaults and kills the process if chunks contain a variety of features/

## Steps to reproduce the bug

Download an example file:
```
wget https://gist.githubusercontent.com/TevenLeScao/11e2184394b3fa47d693de2550942c6b/raw/4232704d08fbfcaf93e5b51def9e5051507651ad/tiny_kelm.jsonl
```
Then in Python:
```
import datasets
tiny_kelm = datasets.load_dataset("json", data_files="tiny_kelm.jsonl", chunksize=100000)
```

## Expected results
a `tiny_kelm` functional dataset

## Actual results
☠️ `Segmentation fault (core dumped)` ☠️

## Environment info
- `datasets` version: 1.14.0
- Platform: Linux-5.11.0-38-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyArrow version: 5.0.0
OSError with a custom dataset loading script
State: closed
Labels: name: bug

## Describe the bug
I am getting an OS error when trying to load the newly uploaded dataset classla/janes_tag. What puzzles me is that I have already uploaded a very similar dataset - classla/reldi_hr - with no issues. The loading scripts for the two datasets are almost identical and they have the same directory structure, yet I am only getting an error with janes_tag.

## Steps to reproduce the bug
```python
dataset = datasets.load_dataset('classla/janes_tag', split='validation')
```

## Expected results
Dataset correctly loaded.

## Actual results

Traceback (most recent call last):
  File "C:/mypath/test.py", line 91, in <module>
    load_and_print('janes_tag')
  File "C:/mypath/test.py", line 32, in load_and_print
    dataset = datasets.load_dataset('classla/{}'.format(ds_name), split='validation')
  File "C:\mypath\venv\lib\site-packages\datasets\load.py", line 1632, in load_dataset
    use_auth_token=use_auth_token,
  File "C:\mypath\venv\lib\site-packages\datasets\builder.py", line 608, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "C:\mypath\venv\lib\site-packages\datasets\builder.py", line 704, in _download_and_prepare
    ) from None
OSError: Cannot find data file. 
Original error:
[Errno 2] No such file or directory: 'C:\\mypath\\.cache\\huggingface\\datasets\\downloads\\2c9996e44bdc5af9c89bffb9e6d7a3e42fdb2f56bacab45de13b20f3032ea7ca\\data\\train_all.conllup'

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.14.0
- Platform: Windows-10-10.0.19041-SP0
- Python version: 3.7.5
- PyArrow version: 3.0.0
Use huggingface_hub.HfApi to list datasets/metrics
State: closed

Delete `datasets.inspect.HfApi` and use `huggingface_hub.HfApi` instead.

WIP until https://github.com/huggingface/huggingface_hub/pull/429 is merged, then wait for the new release of  `huggingface_hub`, update the `huggingface_hub` version in `setup.py` and merge this PR.

cc: @lhoestq
Correctly update metadata to preserve features when concatenating datasets with axis=1
State: closed

This PR correctly updates metadata to preserve higher-level feature types (e.g. `ClassLabel`) in `datasets.concatenate_datasets` when `axis=1`. Previously, we would delete the feature metadata in `datasets.concatenate_datasets` if `axis=1` and restore the feature types from the arrow table schema in `Dataset.__init__`. However, this approach only works for simple feature types (e.g. `Value`).

Fixes #3111
Add OpenSLR 83 - Crowdsourced high-quality UK and Ireland English Dialect speech
State: closed
Labels: name: dataset request

## Adding a Dataset
- **Name:** *openslr**
- **Description:** *Data set which contains male and female recordings of English from various dialects of the UK and Ireland.*
- **Paper:** *https://www.openslr.org/resources/83/about.html*
- **Data:** *Eleven separate data files can be found via https://www.openslr.org/resources/83/*
- **Motivation:** *Increase english ASR data with UK and Irish dialects*

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

The *openslr* dataset already exists, this will add additional subset, *SLR83*.
CI error at each release commit
State: closed
Labels: name: bug

After 1.12.0, there is a recurrent CI error at each release commit: https://app.circleci.com/pipelines/github/huggingface/datasets/8289/workflows/665d954d-e409-4602-8202-e678594d2946/jobs/51110

```
____________________ LoadTest.test_load_dataset_canonical _____________________
[gw0] win32 -- Python 3.6.8 C:\tools\miniconda3\python.exe

self = <tests.test_load.LoadTest testMethod=test_load_dataset_canonical>

    def test_load_dataset_canonical(self):
        scripts_version = os.getenv("HF_SCRIPTS_VERSION", SCRIPTS_VERSION)
        with self.assertRaises(FileNotFoundError) as context:
            datasets.load_dataset("_dummy")
        self.assertIn(
            f"https://raw.githubusercontent.com/huggingface/datasets/{scripts_version}/datasets/_dummy/_dummy.py",
>           str(context.exception),
        )
E       AssertionError: 'https://raw.githubusercontent.com/huggingface/datasets/1.14.0/datasets/_dummy/_dummy.py' not found in "Couldn't find a dataset script at C:\\Users\\circleci\\datasets\\_dummy\\_dummy.py or any data file in the same directory. Couldn't find '_dummy' on the Hugging Face Hub either: FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/huggingface/datasets/master/datasets/_dummy/_dummy.py"

tests\test_load.py:358: AssertionError
```
Update doc links to point to new docs
State: closed
Labels: name: documentation

This PR:
* updates the README links and the ADD_NEW_DATASET template to point to the new docs (the new docs don't have a section with the list of all the possible features, so I added that info to the `Features` docstring, which is then referenced in the ADD_NEW_DATASET template)
* fixes some broken links in the `.rst` files (fixed with the `make linkcheck` tool)
Loading Data from HDF files
State: open
Labels: name: enhancement, name: good second issue

**Is your feature request related to a problem? Please describe.**
More often than not I come along big HDF datasets, and currently there is no straight forward way to feed them to a dataset.

**Describe the solution you'd like**
I would love to see a `from_h5` method that gets an interface implemented by the user on how items are extracted from dataset (in case of multiple datasets containing elements like arrays and metadata and etc).

**Describe alternatives you've considered**
Currently I manually load hdf files using `h5py` and implement PyTorch dataset interface. For small h5 files I load them into a pandas dataframe and use `from_pandas` function in the `datasets` package to load them, but for big datasets this is not feasible.

**Additional context**
HDF files are widespread throughout different domains and are one of the go to's for many researchers/scientists/engineers who work with numerical data. Given `datasets`' usecases have outgrown NLP use cases, it will make a lot of sense focusing on things like supporting HDF files.
concatenate_datasets removes ClassLabel typing.
State: closed
Labels: name: bug

## Describe the bug

When concatenating two datasets, we lose typing of ClassLabel columns.

I can work on this if this is a legitimate bug,

## Steps to reproduce the bug
```python
import datasets
from datasets import Dataset, ClassLabel, Value, concatenate_datasets

DS_LEN = 100
my_dataset = Dataset.from_dict(
    {
        "sentence": [f"{chr(i % 10)}" for i in range(DS_LEN)],
        "label": [i % 2 for i in range(DS_LEN)]
    }
)
my_predictions = Dataset.from_dict(
    {
        "pred": [(i + 1) % 2 for i in range(DS_LEN)]
    }
)

my_dataset = my_dataset.cast(datasets.Features({"sentence": Value("string"), "label": ClassLabel(2, names=["POS", "NEG"])}))
print("Original")
print(my_dataset)
print(my_dataset.features)


concat_ds = concatenate_datasets([my_dataset, my_predictions], axis=1)
print("Concatenated")
print(concat_ds)
print(concat_ds.features)


```

## Expected results
The features of `concat_ds` should contain ClassLabel.

## Actual results

On master, I get: 
```
{'sentence': Value(dtype='string', id=None), 'label': Value(dtype='int64', id=None), 'pred': Value(dtype='int64', id=None)}
```

## Environment info
- `datasets` version: 1.14.1.dev0
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.11
- PyArrow version: 4.0.1
Add Google BLEU (aka GLEU) metric
State: closed

This PR adds the NLTK implementation of Google BLEU metric. This is also a part of an effort to resolve an unfortunate naming collision between GLEU for machine translation and GLEU for grammatical error correction.
I used [this page](https://huggingface.co/docs/datasets/add_metric.html)  for reference. Please, point me to the right direction if I missed anything.
download_mode=`force_redownload` does not work on removed datasets
State: open
Labels: name: bug, name: dataset-viewer

## Describe the bug

If a cached dataset is removed from the library, I don't see how to delete it programmatically. I thought that using `force_redownload` would try to refresh the cache, then raise an exception, but it reuses the cache instead.

## Steps to reproduce the bug

_requires to already have `wit` in the cache_: see https://github.com/huggingface/datasets/pull/2981

```python
import datasets as ds
dataset = ds.load_dataset("wit", split="train", download_mode='force_redownload')
```

## Expected results

It should raise an exception, since the dataset does not exist anymore.

## Actual results

It uses the cached result

```
Using the latest cached version of the module from /home/slesage/.cache/huggingface/modules/datasets_modules/datasets/wit/107afbffd48e058b19101bddc47fbee25fa68eb6d50a733e262875f1285a5171 (last modified on Wed Sep 29 08:21:10 2021) since it couldn't be found locally at wit, or remotely on the Hugging Face Hub.
```

## Environment info

- `datasets` version: 1.13.4.dev0
- Platform: Linux-5.11.0-1019-aws-x86_64-with-glibc2.31
- Python version: 3.9.6
- PyArrow version: 4.0.1
Replace FSTimeoutError with parent TimeoutError
State: closed

PR #3050 introduced a dependency on `fsspec.FSTiemoutError`. Note that this error only exists from `fsspec` version `2021.06.0` (June 2021).

To fix #3097, there are 2 alternatives:
- Either pinning `fsspec` to versions newer or equal to `2021.06.0`
- Or replacing `fsspec.FSTimeoutError` wth its parent `asyncio.TimeoutError`, which exists from Python 3.8.0 (Sep 2018).

This PR implements the second approach.

Fix #3097.
AttributeError: module 'huggingface_hub.hf_api' has no attribute 'DatasetInfo'
State: closed
Labels: name: bug

## Describe the bug
When using `pip install datasets`
or use `conda install -c huggingface -c conda-forge datasets`
cannot install datasets

## Steps to reproduce the bug
```python
from datasets import load_dataset

dataset = load_dataset("sst", "default")
```

## Actual results
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-5-fbe7981e6e21> in <module>
      1 import torch
      2 import transformers
----> 3 from datasets import load_dataset
      4 
      5 dataset = load_dataset("sst", "default")

~/miniforge3/envs/actor/lib/python3.8/site-packages/datasets/__init__.py in <module>
     35 from .arrow_reader import ArrowReader, ReadInstruction
     36 from .arrow_writer import ArrowWriter
---> 37 from .builder import ArrowBasedBuilder, BeamBasedBuilder, BuilderConfig, DatasetBuilder, GeneratorBasedBuilder
     38 from .combine import interleave_datasets
     39 from .dataset_dict import DatasetDict, IterableDatasetDict

~/miniforge3/envs/actor/lib/python3.8/site-packages/datasets/builder.py in <module>
     42 )
     43 from .arrow_writer import ArrowWriter, BeamWriter
---> 44 from .data_files import DataFilesDict, _sanitize_patterns
     45 from .dataset_dict import DatasetDict, IterableDatasetDict
     46 from .fingerprint import Hasher

~/miniforge3/envs/actor/lib/python3.8/site-packages/datasets/data_files.py in <module>
    118 
    119 def _exec_patterns_in_dataset_repository(
--> 120     dataset_info: huggingface_hub.hf_api.DatasetInfo,
    121     patterns: List[str],
    122     allowed_extensions: Optional[list] = None,

AttributeError: module 'huggingface_hub.hf_api' has no attribute 'DatasetInfo'



## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.13.3
- Platform: macOS-11.3.1-arm64-arm-64bit
- Python version: 3.8.10
- PyArrow version: 5.0.0