sunny2309/githug-issues huggingface dataset:
Can not correctly parse the classes with imagefolder

## Describe the bug
I try to load my own image dataset with imagefolder, but the parsing of classes is incorrect.

## Steps to reproduce the bug
I organized my dataset (ImageNet) in the following structure:
```
  - imagenet/
      - train/
          - n01440764/
              - ILSVRC2012_val_00000293.jpg
              - ...... 
          - n01695060/
          - ......
      - val/ 
          - n01440764/
          - n01695060/
          - ......
```

At first, I followed the instructions from the Huggingface [example](https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-classification#using-your-own-data) to load my data as:
```
from datasets import load_dataset
data_files = {'train': 'imagenet/train', 'val': 'imagenet/val'}
ds = load_dataset("nateraw/image-folder", data_files=data_files, task="image-classification")
```
but it resulted following error (I mask my personal path as <PERSONAL_PATH>):
```
FileNotFoundError: Unable to find 'https://huggingface.co/datasets/nateraw/image-folder/resolve/main/imagenet/train' at <PERSONAL_PATH>/ImageNet/https:/huggingface.co/datasets/nateraw/image-folder/resolve/main
```

Next, I followed a recent issue #3960 to load data as:
```
from datasets import load_dataset
data_files = {'train': ['imagenet/train/**'], 'val': ['imagenet/val/**']}
ds = load_dataset("imagefolder", data_files=data_files, task="image-classification")
```
and the data can be loaded without error as: (I copy val folder to train folder for illustration)
```
>>> ds
DatasetDict({
    train: Dataset({
        features: ['image', 'labels'],
        num_rows: 50000
    })
    val: Dataset({
        features: ['image', 'labels'],
        num_rows: 50000
    })
})
```
However, the parsed classes is wrong (should be 1000 classes):
```
>>> ds["train"].features
{'image': Image(decode=True, id=None), 'labels': ClassLabel(num_classes=1, names=['val'], id=None)}
```

## Expected results
I expect that the "labels" in ds["train"].features should contain 1000 classes.

## Actual results
The "labels" in ds["train"].features contains only 1 wrong class.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.0.0
- Platform: Ubuntu 18.04
- Python version: Python 3.7.12
- PyArrow version: 7.0.0
Cannot preview "hazal/Turkish-Biomedical-corpus-trM"

## Dataset viewer issue for '*hazal/Turkish-Biomedical-corpus-trM'

**Link:** *https://huggingface.co/datasets/hazal/Turkish-Biomedical-corpus-trM*

*I cannot see the dataset preview.*

```
Server Error

Status code:   400
Exception:     HTTPError
Message:       403 Client Error: Forbidden for url: https://huggingface.co/api/datasets/hazal/Turkish-Biomedical-corpus-trM?full=true
```


Am I the one who added this dataset ? Yes
Fix None issue with Sequence of dict

`Features.encode_example` currently fails if it contains a sequence if dict like `Sequence({"subcolumn": Value("int32")})` and if `None` is passed instead of the dict.

```python
  File "/Users/quentinlhoest/Desktop/hf/datasets/src/datasets/features/features.py", line 1310, in encode_example
    return encode_nested_example(self, example)
  File "/Users/quentinlhoest/Desktop/hf/datasets/src/datasets/features/features.py", line 973, in encode_nested_example
    return {k: encode_nested_example(sub_schema, sub_obj) for k, (sub_schema, sub_obj) in zip_dict(schema, obj)}
  File "/Users/quentinlhoest/Desktop/hf/datasets/src/datasets/features/features.py", line 973, in <dictcomp>
    return {k: encode_nested_example(sub_schema, sub_obj) for k, (sub_schema, sub_obj) in zip_dict(schema, obj)}
  File "/Users/quentinlhoest/Desktop/hf/datasets/src/datasets/features/features.py", line 998, in encode_nested_example
    for k, (sub_schema, sub_objs) in zip_dict(schema.feature, obj):
  File "/Users/quentinlhoest/Desktop/hf/datasets/src/datasets/utils/py_utils.py", line 207, in zip_dict
    yield key, tuple(d[key] for d in dicts)
  File "/Users/quentinlhoest/Desktop/hf/datasets/src/datasets/utils/py_utils.py", line 207, in <genexpr>
    yield key, tuple(d[key] for d in dicts)
TypeError: 'NoneType' object is not subscriptable
```

I fixed this issue and updated the tests (this case was missing in the tests)
AMI load_dataset error: sndfile library not found

## Describe the bug
Getting error message when loading AMI dataset.

## Steps to reproduce the bug
`python3 -c "from datasets import load_dataset; print(load_dataset('ami', 'headset-single', split='validation')[0])"

`

## Expected results
A clear and concise description of the expected results.

## Actual results
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/load.py", line 1707, in load_dataset
    use_auth_token=use_auth_token,
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 595, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 690, in _download_and_prepare
    ) from None
OSError: Cannot find data file. 
Original error:
sndfile library not found

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.18.3
- Platform: Linux-4.19.0-18-cloud-amd64-x86_64-with-debian-10.11
- Python version: 3.7.3
- PyArrow version: 7.0.0
set_format does not work with multi dimension tensor

## Describe the bug
set_format only transforms the last dimension of a multi-dimension list to tensor

## Steps to reproduce the bug
```python
import torch
from datasets import Dataset

ds = Dataset.from_dict({"A": [torch.rand((2, 2))]})
# ds = Dataset.from_dict({"A": [np.random.rand(2, 2)]})  # => same result
ds = ds.with_format("torch")
print(ds[0])
```

## Expected results
```
{'A': [tensor([[0.6689, 0.1516], [0.1403, 0.5567]])]}
```

## Actual results
```
{'A': [tensor([0.6689, 0.1516]), tensor([0.1403, 0.5567])]}
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- datasets version: 2.0.0
- Platform: Mac OSX
- Python version: 3.8.12
- PyArrow version: 7.0.0
Use audio feature in ASR task template

The AutomaticSpeechRecognition task template is outdated: it still uses the file path column as input instead of the audio column.

I changed that and updated all the datasets as well as the tests.

The only community dataset that will need to be updated is `facebook/multilingual_librispeech`. It has almost zero usage unfortunately (probably because users load the duplicate `multilingual_librispeech` directly instead), but it means we can update it.

(this makes me think that we should deprecate `multilingual_librispeech` it and redirect users to `facebook/multilingual_librispeech`).

This PR is also useful for the AudioFolder in https://github.com/huggingface/datasets/pull/3963
Yelp not working

## Dataset viewer issue for '*name of the dataset*'

**Link:** https://huggingface.co/datasets/yelp_review_full/viewer/yelp_review_full/train

Doesn't work:
```
Server error
Status code:   400
Exception:     Error
Message:       line contains NULL
```

Am I the one who added this dataset ? No

A seamingly copy of the dataset: https://huggingface.co/datasets/SetFit/yelp_review_full works . The original one: https://huggingface.co/datasets/yelp_review_full has > 20K downloads.
ASSIN2 dataset checksum bug

## Describe the bug

Checksum error after trying to load the [ASSIN 2 dataset](https://huggingface.co/datasets/assin2).

`NonMatchingChecksumError` triggered by calling `load_dataset("assin2")`.

Similar to #3952 , #3942 , #3941 , etc.

```
---------------------------------------------------------------------------
NonMatchingChecksumError                  Traceback (most recent call last)
[<ipython-input-13-c664a92ad5e7>](https://localhost:8080/#) in <module>()
----> 1 load_dataset('assin2')

4 frames
[/usr/local/lib/python3.7/dist-packages/datasets/utils/info_utils.py](https://localhost:8080/#) in verify_checksums(expected_checksums, recorded_checksums, verification_name)
     38     if len(bad_urls) > 0:
     39         error_msg = "Checksums didn't match" + for_verification_name + ":\n"
---> 40         raise NonMatchingChecksumError(error_msg + str(bad_urls))
     41     logger.info("All the checksums matched successfully" + for_verification_name)
     42 

NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/u/0/uc?id=1Q9j1a83CuKzsHCGaNulSkNxBm7Dkn7Ln&export=download']
```

## Steps to reproduce the bug
```python
from datasets import load_dataset
load_dataset("assin2")
```

## Expected results
Load the dataset.

## Actual results
The dataset won't load. 

## Environment info
- `datasets` version: 2.0.1.dev0
- Platform: Google Colab
- Python version:  3.7.12
- PyArrow version: 6.0.1
How to use generate this multitask dataset for SQUAD? I am getting a value error. 

## Dataset viewer issue for 'squad_multitask*'

**Link:** https://huggingface.co/datasets/vershasaxena91/squad_multitask

*short description of the issue*

I am trying to generate the multitask dataset for squad dataset. However, gives the error in dataset explorer as well as my local machine. 
I tried the command: dataset = load_dataset("vershasaxena91/squad_multitask", 'highlight_qg_format')
Error:
Status code:   400
Exception:     TypeError
Message:       argument of type 'Value' is not iterable

Kindly advice.
load_dataset error: sndfile library not found

## Describe the bug
Can't load ami dataset

## Steps to reproduce the bug
```
python3 -c "from datasets import load_dataset; print(load_dataset('ami', 'headset-single', split='validation')[0])"
```

## Expected results


## Actual results
Downloading and preparing dataset ami/headset-single (download: 10.71 GiB, generated: 49.99 MiB, post-processed: Unknown size, total: 10.76 GiB) to /home/neo/.cache/huggingface/datasets/ami/headset-single/1.6.2/2accdf810f7c0585f78f4bcfa47684fbb980e35d29ecf126e6906dbecb872d9e...
AMI corpus cannot be downloaded using multi-processing. Setting number of downloaded processes `num_proc` to 1. 
100%|██████████████████████████████████████████████████████| 136/136 [00:00<00:00, 36004.88it/s]
100%|█████████████████████████████████████████████████████████| 136/136 [00:01<00:00, 79.10it/s]
100%|████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 25343.23it/s]
100%|█████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 2874.78it/s]
100%|████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 27950.38it/s]
100%|█████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 2892.25it/s]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/load.py", line 1707, in load_dataset
    use_auth_token=use_auth_token,
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 595, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 690, in _download_and_prepare
    ) from None
OSError: Cannot find data file. 
Original error:
sndfile library not found

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.18.3
- Platform: Linux-4.19.0-18-cloud-amd64-x86_64-with-debian-10.11
- Python version: 3.7.3
- PyArrow version: 7.0.0
Sync Features dictionaries

This PR adds a wrapper to the `Features` class to keep the secondary dict, `_column_requires_decoding`, aligned with the main dict (as discussed in https://github.com/huggingface/datasets/pull/3723#discussion_r806912731).

A more elegant approach would be to subclass `UserDict` and override `__setitem__` and `__delitem__`, but this PR doesn't implement it for the following reasons:
* it requires replacing all occurrences of `isinstance(obj, dict)` with `isinstance(obj, Mapping)`, which is five times slower than `isinstance(obj, dict)` on my machine, in `features.py`
* is a breaking change, i.e., `isinstance(Features(...), dict)` would return `False` after it
* IMO, it makes sense to be consistent in the user-facing API and subclass either `dict` or `UserDict`. The problem with the latter is that it can't be used for `DatasetDict` because `DatasetDict` exposes the `data` property, which is also used by `UserDict`, so this would result in a collision.
Close `PIL.Image` file handler in `Image.decode_example`

Closes the file handler of the PIL image object in `Image.decode_example` to avoid the `Too many open files` error.

To pass [the image equality checks](https://app.circleci.com/pipelines/github/huggingface/datasets/10774/workflows/d56670e6-16bb-4c64-b601-a152c5acf5ed/jobs/65825) in CI, `Image.decode_example` calls `image.load()` regardless of how the image object is created (not only for the `PIL.Image.open(local_path)` case). This is needed because `load()` sets the `readonly` attribute of a `PIL.Image` object to 0 (it's 1 after `PIL.Image.open(file_like)`), and in the older PIL versions (only fixed on main), that attribute is considered in `PIL.Image.__eq__`. More info can be found here: https://github.com/python-pillow/Pillow/issues/5926.

Fix #3985
Streaming dataset + interleave + DataLoader hangs with multiple workers

## Describe the bug
Interleaving multiple iterable datasets that use `load_dataset` on streaming mode hangs when passed to `torch.utils.data.DataLoader` with multiple workers.

## Steps to reproduce the bug
```python
from datasets import interleave_datasets, load_dataset
from torch.utils.data import DataLoader

en_dataset = load_dataset('oscar', "unshuffled_deduplicated_en", split='train', streaming=True)
fr_dataset = load_dataset('oscar', "unshuffled_deduplicated_fr", split='train', streaming=True)
it_dataset = load_dataset('oscar', "unshuffled_deduplicated_it", split='train', streaming=True)
de_dataset = load_dataset('oscar', "unshuffled_deduplicated_de", split='train', streaming=True)
multilingual_dataset = interleave_datasets([en_dataset, fr_dataset, de_dataset, it_dataset])
multilingual_dataset = multilingual_dataset.with_format('torch')

next(iter(multilingual_dataset))  # works fairly fast

dataloader = DataLoader(multilingual_dataset, batch_size=8, num_workers=4)
for batch in dataloader:
    print(len(batch))  # prints nothing after 30 min of waiting

dataloader = DataLoader(multilingual_dataset, batch_size=8, num_workers=0)
for batch in dataloader:
    print(len(batch))  # prints right away

```

## Expected results
It should be able to iterate the dataset with multiple workers.

## Actual results
Prints with results with `next(iter(multilingual_dataset)) ` and `num_workers=0`  but it prints nothing with `num_workers=4` or any number above 0.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.0.1.dev0
- `pytorch` version: 1.10.0+cu113
- Python version: 3.7
- PyArrow version: 6.0.1
Image column is not decoded in map when using with with_transform

## Describe the bug
Image column is not _decoded_ in **map** when using with `with_transform`

## Steps to reproduce the bug
```python
from datasets import Image, Dataset

def add_C(batch):
    batch["C"] = batch["A"]
    return batch

ds = Dataset.from_dict({"A": ["image.png"]}).cast_column("A",  Image())
ds = ds.with_transform(lambda x: x)  # <= This line causes the problem
ds = ds.map(add_C, batched=True)
print(ds[0])
```

## Expected results
```
{'C': <PIL.PngImagePlugin.PngImageFile>,  ...}
```

## Actual results
```
{'C': {'bytes': None, 'path': 'image.png'}, ...}
```

If we remove the `with_transform` line, we get the expected result.

## Environment info
- `datasets` version: 2.0.0
- Platform: Mac OSX
- Python version: 3.8.12
- PyArrow version: 7.0.0
Improve AutomaticSpeechRecognition task template

**Is your feature request related to a problem? Please describe.**
[AutomaticSpeechRecognition task template](https://github.com/huggingface/datasets/blob/master/src/datasets/tasks/automatic_speech_recognition.py) is outdated as it uses path to audiofile as an audio column instead of a Audio feature itself (I guess it's because Audio feature didn't exist at the time this template was created).

**Describe the solution you'd like**
Change audio columns from string path to Audio feature.
Dataset loads indefinitely after modifying default cache path (~/.cache/huggingface)

## Describe the bug
Dataset loads indefinitely after modifying cache path (~/.cache/huggingface)
If none of the environment variables are set, this custom dataset loads fine ( json-based dataset with custom dataset load script)
** Update: Transformer modules faces the same issue as well during loading

## A clear and concise description of what the bug is.

Issue:
- Dataset loading stalls / freezes indefinitely when HF_HOME is changed to a custom directory
- No error code, had to terminate the process
- There are some files created in the cache directory:

```
custom_cache_dir  
  | -- modules  
            | -- __init__.py  
            | -- datasets_modules  
                      | -- __init__.py  
                      | -- datasets  
                             | -- __init__.py  
                             | -- script.py (Dataset loading script)  
                             | -- script.lock   

```

There's no error nor any logs thrown so I'm out of ideas of how to to debug this. The custom dataset works fine if the default ~/.cache dir is used, but unfortunately it's out of space and we do not have permissions to modify the disk.

## Steps to reproduce the bug

What I've tried:
- Modifying HF_HOME  (https://github.com/huggingface/transformers/issues/8703)
- Modifying HF_DATASETS_CACHE (https://huggingface.co/docs/datasets/v1.12.0/cache.html)
- Modifying cache_dir param during runtime
```python
>>> from datasets import load_dataset
>>> dataset = load_dataset('test_dataset', cache_dir='/path/to/new/cache')
```
- Disabling dataset cache
```python
>>> from datasets import set_caching_enabled
>>> set_caching_enabled(False)
```

## Expected results
Datasets should load / cache as usual with the only exception that cache directory is different

## Actual results

Any actions taken above to change the cache directory results in loading indefinitely without terminating.

## Environment info

- `transformers` version: 4.18.0.dev0
- Platform: Linux-4.15.0-54-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU?): 1.8.1+cu102 (True)
- Tensorflow version (GPU?): 2.4.1 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No
Exclude Google Drive tests of the CI

These tests make the CI spam the Google Drive API, the CI now gets banned by Google Drive very often.

I think we can just skip these tests from the CI for now.

In the future we could have a CI job that runs only once a day or once a week for such cases

cc @albertvillanova @mariosasko @severo 

Close #3415

![image](https://user-images.githubusercontent.com/42851186/159283608-fdeca1ac-b57f-4fa3-bf09-6fa5361c494f.png)
Add tip on how to speed up loading with ImageFolder

This PR does two things:
* adds a tip on how to speed up loading of a large number of files with ImageFolder (motivated by [this issue](https://github.com/huggingface/datasets/issues/3960))
* replaces the current references to the `Dataset` methods in the Image Processing doc with their fully qualified counterparts (to align it with the Audio Processing doc)

cc @stevhliu
I can't view HFcallback dataset for ASR Space

## Dataset viewer issue for '*Urdu-ASR-flags*'

**Link:** *[link to the dataset viewer page](https://huggingface.co/datasets/kingabzpro/Urdu-ASR-flags)*

*I think dataset should show some thing and if you want me to add script, please show me the documentation. I thought this was suppose to be automatic task.*

Am I the one who added this dataset ? Yes
Cannot preview 'indonesian-nlp/eli5_id' dataset

## Dataset viewer issue for '*indonesian-nlp/eli5_id*'

**Link:** https://huggingface.co/datasets/indonesian-nlp/eli5_id

I can not see the dataset preview.
```
Server Error
Status code:   400
Exception:     Status400Error
Message:       Not found. Maybe the cache is missing, or maybe the dataset does not exist.
```
Am I the one who added this dataset ? Yes
Add Audio Folder

Would resolve  #3964

AudioFolder loads a .txt file with transcriptions and creates a dataset with all audiofiles in provided directory that has a transcription (independently of the directory structure) as a single split (train).

Can be loaded via:
```python
# for local dirs
dataset = load_dataset("audiofolder", data_dir="/path/to/folder", transcripts_filename="transcripts.txt")
```
```python
# for local and remote zip archives
dataset = load_dataset("audiofolder", data_files="path/to/archive/archive.zip", transcripts_filename="transcripts.txt")
```
default transcriptions filename is `transcripts.txt`. it should have the following structure:
```
audio_id_1   transcription text 1
audio_id_1   transcription text 1
```
separator is `\t`!

---
sorry for first old commits from other branch, don't know how that happened...
Scores from Index at extra positions are not filtered out

If a FAISS index has fewer records than the requested number of top results (k), then it returns -1 in indices for the additional positions. The get_nearest_examples method only filters out the extra results from the dataset samples. It would be better to filter out extra scores too.

Reference: https://github.com/huggingface/datasets/blob/2.0.0/src/datasets/search.py#L693