Changing sampling rate in audio dataset and subsequently mapping with `num_proc > 1` leads to weird bug

Running: 

```python
from datasets import load_dataset, DatasetDict
import datasets
from transformers import AutoFeatureExtractor

raw_datasets = DatasetDict()

raw_datasets["train"] = load_dataset("common_voice", "ab", split="train")

feature_extractor = AutoFeatureExtractor.from_pretrained("facebook/wav2vec2-base")

raw_datasets = raw_datasets.cast_column(
    "audio", datasets.features.Audio(sampling_rate=feature_extractor.sampling_rate)
)

num_workers = 16

def prepare_dataset(batch):
    sample = batch["audio"]

    inputs = feature_extractor(sample["array"], sampling_rate=sample["sampling_rate"])
    batch["input_values"] = inputs.input_values[0]
    batch["input_length"] = len(batch["input_values"])
    return batch

raw_datasets.map(
    prepare_dataset,
    remove_columns=next(iter(raw_datasets.values())).column_names,
    num_proc=16,
    desc="preprocess datasets",
)
```

gives

```bash
  File "/home/patrick/experiments/run_bug.py", line 25, in <module>
    raw_datasets.map(
  File "/home/patrick/python_bin/datasets/dataset_dict.py", line 492, in map
    {
  File "/home/patrick/python_bin/datasets/dataset_dict.py", line 493, in <dictcomp>
    k: dataset.map(
  File "/home/patrick/python_bin/datasets/arrow_dataset.py", line 2139, in map
    shards = [
  File "/home/patrick/python_bin/datasets/arrow_dataset.py", line 2140, in <listcomp>
    self.shard(num_shards=num_proc, index=rank, contiguous=True, keep_in_memory=keep_in_memory)
  File "/home/patrick/python_bin/datasets/arrow_dataset.py", line 3164, in shard
    return self.select(
  File "/home/patrick/python_bin/datasets/arrow_dataset.py", line 485, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "/home/patrick/python_bin/datasets/fingerprint.py", line 411, in wrapper
    out = func(self, *args, **kwargs)
  File "/home/patrick/python_bin/datasets/arrow_dataset.py", line 2756, in select
    return self._new_dataset_with_indices(indices_buffer=buf_writer.getvalue(), fingerprint=new_fingerprint)
  File "/home/patrick/python_bin/datasets/arrow_dataset.py", line 2667, in _new_dataset_with_indices
    return Dataset(
  File "/home/patrick/python_bin/datasets/arrow_dataset.py", line 659, in __init__
    raise ValueError(
ValueError: External features info don't match the dataset:
Got
{'client_id': Value(dtype='string', id=None), 'path': Value(dtype='string', id=None), 'audio': Audio(sampling_rate=16000, mono=True, _storage_dtype='string', id=None), 'sentence': Value(dtype='string', id=None), 'up_votes': Value(dtype='int64', id=None), 'down_votes': Value(dtype='int64', id=None), 'age': Value(dtype='string', id=None), 'gender': Value(dtype='string', id=None), 'accent': Value(dtype='string', id=None), 'locale': Value(dtype='string', id=None), 'segment': Value(dtype='string', id=None)}
with type
struct<client_id: string, path: string, audio: string, sentence: string, up_votes: int64, down_votes: int64, age: string, gender: string, accent: string, locale: string, segment: string>

but expected something like
{'client_id': Value(dtype='string', id=None), 'path': Value(dtype='string', id=None), 'audio': {'path': Value(dtype='string', id=None), 'bytes': Value(dtype='binary', id=None)}, 'sentence': Value(dtype='string', id=None), 'up_votes': Value(dtype='int64', id=None), 'down_votes': Value(dtype='int64', id=None), 'age': Value(dtype='string', id=None), 'gender': Value(dtype='string', id=None), 'accent': Value(dtype='string', id=None), 'locale': Value(dtype='string', id=None), 'segment': Value(dtype='string', id=None)}
with type
struct<client_id: string, path: string, audio: struct<path: string, bytes: binary>, sentence: string, up_votes: int64, down_votes: int64, age: string, gender: string, accent: string, locale: string, segment: string>
```

Versions:

```python
- `datasets` version: 1.16.2.dev0
- Platform: Linux-5.15.8-76051508-generic-x86_64-with-glibc2.33
- Python version: 3.9.7
- PyArrow version: 6.0.1
```

and `transformers`:

```
- `transformers` version: 4.16.0.dev0
- Platform: Linux-5.15.8-76051508-generic-x86_64-with-glibc2.33
- Python version: 3.9.7
```

Labels: name: bug

Same error occures when using max samples with https://github.com/huggingface/transformers/blob/master/examples/pytorch/speech-recognition/run_speech_recognition_seq2seq.py

I'm seeing this too, when using preprocessing_num_workers with 
https://github.com/huggingface/transformers/blob/master/examples/pytorch/speech-recognition/run_speech_recognition_ctc.py