Error loading file audio when downloading the Common Voice dataset directly from the Hub
Opened by aliceinland (closed)

## Describe the bug
When loading the Common_Voice dataset, by downloading it directly from the Hugging Face hub, some files can not be opened. 

## Steps to reproduce the bug
```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

test_dataset = load_dataset("common_voice", "it", split="test")
#test_dataset = load_dataset('csv', data_files = {'test': '/workspace/Dataset/Common_Voice/cv-corpus80/it/test.csv'})
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("joorock12/wav2vec2-large-xlsr-italian")
model = Wav2Vec2ForCTC.from_pretrained("joorock12/wav2vec2-large-xlsr-italian")
model.to("cuda")

chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"\“\'\�]'
resampler = torchaudio.transforms.Resample(48_000, 16_000)
```
## Expected results
The common voice dataset downloaded and correctly loaded whit the use of the hugging face datasets library. 

## Actual results
The error is:
```python
0ex [00:00, ?ex/s]
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-48-ef87f4129e6e> in <module>
      7     return batch
      8 
----> 9 test_dataset = test_dataset.map(speech_file_to_array_fn)

/opt/conda/lib/python3.8/site-packages/datasets/arrow_dataset.py in map(self, function, with_indices, with_rank, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, num_proc, suffix_template, new_fingerprint, desc)
   2107 
   2108         if num_proc is None or num_proc == 1:
-> 2109             return self._map_single(
   2110                 function=function,
   2111                 with_indices=with_indices,

/opt/conda/lib/python3.8/site-packages/datasets/arrow_dataset.py in wrapper(*args, **kwargs)
    516             self: "Dataset" = kwargs.pop("self")
    517         # apply actual function
--> 518         out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
    519         datasets: List["Dataset"] = list(out.values()) if isinstance(out, dict) else [out]
    520         for dataset in datasets:

/opt/conda/lib/python3.8/site-packages/datasets/arrow_dataset.py in wrapper(*args, **kwargs)
    483         }
    484         # apply actual function
--> 485         out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
    486         datasets: List["Dataset"] = list(out.values()) if isinstance(out, dict) else [out]
    487         # re-apply format to the output

/opt/conda/lib/python3.8/site-packages/datasets/fingerprint.py in wrapper(*args, **kwargs)
    411             # Call actual function
    412 
--> 413             out = func(self, *args, **kwargs)
    414 
    415             # Update fingerprint of in-place transforms + update in-place history of transforms

/opt/conda/lib/python3.8/site-packages/datasets/arrow_dataset.py in _map_single(self, function, with_indices, with_rank, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, new_fingerprint, rank, offset, disable_tqdm, desc, cache_only)
   2465                 if not batched:
   2466                     for i, example in enumerate(pbar):
-> 2467                         example = apply_function_on_filtered_inputs(example, i, offset=offset)
   2468                         if update_data:
   2469                             if i == 0:

/opt/conda/lib/python3.8/site-packages/datasets/arrow_dataset.py in apply_function_on_filtered_inputs(inputs, indices, check_same_num_examples, offset)
   2372             if with_rank:
   2373                 additional_args += (rank,)
-> 2374             processed_inputs = function(*fn_args, *additional_args, **fn_kwargs)
   2375             if update_data is None:
   2376                 # Check if the function returns updated examples

/opt/conda/lib/python3.8/site-packages/datasets/arrow_dataset.py in decorated(item, *args, **kwargs)
   2067                 )
   2068                 # Use the LazyDict internally, while mapping the function
-> 2069                 result = f(decorated_item, *args, **kwargs)
   2070                 # Return a standard dict
   2071                 return result.data if isinstance(result, LazyDict) else result

<ipython-input-48-ef87f4129e6e> in speech_file_to_array_fn(batch)
      3 def speech_file_to_array_fn(batch):
      4     batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
----> 5     speech_array, sampling_rate = torchaudio.load(batch["path"])
      6     batch["speech"] = resampler(speech_array).squeeze().numpy()
      7     return batch

/opt/conda/lib/python3.8/site-packages/torchaudio/backend/sox_io_backend.py in load(filepath, frame_offset, num_frames, normalize, channels_first, format)
    150                 filepath, frame_offset, num_frames, normalize, channels_first, format)
    151         filepath = os.fspath(filepath)
--> 152     return torch.ops.torchaudio.sox_io_load_audio_file(
    153         filepath, frame_offset, num_frames, normalize, channels_first, format)
    154 

RuntimeError: Error loading audio file: failed to open file common_voice_it_17415776.mp3  ```


## Environment info

- `datasets` version: 1.18.4
- Platform: Linux-5.4.0-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyArrow version: 7.0.0

Comments:
Hi ! It could an issue with torchaudio, which version of torchaudio are you using ? Can you also try updating `datasets` to 2.0.0 and see if it works ?

I _might_ have a similar issue. I'm trying to use the librispeech_asr dataset and read it with soundfile.

```python
from datasets import load_dataset, load_metric
from transformers import Speech2TextForConditionalGeneration, Speech2TextProcessor
import soundfile as sf

librispeech_eval = load_dataset("librispeech_asr", "clean", split="test")  # change to "other" for other test dataset
wer = load_metric("wer")

model = Speech2TextForConditionalGeneration.from_pretrained("facebook/s2t-small-librispeech-asr").to("cuda")
processor = Speech2TextProcessor.from_pretrained("facebook/s2t-small-librispeech-asr", do_upper_case=True)

def map_to_array(batch):
    speech, _ = sf.read(batch["file"])
    batch["speech"] = speech
    return batch

librispeech_eval = librispeech_eval.map(map_to_array)

def map_to_pred(batch):
    features = processor(batch["speech"], sampling_rate=16000, padding=True, return_tensors="pt")
    input_features = features.input_features.to("cuda")
    attention_mask = features.attention_mask.to("cuda")

    gen_tokens = model.generate(input_ids=input_features, attention_mask=attention_mask)
    batch["transcription"] = processor.batch_decode(gen_tokens, skip_special_tokens=True)
    return batch

result = librispeech_eval.map(map_to_pred, batched=True, batch_size=8, remove_columns=["speech"])

print("WER:", wer(predictions=result["transcription"], references=result["text"]))
```

The code is taken directly from "https://huggingface.co/facebook/s2t-small-librispeech-asr".

The short error code is "RuntimeError: Error opening '6930-75918-0000.flac': System error." (it can't find the first file), and I agree, I can't find the file either. The dataset has downloaded correctly (it says), but on the location, there are only ".arrow" files, no ".flac" files.

**Error message:**

```python
RuntimeError                              Traceback (most recent call last)
Input In [15], in <cell line: 16>()
     13     batch["speech"] = speech
     14     return batch
---> 16 librispeech_eval = librispeech_eval.map(map_to_array)
     18 def map_to_pred(batch):
     19     features = processor(batch["speech"], sampling_rate=16000, padding=True, return_tensors="pt")

File C:\ProgramData\Miniconda3\envs\noise_cancel\lib\site-packages\datasets\arrow_dataset.py:1953, in Dataset.map(self, function, with_indices, with_rank, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, num_proc, suffix_template, new_fingerprint, desc)
   1950 disable_tqdm = not logging.is_progress_bar_enabled()
   1952 if num_proc is None or num_proc == 1:
-> 1953     return self._map_single(
   1954         function=function,
   1955         with_indices=with_indices,
   1956         with_rank=with_rank,
   1957         input_columns=input_columns,
   1958         batched=batched,
   1959         batch_size=batch_size,
   1960         drop_last_batch=drop_last_batch,
   1961         remove_columns=remove_columns,
   1962         keep_in_memory=keep_in_memory,
   1963         load_from_cache_file=load_from_cache_file,
   1964         cache_file_name=cache_file_name,
   1965         writer_batch_size=writer_batch_size,
   1966         features=features,
   1967         disable_nullable=disable_nullable,
   1968         fn_kwargs=fn_kwargs,
   1969         new_fingerprint=new_fingerprint,
   1970         disable_tqdm=disable_tqdm,
   1971         desc=desc,
   1972     )
   1973 else:
   1975     def format_cache_file_name(cache_file_name, rank):

File C:\ProgramData\Miniconda3\envs\noise_cancel\lib\site-packages\datasets\arrow_dataset.py:519, in transmit_tasks.<locals>.wrapper(*args, **kwargs)
    517     self: "Dataset" = kwargs.pop("self")
    518 # apply actual function
--> 519 out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
    520 datasets: List["Dataset"] = list(out.values()) if isinstance(out, dict) else [out]
    521 for dataset in datasets:
    522     # Remove task templates if a column mapping of the template is no longer valid

File C:\ProgramData\Miniconda3\envs\noise_cancel\lib\site-packages\datasets\arrow_dataset.py:486, in transmit_format.<locals>.wrapper(*args, **kwargs)
    479 self_format = {
    480     "type": self._format_type,
    481     "format_kwargs": self._format_kwargs,
    482     "columns": self._format_columns,
    483     "output_all_columns": self._output_all_columns,
    484 }
    485 # apply actual function
--> 486 out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
    487 datasets: List["Dataset"] = list(out.values()) if isinstance(out, dict) else [out]
    488 # re-apply format to the output

File C:\ProgramData\Miniconda3\envs\noise_cancel\lib\site-packages\datasets\fingerprint.py:458, in fingerprint_transform.<locals>._fingerprint.<locals>.wrapper(*args, **kwargs)
    452             kwargs[fingerprint_name] = update_fingerprint(
    453                 self._fingerprint, transform, kwargs_for_fingerprint
    454             )
    456 # Call actual function
--> 458 out = func(self, *args, **kwargs)
    460 # Update fingerprint of in-place transforms + update in-place history of transforms
    462 if inplace:  # update after calling func so that the fingerprint doesn't change if the function fails

File C:\ProgramData\Miniconda3\envs\noise_cancel\lib\site-packages\datasets\arrow_dataset.py:2318, in Dataset._map_single(self, function, with_indices, with_rank, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, new_fingerprint, rank, offset, disable_tqdm, desc, cache_only)
   2316 if not batched:
   2317     for i, example in enumerate(pbar):
-> 2318         example = apply_function_on_filtered_inputs(example, i, offset=offset)
   2319         if update_data:
   2320             if i == 0:

File C:\ProgramData\Miniconda3\envs\noise_cancel\lib\site-packages\datasets\arrow_dataset.py:2218, in Dataset._map_single.<locals>.apply_function_on_filtered_inputs(inputs, indices, check_same_num_examples, offset)
   2216 if with_rank:
   2217     additional_args += (rank,)
-> 2218 processed_inputs = function(*fn_args, *additional_args, **fn_kwargs)
   2219 if update_data is None:
   2220     # Check if the function returns updated examples
   2221     update_data = isinstance(processed_inputs, (Mapping, pa.Table))

File C:\ProgramData\Miniconda3\envs\noise_cancel\lib\site-packages\datasets\arrow_dataset.py:1913, in Dataset.map.<locals>.decorate.<locals>.decorated(item, *args, **kwargs)
   1909 decorated_item = (
   1910     Example(item, features=self.features) if not batched else Batch(item, features=self.features)
   1911 )
   1912 # Use the LazyDict internally, while mapping the function
-> 1913 result = f(decorated_item, *args, **kwargs)
   1914 # Return a standard dict
   1915 return result.data if isinstance(result, LazyDict) else result

Input In [15], in map_to_array(batch)
     11 def map_to_array(batch):
---> 12     speech, _ = sf.read(batch["file"])
     13     batch["speech"] = speech
     14     return batch

File C:\ProgramData\Miniconda3\envs\noise_cancel\lib\site-packages\soundfile.py:256, in read(file, frames, start, stop, dtype, always_2d, fill_value, out, samplerate, channels, format, subtype, endian, closefd)
    170 def read(file, frames=-1, start=0, stop=None, dtype='float64', always_2d=False,
    171          fill_value=None, out=None, samplerate=None, channels=None,
    172          format=None, subtype=None, endian=None, closefd=True):
    173     """Provide audio data from a sound file as NumPy array.
    174 
    175     By default, the whole file is read from the beginning, but the
   (...)
    254 
    255     """
--> 256     with SoundFile(file, 'r', samplerate, channels,
    257                    subtype, endian, format, closefd) as f:
    258         frames = f._prepare_read(start, stop, frames)
    259         data = f.read(frames, dtype, always_2d, fill_value, out)

File C:\ProgramData\Miniconda3\envs\noise_cancel\lib\site-packages\soundfile.py:629, in SoundFile.__init__(self, file, mode, samplerate, channels, subtype, endian, format, closefd)
    626 self._mode = mode
    627 self._info = _create_info_struct(file, mode, samplerate, channels,
    628                                  format, subtype, endian)
--> 629 self._file = self._open(file, mode_int, closefd)
    630 if set(mode).issuperset('r+') and self.seekable():
    631     # Move write position to 0 (like in Python file objects)
    632     self.seek(0)

File C:\ProgramData\Miniconda3\envs\noise_cancel\lib\site-packages\soundfile.py:1183, in SoundFile._open(self, file, mode_int, closefd)
   1181 else:
   1182     raise TypeError("Invalid file: {0!r}".format(self.name))
-> 1183 _error_check(_snd.sf_error(file_ptr),
   1184              "Error opening {0!r}: ".format(self.name))
   1185 if mode_int == _snd.SFM_WRITE:
   1186     # Due to a bug in libsndfile version <= 1.0.25, frames != 0
   1187     # when opening a named pipe in SFM_WRITE mode.
   1188     # See http://github.com/erikd/libsndfile/issues/77.
   1189     self._info.frames = 0

File C:\ProgramData\Miniconda3\envs\noise_cancel\lib\site-packages\soundfile.py:1357, in _error_check(err, prefix)
   1355 if err != 0:
   1356     err_str = _snd.sf_error_number(err)
-> 1357     raise RuntimeError(prefix + _ffi.string(err_str).decode('utf-8', 'replace'))

RuntimeError: Error opening '6930-75918-0000.flac': System error.
```

**Package versions:**
```python
python: 3.9
transformers: 4.17.0
datasets: 2.0.0
SoundFile: 0.10.3.post1
```


Hi ! In `datasets` 2.0 can access the audio array with `librispeech_eval[0]["audio"]["array"]` already, no need to use `map_to_array`. See our documentation on [how to process audio data](https://huggingface.co/docs/datasets/audio_process) :)

cc @patrickvonplaten we will need to update the readme at [facebook/s2t-small-librispeech-asr](https://huggingface.co/facebook/s2t-small-librispeech-asr) as well as https://huggingface.co/docs/transformers/model_doc/speech_to_text

Thanks!

And sorry for posting this problem in what turned on to be an unrelated thread.

I rewrote the code, and the model works. The WER is 0.137 however, so I'm not sure if I have missed a step. I will look further into that at a later point. The transcriptions look good through manual inspection.

The rewritten code:
```python
from datasets import load_dataset, load_metric
from transformers import Speech2TextForConditionalGeneration, Speech2TextProcessor, Wav2Vec2Processor

librispeech_eval = load_dataset("librispeech_asr", "clean", split="test")  # change to "other" for other test dataset
wer = load_metric("wer")

model = Speech2TextForConditionalGeneration.from_pretrained("facebook/s2t-small-librispeech-asr").to("cuda")
processor = Speech2TextProcessor.from_pretrained("facebook/s2t-small-librispeech-asr", do_upper_case=True)

def map_to_pred(batch):
    audio = batch["audio"]
    features = processor(audio["array"], sampling_rate=audio["sampling_rate"], padding=True, return_tensors="pt")
    input_features = features.input_features.to("cuda")
    attention_mask = features.attention_mask.to("cuda")

    gen_tokens = model.generate(input_features=input_features, attention_mask=attention_mask)
    batch["transcription"] = processor.batch_decode(gen_tokens, skip_special_tokens=True)
    return batch

result = librispeech_eval.map(map_to_pred)#, batched=True, batch_size=8)

print("WER:", wer.compute(predictions=result["transcription"], references=result["text"]))
```

I think the issue comes from the fact that you set `batched=False` while `map_to_pred` still returns a list of strings for "transcription". You can fix it by adding `[0]` at the end of this line to get the string:
```python
batch["transcription"] = processor.batch_decode(gen_tokens, skip_special_tokens=True)[0]
```

Updating as many model cards now as I can find

https://github.com/huggingface/transformers/pull/16611

We no longer use `torchaudio` for decoding MP3 files, and the problem with model cards has been addressed, so I'm closing this issue.