Error when running "Quick Tour" code snippets
Author: srujanjoshi
Pull request: no

## Environment info

- `transformers` version: 4.9.2
- Platform: Linux-5.13.0-39-generic-x86_64-with-glibc2.17
- Python version: 3.8.11
- PyTorch version (GPU?): 1.9.1 (True)
- Tensorflow version (GPU?): 2.6.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Parallel 


@sgugger @patrickvonplaten @anton-l @Narsil 




## Information

Model I am using: wav2vec2

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

Hey, I'm new to Transformers so pardon me if this issue has an obvious fix I can't think of. I was trying to go through the Quick Tour (https://huggingface.co/docs/transformers/quicktour), and I encountered an error when running the code snippets mentioned there. 

## To reproduce

Steps to reproduce the behavior:

```

from transformers import pipeline
import datasets
speech_recognizer = pipeline ("automatic-speech-recognition", model = "facebook/wav2vec2-base-960h" ,device = 0)
dataset = datasets.load_dataset("superb", name ="asr", split = "test")
files = dataset["file"]
speech_recognizer(files[:4])
```

Here's the Stack Trace:

```
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
/tmp/ipykernel_16600/2678924457.py in <module>
----> 1 speech_recognizer(files[:4])

~/miniconda3/envs/mytextattack/lib/python3.8/site-packages/transformers/pipelines/automatic_speech_recognition.py in __call__(self, inputs, **kwargs)
    131             inputs = ffmpeg_read(inputs, self.feature_extractor.sampling_rate)
    132 
--> 133         assert isinstance(inputs, np.ndarray), "We expect a numpy ndarray as input"
    134         assert len(inputs.shape) == 1, "We expect a single channel audio input for AutomaticSpeechRecognitionPipeline"
    135 

AssertionError: We expect a numpy ndarray as input

```

I tried mitigating this error by converting the list of filenames to a numpy array, but I seem to get another error that I don't know how to deal with:

```

from transformers import pipeline
import datasets
import numpy as np
speech_recognizer = pipeline ("automatic-speech-recognition", model = "facebook/wav2vec2-base-960h" ,device = 0)
dataset = datasets.load_dataset("superb", name ="asr", split = "test")
files = dataset["file"]
speech_recognizer(np.array(files[:4]))
```

Stack Trace:

```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_16600/437131926.py in <module>
      1 import numpy as np
      2 
----> 3 speech_recognizer(np.array(files[:4]))

~/miniconda3/envs/mytextattack/lib/python3.8/site-packages/transformers/pipelines/automatic_speech_recognition.py in __call__(self, inputs, **kwargs)
    134         assert len(inputs.shape) == 1, "We expect a single channel audio input for AutomaticSpeechRecognitionPipeline"
    135 
--> 136         processed = self.feature_extractor(
    137             inputs, sampling_rate=self.feature_extractor.sampling_rate, return_tensors="pt"
    138         )

~/miniconda3/envs/mytextattack/lib/python3.8/site-packages/transformers/models/wav2vec2/feature_extraction_wav2vec2.py in __call__(self, raw_speech, padding, max_length, pad_to_multiple_of, return_attention_mask, return_tensors, sampling_rate, **kwargs)
    179         # zero-mean and unit-variance normalization
    180         if self.do_normalize:
--> 181             raw_speech = self.zero_mean_unit_var_norm(raw_speech)
    182 
    183         # convert into correct format for padding

~/miniconda3/envs/mytextattack/lib/python3.8/site-packages/transformers/models/wav2vec2/feature_extraction_wav2vec2.py in zero_mean_unit_var_norm(input_values)
     84         Every array in the list is normalized to have zero mean and unit variance
     85         """
---> 86         return [(x - np.mean(x)) / np.sqrt(np.var(x) + 1e-5) for x in input_values]
     87 
     88     def __call__(

~/miniconda3/envs/mytextattack/lib/python3.8/site-packages/transformers/models/wav2vec2/feature_extraction_wav2vec2.py in <listcomp>(.0)
     84         Every array in the list is normalized to have zero mean and unit variance
     85         """
---> 86         return [(x - np.mean(x)) / np.sqrt(np.var(x) + 1e-5) for x in input_values]
     87 
     88     def __call__(

<__array_function__ internals> in mean(*args, **kwargs)

~/miniconda3/envs/mytextattack/lib/python3.8/site-packages/numpy/core/fromnumeric.py in mean(a, axis, dtype, out, keepdims, where)
   3417             return mean(axis=axis, dtype=dtype, out=out, **kwargs)
   3418 
-> 3419     return _methods._mean(a, axis=axis, dtype=dtype,
   3420                           out=out, **kwargs)
   3421 

~/miniconda3/envs/mytextattack/lib/python3.8/site-packages/numpy/core/_methods.py in _mean(a, axis, dtype, out, keepdims, where)
    176             is_float16_result = True
    177 
--> 178     ret = umr_sum(arr, axis, dtype, out, keepdims, where=where)
    179     if isinstance(ret, mu.ndarray):
    180         ret = um.true_divide(

TypeError: cannot perform reduce with flexible type

```

I was wondering if someone could provide some insight on how to fix this?



Comments:
@srujanjoshi Did you try to upgrade your `transformers` version ? It works as advertised on latest release.

There were some improvements made the the pipeline made since `4.9` including easier integration with datasets.
I would like to add that even you pass 4 items, it's not batched, you need to use `batch_size` if you want batching (and I don't recommend baching blindly whole files, as padding can become extremely big :)). Just because the number `4` seemed to have been intended as a sort of batching method :)

Cheers.

We also just updated the quicktour example @srujanjoshi - could you try out the new example? :-)

@patrickvonplaten @Narsil 

Okay, I updated my transformers version to 4.18. When I run the updated code snippet (listed  below):

```python
from transformers import pipeline
import datasets
speech_recognizer = pipeline ("automatic-speech-recognition", model = "facebook/wav2vec2-base-960h" ,device = 0)
dataset = datasets.load_dataset("PolyAI/minds14", name="en-US", split="train")
files = dataset["path"]
speech_recognizer(files[:4])
```
I get the following stack trace: 
```
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
File ~/miniconda3/envs/transformersenv/lib/python3.8/site-packages/transformers/pipelines/audio_utils.py:32, in ffmpeg_read(bpayload, sampling_rate)
     31 try:
---> 32     with subprocess.Popen(ffmpeg_command, stdin=subprocess.PIPE, stdout=subprocess.PIPE) as ffmpeg_process:
     33         output_stream = ffmpeg_process.communicate(bpayload)

File ~/miniconda3/envs/transformersenv/lib/python3.8/subprocess.py:858, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, encoding, errors, text)
    855             self.stderr = io.TextIOWrapper(self.stderr,
    856                     encoding=encoding, errors=errors)
--> 858     self._execute_child(args, executable, preexec_fn, close_fds,
    859                         pass_fds, cwd, env,
    860                         startupinfo, creationflags, shell,
    861                         p2cread, p2cwrite,
    862                         c2pread, c2pwrite,
    863                         errread, errwrite,
    864                         restore_signals, start_new_session)
    865 except:
    866     # Cleanup if the child failed starting.

File ~/miniconda3/envs/transformersenv/lib/python3.8/subprocess.py:1704, in Popen._execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session)
   1703         err_msg = os.strerror(errno_num)
-> 1704     raise child_exception_type(errno_num, err_msg, err_filename)
   1705 raise child_exception_type(err_msg)

FileNotFoundError: [Errno 2] No such file or directory: 'ffmpeg'

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
Input In [37], in <cell line: 1>()
----> 1 speech_recognizer(files[:4])

File ~/miniconda3/envs/transformersenv/lib/python3.8/site-packages/transformers/pipelines/automatic_speech_recognition.py:167, in AutomaticSpeechRecognitionPipeline.__call__(self, inputs, **kwargs)
    126 def __call__(
    127     self,
    128     inputs: Union[np.ndarray, bytes, str],
    129     **kwargs,
    130 ):
    131     """
    132     Classify the sequence(s) given as inputs. See the [`AutomaticSpeechRecognitionPipeline`] documentation for more
    133     information.
   (...)
    165                     `"".join(chunk["text"] for chunk in output["chunks"])`.
    166     """
--> 167     return super().__call__(inputs, **kwargs)

File ~/miniconda3/envs/transformersenv/lib/python3.8/site-packages/transformers/pipelines/base.py:1015, in Pipeline.__call__(self, inputs, num_workers, batch_size, *args, **kwargs)
   1011 if can_use_iterator:
   1012     final_iterator = self.get_iterator(
   1013         inputs, num_workers, batch_size, preprocess_params, forward_params, postprocess_params
   1014     )
-> 1015     outputs = [output for output in final_iterator]
   1016     return outputs
   1017 else:

File ~/miniconda3/envs/transformersenv/lib/python3.8/site-packages/transformers/pipelines/base.py:1015, in <listcomp>(.0)
   1011 if can_use_iterator:
   1012     final_iterator = self.get_iterator(
   1013         inputs, num_workers, batch_size, preprocess_params, forward_params, postprocess_params
   1014     )
-> 1015     outputs = [output for output in final_iterator]
   1016     return outputs
   1017 else:

File ~/miniconda3/envs/transformersenv/lib/python3.8/site-packages/transformers/pipelines/pt_utils.py:111, in PipelineIterator.__next__(self)
    108     return self.loader_batch_item()
    110 # We're out of items within a batch
--> 111 item = next(self.iterator)
    112 processed = self.infer(item, **self.params)
    113 # We now have a batch of "inferred things".

File ~/miniconda3/envs/transformersenv/lib/python3.8/site-packages/transformers/pipelines/pt_utils.py:253, in PipelinePackIterator.__next__(self)
    250             return accumulator
    252 while not is_last:
--> 253     processed = self.infer(next(self.iterator), **self.params)
    254     if self.loader_batch_size is not None:
    255         if isinstance(processed, torch.Tensor):

File ~/miniconda3/envs/transformersenv/lib/python3.8/site-packages/torch/utils/data/dataloader.py:530, in _BaseDataLoaderIter.__next__(self)
    528 if self._sampler_iter is None:
    529     self._reset()
--> 530 data = self._next_data()
    531 self._num_yielded += 1
    532 if self._dataset_kind == _DatasetKind.Iterable and \
    533         self._IterableDataset_len_called is not None and \
    534         self._num_yielded > self._IterableDataset_len_called:

File ~/miniconda3/envs/transformersenv/lib/python3.8/site-packages/torch/utils/data/dataloader.py:570, in _SingleProcessDataLoaderIter._next_data(self)
    568 def _next_data(self):
    569     index = self._next_index()  # may raise StopIteration
--> 570     data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
    571     if self._pin_memory:
    572         data = _utils.pin_memory.pin_memory(data)

File ~/miniconda3/envs/transformersenv/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py:32, in _IterableDatasetFetcher.fetch(self, possibly_batched_index)
     30 for _ in possibly_batched_index:
     31     try:
---> 32         data.append(next(self.dataset_iter))
     33     except StopIteration:
     34         self.ended = True

File ~/miniconda3/envs/transformersenv/lib/python3.8/site-packages/transformers/pipelines/pt_utils.py:170, in PipelineChunkIterator.__next__(self)
    167     self.subiterator = self.infer(next(self.iterator), **self.params)
    168 try:
    169     # Try to return next item
--> 170     processed = next(self.subiterator)
    171 except StopIteration:
    172     # When a preprocess iterator ends, we can start lookig at the next item
    173     # ChunkIterator will keep feeding until ALL elements of iterator
   (...)
    176     # Another way to look at it, is we're basically flattening lists of lists
    177     # into a single list, but with generators
    178     self.subiterator = self.infer(next(self.iterator), **self.params)

File ~/miniconda3/envs/transformersenv/lib/python3.8/site-packages/transformers/pipelines/automatic_speech_recognition.py:191, in AutomaticSpeechRecognitionPipeline.preprocess(self, inputs, chunk_length_s, stride_length_s)
    188         inputs = f.read()
    190 if isinstance(inputs, bytes):
--> 191     inputs = ffmpeg_read(inputs, self.feature_extractor.sampling_rate)
    193 stride = None
    194 extra = {}

File ~/miniconda3/envs/transformersenv/lib/python3.8/site-packages/transformers/pipelines/audio_utils.py:35, in ffmpeg_read(bpayload, sampling_rate)
     33         output_stream = ffmpeg_process.communicate(bpayload)
     34 except FileNotFoundError as error:
---> 35     raise ValueError("ffmpeg was not found but is required to load audio files from filename") from error
     36 out_bytes = output_stream[0]
     37 audio = np.frombuffer(out_bytes, np.float32)

ValueError: ffmpeg was not found but is required to load audio files from filename

```

Just to clarify (since I updated Transformers) this is the output of running 'transformers-cli env' command:

```
- `transformers` version: 4.18.0
- Platform: Linux-5.13.0-39-generic-x86_64-with-glibc2.17
- Python version: 3.8.13
- Huggingface_hub version: 0.5.1
- PyTorch version (GPU?): 1.11.0+cu113 (True)
- Tensorflow version (GPU?): 2.8.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed

```


As the error suggests:

```
ValueError: ffmpeg was not found but is required to load audio files from filename
```

You need to load the files from filename to get a wave representation of sound. `ffmpeg` is leveraged in `transformers` since it covers a super large array of different files.

If you can't install `ffmpeg` for whatever reason, you need to find a way to get those soundfiles into a 1d array at the expected sampling rate of the model (usually 16k Hz). (Everything is taken care of for you if you have `ffmpeg` installed)

@patrickvonplaten @sgugger should we change that example to avoid relying on `ffmpeg` ? Or should we make it explicit ?
I like doing ASR very simply in this example, also casually dropping to CUDA with a single `device=0` but maybe for a quicktour we want something simpler ? (The first example is a classifier so requires basically nothing).

(For audio without `ffmpeg` we would still rely on `librosa` or `soundfile` both of which also require a library to function (like `libsndfile` , librosa can also use ffmpeg if present)). `libsndfile` is not necessarily always present on all systems

On another note.

- `superb` is `6Go`  which might be a little too much for a Quicktour, wdyt ? `datasets` is using `common_voice` , `tr` which is 600Mo: https://huggingface.co/docs/datasets/audio_process, maybe better ?  (Well `common_voice`, `en` is super large too, but maybe since we now have a lot of non English models it could be OK to promote non english
- `datasets` can also do the preprocessing (without the resampling) using `librosa`. So we can write `pip install datasets[audio]`, the `pipeline` can do the resampling using `torchaudio` which is included in the docs for audio in `datasets`. It is not optimal since it does require another resampling (while `ffmpeg` resamples as it decompresses and reads the file).

In the Quicktour making things as simple as possible trumps "no resampling" thing, but it's also good if we can avoid promoting slightly incorrect usages.

So to put more simply:

```python
!sudo apt-get install libsndfile
pip install datasets[audio]
pip install torchaudio librosa

from transformers import pipeline
import datasets
speech_recognizer = pipeline ("automatic-speech-recognition", model = "facebook/wav2vec2-base-960h" ,device = 0)
dataset = datasets.load_dataset("common_voice", name="tr", split="train")
files = dataset["audio"]
for f in files[:4]:
    # Pipeline expects the `raw` key for it's array while `datasets` uses `array`, we can make a change 
    # to accept both in `pipeline` since dicts are exclusively used for raw arrays and `sampling_rate` is already the correct
    # name in both.
    f["raw"] = f.pop("array")
print(speech_recognizer(files[:4]))
```
Pros:
- Easier on windows (where installing sndfile is not necessary I think)

Cons:
- Resampling under the hood


```python
!sudo apt-get install ffmpeg
pip install datasets

from transformers import pipeline
import datasets
speech_recognizer = pipeline ("automatic-speech-recognition", model = "facebook/wav2vec2-base-960h" ,device = 0)
dataset = datasets.load_dataset("common_voice", name="tr", split="train")
files = dataset["path"]
print(speech_recognizer(files[:4]))
```
Pros:
- No resampling
Cons:
- Installing ffmpeg is not simple necessarily on Windows (especially since we need it in the PATH to use it)


Hey @Narsil, 

I changed the quicktour example from `superb` to `minds14` which is only some hundred MB (PR is already merged).
https://huggingface.co/datasets/PolyAI/minds14 also has the `.wav` format which means we wouldn't need ffmpeg here. 

Will open a PR to change the dependency on `ffmpeg` - fully agree with you @Narsil !

PR to avoid `ffmpeg` and to make clear that resampling is needed in 90% of the cases: https://github.com/huggingface/transformers/pull/16723

Thank you guys for the detailed feedback on my issue!
Really appreciate it, especially as someone who was new to the Transformers library.