wav2vec2 : Speech to text conversion fails when using large file 
Author: iamshreeram
Pull request: no

I'm trying to get the translation of an English Speech ( WAV file : 1.2 GB); But, Seeing error due to vector size. Same code works when the file size is low.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.17.0
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.8.12
- PyTorch version (GPU?): 1.11.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help

Models: - Wav2Vec2: @patrickvonplaten, @anton-l

<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models - Wav2Vec2: @patrickvonplaten, @anton-l

- ALBERT, BERT, XLM, DeBERTa, DeBERTa-v2, ELECTRA, MobileBert, SqueezeBert: @LysandreJik
- T5, Pegasus, EncoderDecoder: @patrickvonplaten
- Blenderbot, MBART, BART, Marian, Pegasus: @patil-suraj
- Reformer, TransfoXL, XLNet, FNet: @patrickvonplaten
- Longformer, BigBird: @ydshieh
- FSMT: @stas00
- Funnel: @sgugger
- GPT-2, GPT: @patil-suraj, @patrickvonplaten, @LysandreJik
- RAG, DPR: @patrickvonplaten, @lhoestq
- TensorFlow: @Rocketknight1
- JAX/Flax: @patil-suraj
- TAPAS, LayoutLM, LayoutLMv2, LUKE, ViT, BEiT, DEiT, DETR, CANINE: @NielsRogge
- GPT-Neo, GPT-J, CLIP: @patil-suraj
- Wav2Vec2, HuBERT, SpeechEncoderDecoder, UniSpeech, UniSpeechSAT, SEW, SEW-D, Speech2Text: @patrickvonplaten, @anton-l

If the model isn't in the list, ping @LysandreJik who will redirect you to the correct contributor.

Library:

- Benchmarks: @patrickvonplaten
- Deepspeed: @stas00
- Ray/raytune: @richardliaw, @amogkam
- Text generation: @patrickvonplaten @narsil
- Tokenizers: @SaulLu
- Trainer: @sgugger
- Pipelines: @Narsil
- Speech: @patrickvonplaten, @anton-l
- Vision: @NielsRogge, @sgugger

Documentation: @sgugger

Model hub:

- for issues with a model, report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj

For research projetcs, please ping the contributor directly. For example, on the following projects:

- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...): [Wav2Vec2](https://huggingface.co/docs/transformers/model_doc/wav2vec2)

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Download a english speech from youtube as mp3
2. Convert the mp3 to wav (almost 1.2 GB file size)
3. Use wav2vec2 to provide speech to text translation 
4. But, The model fails due to file length or the Frame rate. Is there any limitation on these vectors?
5. Further details of the `.wav` file

Metadata of the `.wav` file for which the model works :
```
Framerate :  8000
Channel info :  1
Bytes/sample :  2
Maximum amplitude :  32767
Length of audio :  51028
```
Metadata of the `.wav` file for which the model doesn't work :
```
Framerate :  44100
Channel info :  2
Bytes/sample :  2
Maximum amplitude :  32768
Length of audio :  5901073
``` 

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

### Stack Trace :
```
Traceback (most recent call last):
  File "main.py", line 59, in <module>
    logits = model(input_values)["logits"]
  File "/Applications/anaconda3/envs/speechml/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Applications/anaconda3/envs/speechml/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py", line 1751, in forward
    outputs = self.wav2vec2(
  File "/Applications/anaconda3/envs/speechml/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Applications/anaconda3/envs/speechml/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py", line 1347, in forward
    extract_features = self.feature_extractor(input_values)
  File "/Applications/anaconda3/envs/speechml/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Applications/anaconda3/envs/speechml/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py", line 515, in forward
    hidden_states = conv_layer(hidden_states)
  File "/Applications/anaconda3/envs/speechml/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Applications/anaconda3/envs/speechml/lib/python3.8/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py", line 415, in forward
    hidden_states = self.conv(hidden_states)
  File "/Applications/anaconda3/envs/speechml/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Applications/anaconda3/envs/speechml/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 302, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/Applications/anaconda3/envs/speechml/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 298, in _conv_forward
    return F.conv1d(input, weight, bias, self.stride,
RuntimeError: Expected 2D (unbatched) or 3D (batched) input to conv1d, but got input of size: [1, 1, 2, 94417166]
```

## Expected behavior
<!-- A clear and concise description of what you would expect to happen. -->
To print the text that is converted from the audio


Comments:
Hello @iamshreeram!

The error actually refers to the shape of the input: it should be `(batch_size, 1, sequence_length)`, meaning that the inputs have to be single-channel arrays (mono audio) while your second file has 2 channels (stereo audio).

But anyways, the model itself won't be able to run on the whole 1.2GB file since it's too long for a single batch (you'll see a memory error). For such long files we have an ASR pipeline with chunked inference, which you can learn about in this tutorial: https://huggingface.co/blog/asr-chunking 
The pipeline will handle stereo-to-mono conversion too, so you'll just have to specify an input filename.

Let me know if it works for you :) 

Hi, I have converted from stereo to mono (400MB) but inference still fails with below error. 
```
fish: Job 1, 'python main.py' terminated by signal SIGKILL (Forced quit)
```
I believe OS is terminating the process due to heavy memory usage. What is the max size limit for this inference to work? 



Hi! Could you please post a code snippet that shows the `pipeline` parameters that you've used for inference here? :)

Also see docs here: https://huggingface.co/docs/transformers/main/en/main_classes/pipelines#transformers.AutomaticSpeechRecognitionPipeline.chunk_length_s

Thanks for quick reply. The memory error was while performing inference with logits 
```
logits = model(input_values)["logits"]
logits.shape
```

While using `pipeline`, It took almost an hour to process but the output was empty.

Below is the snippet - 
```
from transformers import pipeline
import os
import logging

logging.basicConfig(
    format='%(asctime)s %(levelname)-8s %(message)s',
    level=logging.INFO,
    datefmt='%Y-%m-%d %H:%M:%S')

logging.info("Decoding the audio files..")
pipe = pipeline(model="facebook/wav2vec2-base-960h")
audio_url = "youtube/stevejobs-speech-mono.wav"
logging.info("Starting pipeline..")
output = pipe(audio_url, chunk_length_s=10, stride_length_s=(4, 2))
logging.info("Converted text :", output)
```

Output - 
```
2022-04-25 20:36:25 INFO     Decoding the audio files..
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.05k/2.05k [00:00<00:00, 688kB/s]
Some weights of Wav2Vec2ForCTC were not initialized from the model checkpoint at facebook/wav2vec2-base-960h and are newly initialized: ['wav2vec2.masked_spec_embed']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
2022-04-25 20:36:40 INFO     Starting pipeline..
2022-04-25 21:25:26 INFO     Converted text :
```

Hey @iamshreeram,

Can you maybe upload the audio of the video to somewhere it can be downloaded?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.