Finetuned Wav2Vec2+LM Inference on my `wav` audio files 
Author: elsheikh21
Pull request: no

## Environment info
- `transformers` version: 4.17.0
- Platform: Windows-10-10.0.22000-SP0
- Python version: 3.8.13
- PyTorch version (GPU?): 1.9.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@patrickvonplaten

## Information

The model I am using (Wav2Vec2.0 Large XLS-R 53 English):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. I am [fine-tuning Wav2Vec with LM Head](https://huggingface.co/blog/fine-tune-wav2vec2-english)  using WikiText to produce tri-grams LM. I downloaded the fine-tuned model dir locally and tried to perform inference on my audio `.wav` file
2. Please find [here](https://drive.google.com/drive/folders/1IBUTglXLw4IX8uKC0qmGKKhkoCvc3s94?usp=sharing), model files, test audio file, and requirements.txt if needed to reproduce the problem

### Code snippet
```{python}
    import torchaudio
    import torch
    from transformers import Wav2Vec2ForCTC, Wav2Vec2ProcessorWithLM
    from os import getcwd
    from os.path import join as path_join


    model_name = "jonatasgrosman/wav2vec2-large-xlsr-53-english"
    model = Wav2Vec2ForCTC.from_pretrained(model_name)

    processor_path = path_join(getcwd(), "stt_assets", "stt_model")
    processor = Wav2Vec2ProcessorWithLM.from_pretrained(processor_path)

    audio_file = 'test_audio_send.wav'
    speech_array, sampling_rate = torchaudio.load(audio_file)

    resampler = torchaudio.transforms.Resample(48_000, 16_000)
    speech = resampler(speech_array).squeeze().numpy()

    features = processor(speech, sampling_rate=16_000, return_tensors="pt", padding=True)

    with torch.no_grad():
        logits = model(**features).logits

    transcription = processor.batch_decode(logits.numpy())
    stt_output = transcription.text[0].lower()

    print(stt_output)

```

## Expected behavior

- The expected behavior was to print out the transcription but `logits = model(**features).logits` raises the following Exception: 
`RuntimeError: Expected 3-dimensional input for 3-dimensional weight [512, 1, 10], but got 4-dimensional input of size [1, 1, 2, 50582] instead`




Comments:
Hey @elsheikh21,

We cannot really guarantee that costume models work correctly. For the future, could you instead please post a **minimal** reproducible code snippet here? E.g. the error happens in the line `model(**features).logits` and it should be necessary to download a whole directory to reproduce the error. Could you maybe try to shorten your code example to 4,5 lines including a path to download the `test_audio_sent.wav` file?

Regarding your error, the problem is that the audio file is not mono-channel but has a duo-channel format. E.g. the following snippet gives:

```py
import torchaudio

audio_file = 'test_audio_send.wav'
speech_array, sampling_rate = torchaudio.load(audio_file)

speech_array.shape
```

```
torch.Size([2, 151744])
```

So your code would probably work if you replace:

```py
speech_array, sampling_rate = torchaudio.load(audio_file)
```

by 

```py
speech_array, sampling_rate = torchaudio.load(audio_file)
speech_array = speech_array[0]
```


Hello @patrickvonplaten 

Thanks a lot for your help, I have thought that the code snippet I added in the question is reproducible, please correct me if I am wrong for future instances, I have added the `ipynb` that I followed to produce the `wav2vec2 with LM head` if needed

I have followed your correction and it is working fine, yet I have an error when I try to decode the output. 

The first one that when I use 
```
transcription = processor.batch_decode(logits.numpy())
```
 it gives me the following exception `cannot find context for 'fork'` which is related to multiprocessing IMHO, 

so I changed that to

```
transcription = processor.decode(logits.numpy())
```


 and this line raises the following exception 

```
Traceback (most recent call last):
  File "C:\Users\AhmedElSheikh\AppData\Local\JetBrains\Toolbox\apps\PyCharm-P\ch-0\213.6777.50\plugins\python\helpers\pydev\pydevd.py", line 1483, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:\Users\AhmedElSheikh\AppData\Local\JetBrains\Toolbox\apps\PyCharm-P\ch-0\213.6777.50\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "D:/SVC/stt_server_side/main_stt.py", line 69, in <module>
    main_stt()
  File "D:/SVC/stt_server_side/main_stt.py", line 55, in main_stt
    transcription = processor.decode(logits=logits_)
  File "C:\ProgramData\Miniconda3\envs\stt_ws\lib\site-packages\transformers\models\wav2vec2_with_lm\processing_wav2vec2_with_lm.py", line 462, in decode
    decoded_beams = self.decoder.decode_beams(
  File "C:\ProgramData\Miniconda3\envs\stt_ws\lib\site-packages\pyctcdecode\decoder.py", line 527, in decode_beams
    decoded_beams = self._decode_logits(
  File "C:\ProgramData\Miniconda3\envs\stt_ws\lib\site-packages\pyctcdecode\decoder.py", line 347, in _decode_logits
    char = self._idx2vocab[idx_char]
KeyError: 33
```

After some debugging I found the following, 
```
print(logits shape)
>>> (1, 157, 33) 
```

And my tokenizer vocabulary size
```
len(tokenizer.processor.get_vocab()) 
>>> 33
```
the vocabulary itself is

```
print(tokenizer,processor.get_vocab())
>>> '<pad>': 0, '
': 1, '
': 2, '<unk>': 3, '|': 4, "'": 5, '-': 6, 'a': 7, 'b': 8, 'c': 9, 'd': 10, 'e': 11, 'f': 12, 'g': 13, 'h': 14, 'i': 15, 'j': 16, 'k': 17, 'l': 18, 'm': 19, 'n': 20, 'o': 21, 'p': 22, 'q': 23, 'r': 24, 's': 25, 't': 26, 'u': 27, 'v': 28, 'w': 29, 'x': 30, 'y': 31, 'z': 32}
```

For future readers 
we had here 2 problems 

1. During transcribing the audio via my model was expecting 3D input and received 4D input  `[1, 1, 2, 50582]`
2. During decoding the model logits we had a problem that our model produced logits `shape=(1, 157, 33)`

**Solution**:
1. Input should have been parsed and resampled as follows 
```{python}
    import torchaudio 

    # audio_file = path/to/your/wav/audio/file
    speech_array, sampling_rate = torchaudio.load(audio_file)
    # Audio has to be mono-channel
    speech_array = speech_array[0]
    resample = torchaudio.transforms.Resample(sampling_rate, 16_000)
    speech = resample(speech_array).squeeze().numpy()

```

2. Passing audio to the model as follows

```{python}
    features = processor(speech, sampling_rate=req_sampling_rate, return_tensors="pt")

    logging.info("Transcribing the audio file")
    with torch.no_grad():
        _logits = model(**features).logits

    # so instead of logits_ of shape [1, 157, 33], it should be `[157, 33]`
    # in order to remove this `1`, I resorted to using `squeeze()`
    logits_ = _logits.numpy().squeeze()
   
    transcription = processor.decode(logits=logits_)
    print(transcription.text.lower())
```