Decoding with Wav2Vec2 with Language Model
Author: ypirkani
Pull request: no

Dear All, 

I am trying to decode a model with a Language Model:

I made a LM using KenLM from the method provided at : https://huggingface.co/blog/wav2vec2-with-ngram

When decoding, I am getting the following error:
ValueError: Input logits of size 586, but vocabulary is size 44


An brief Introduction:
I am training fine-tuning wav2vec2_base in Urdu Language.
I trained it and tested it without using Language-Model and it was giving results. 

Now I made a new processor using the following code:
```
from transformers import Wav2Vec2ProcessorWithLM
decoder = build_ctcdecoder(
    labels=list(sorted_vocab_dict.keys()),
    kenlm_model_path="5gram_correct_1.arpa",
)

processor_lm = Wav2Vec2ProcessorWithLM(feature_extractor=feature_extractor, tokenizer=tokenizer , decoder=decoder)

```
The LM is made from the same Textual data which was used in training the model.

When I try to decode the model it is giving me the following error:
`ValueError: Input logits of size 586, but vocabulary is size 44`

Any idea why am I getting this error? 

Comments:
Have you checked that you are using the same vocabulary for both and what code are you using for inference?

> Have you checked that you are using the same vocabulary for both and what code are you using for inference?

Hi olafthiele,

I am making my arpa file from the same text I am using for training the model and generating the processor.

for inference code is following,

`import soundfile as sf
import torch
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import os

from transformers import AutoProcessor, AutoModelForCTC

processor = AutoProcessor.from_pretrained("yapak1994/Urdu_ASR_wav2vec2_base_Model")

model = AutoModelForCTC.from_pretrained("yapak1994/Urdu_ASR_wav2vec2_base_Model")`

`path = "urdu/"

#print(os.listdir(path))
for file in os.listdir(path):
# load audio
	audio_input, sample_rate = sf.read(path + file)

	# pad input values and return pt tensor
	input_values = processor(audio_input, sampling_rate=sample_rate, return_tensors="pt").input_values

	# INFERENCE

	# retrieve logits & take argmax
	logits = model(input_values).logits
	predicted_ids = torch.argmax(logits, dim=-1)

	# transcribe
	transcription = processor.decode(predicted_ids[0])
#	print(transcription)
	print(transcription + " (" + file + ")" )
	`

This code is working correctly and giving me output.

Now 

When I use merge this model with my LM.arpa

`from transformers import AutoProcessor, AutoModelForCTC

processor = AutoProcessor.from_pretrained("yapak1994/Urdu_ASR_wav2vec2_base_Model")

model = AutoModelForCTC.from_pretrained("yapak1994/Urdu_ASR_wav2vec2_base_Model")`


`from transformers import Wav2Vec2ProcessorWithLM
decoder = build_ctcdecoder(
    labels=list(sorted_vocab_dict.keys()),
    kenlm_model_path="yapak1994/Urdu_ASR_wav2vec2_base_Model/5gram_correct_1.arpa ",
)

processor_lm = Wav2Vec2ProcessorWithLM(feature_extractor=feature_extractor, tokenizer=tokenizer , decoder=decoder)`

Now when I try to repeat the same code for previous inference, I get error.
`
ValueError: Input logits of size 586, but vocabulary is size 44`

`for file in os.listdir(path):
# load audio
	audio_input, sample_rate = sf.read(path + file)

	# pad input values and return pt tensor
	input_values = processor(audio_input, sampling_rate=sample_rate, return_tensors="pt").input_values

	# INFERENCE

	# retrieve logits & take argmax
	logits = model(input_values).logits
	predicted_ids = torch.argmax(logits, dim=-1)

	# transcribe
	transcription = processor_lm.decode(predicted_ids[0])
#	print(transcription)
	print(transcription + " (" + file + ")" )`



You have some markdown problems, but have you tried building the arpa according to [the blog](https://huggingface.co/blog/wav2vec2-with-ngram)?

And then do inferencing [like I did here](https://github.com/huggingface/transformers/issues/15344). Works correctly now with the pool closing.

Thanks a lot, 

I think there were error while building the file. Now it is working correctly.

Thanks a lot to @olafthiele  and @patrickvonplaten for their wonderful help.