XLSR-53
Author: yagan93
Pull request: no
Labels: name: New model

# 🚀 Feature request

Is it possible to use XLSR-53 with transformers in the near future? 


Comments:
Apparently, someone [just did it](https://huggingface.co/facebook/wav2vec2-large-xlsr). But there are some files missing and it currently unusable. Hopefully the author will soon update it :)

Pinging @patrickvonplaten for knowledge :)

Yeah, I just added the pretrained checkpoint. I'll release a notebook by the middle/end of next week on how to fine-tune the checkpoint. Please ping me here again if you can't find it :-)

@patrickvonplaten Thanks a lot! Cant wait to use it and see how it performs :) 

@patrickvonplaten cool! is it possible to use with Transformers XLSR-53 finetuned with Fairseq?

will release a notebook either tomorrow or on Monday about it :-)

@patrickvonplaten can't wait :)

Notebook is available here: https://huggingface.co/blog/fine-tune-xlsr-wav2vec2 :-) 

We are organizing a "fine-tuning XLSR-53" event. Check this announcement: https://discuss.huggingface.co/t/open-to-the-community-xlsr-wav2vec2-fine-tuning-week-for-low-resource-languages/4467. Would be awesome if you want to participate :-)

@patrickvonplaten 

Hey buddy! 

First and foremost I want to thank you again for all your effort! Really appreciate it! 

Got another litte question:

Fine tuned a wav2vec-large-xlsr-53 model on Swiss German (bernese dialect) as laid out in one of your blogs.

Currently trying to add an already existing 6-Gram-KenLM on top.

Could you give me some hints on how to do it? Or is it yet not even possible?

Kind regards
Yves :wink:



Hey Yves, 

Here a forum post regarding this issue: https://discuss.huggingface.co/t/language-model-for-wav2vec2-0-decoding/4434

Hi all,
I am following up on this issue: I am trying to use the pre-trained Wav2Vec2-XLSR-53 (https://huggingface.co/facebook/wav2vec2-large-xlsr-53) and according to the documentation, it should be available as:

```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("facebook/wav2vec2-large-xlsr-53")
model = AutoModel.from_pretrained("facebook/wav2vec2-large-xlsr-53")
```
The model is available, but the tokenizer is not found (error: OSError: Can't load tokenizer for 'facebook/wav2vec2-large-xlsr-53'. Make sure that: (...) ). I tried using Transformers 4.2.2 and 4.5.0 as well as cloning the repository -- no luck. I am able to successfully load e.g. the French version:

`	tokenizer = AutoTokenizer.from_pretrained("facebook/wav2vec2-large-xlsr-53-french")
`

But not the base XLSR tokenizer? 

Thanks so much for the brilliant work!

Hey @gretatuckute 

Check out my HuggingFace Profile https://huggingface.co/Yves. There you'll find what you're after. 
If you ask @patrickvonplaten he could also invite you the wav2vec xlsr slack channel :) 

Cheers
Yves

Hi @yagan93, thank you for getting back! On your HF profile I only see the Swiss-German tokenizer? 

@gretatuckute You just got to swop the models and make little adjustments. Check out this notebook for details information on how to do so. 

Closed by https://github.com/huggingface/transformers/pull/10648