ArthurMalajyan/seamless-m4t-v2-large-asr-hyw

SeamlessM4T v2 ASR for Western Armenian

This model is a fine-tuned version of the facebook/seamless-m4t-v2-large for the ASR task. Initially, it was fine-tuned on the Common Voice 16.1 and Google Fleurs datasets. Subsequently, it was further fine-tuned on the ReRooted corpus. The model achieves the following results on the test sets:

• CV_wer: 0.308
• CV_cer: 0.07
• GF_wer: 0.311
• GF_cer: 0.094

After fine-tuning on Western Armenian data, the model occasionally translates Eastern Armenian speech into Western Armenian(Colab link: Test SeamlessM4T v2 ASR for Western Armenian).

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-6
• train_batch_size: 4
• eval_batch_size: 1
• seed: 43
• optimizer: Adam with betas=(0.9, 0.98) and epsilon=1e-08
• lr_scheduler_type: MyleLR
• lr_scheduler_warmup_steps: 100

Framework versions

• Pytorch 2.1.1
• Fairseq2 0.2.0

Citation

For SeamlessM4T v2, please cite :

@inproceedings{seamless2023,
   title="Seamless: Multilingual Expressive and Streaming Speech Translation",
   author="{Seamless Communication}, Lo{\"i}c Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Mark Duppenthaler, Paul-Ambroise Duquenne, Brian Ellis, Hady Elsahar, Justin Haaheim, John Hoffman, Min-Jae Hwang, Hirofumi Inaguma, Christopher Klaiber, Ilia Kulikov, Pengwei Li, Daniel Licht, Jean Maillard, Ruslan Mavlyutov, Alice Rakotoarison, Kaushik Ram Sadagopan, Abinesh Ramakrishnan, Tuan Tran, Guillaume Wenzek, Yilin Yang, Ethan Ye, Ivan Evtimov, Pierre Fernandez, Cynthia Gao, Prangthip Hansanti, Elahe Kalbassi, Amanda Kallet, Artyom Kozhevnikov, Gabriel Mejia, Robin San Roman, Christophe Touret, Corinne Wong, Carleigh Wood, Bokai Yu, Pierre Andrews, Can Balioglu, Peng-Jen Chen, Marta R. Costa-juss{\`a}, Maha Elbayad, Hongyu Gong, Francisco Guzm{\'a}n, Kevin Heffernan, Somya Jain, Justine Kao, Ann Lee, Xutai Ma, Alex Mourachko, Benjamin Peloquin, Juan Pino, Sravya Popuri, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, Anna Sun, Paden Tomasello, Changhan Wang, Jeff Wang, Skyler Wang, Mary Williamson",
  journal={ArXiv},
  year={2023}
}