wav2vec2-large-mms-1b-mos-V1

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2990
• Wer: 0.3782

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
14.4069	0.2786	100	1.0081	0.7762
1.1903	0.5571	200	0.7443	0.6266
0.9562	0.8357	300	0.6051	0.5392
0.9029	1.1142	400	0.5129	0.5056
0.7983	1.3928	500	0.4762	0.4717
0.7577	1.6713	600	0.4335	0.4736
0.7394	1.9499	700	0.3934	0.4486
0.6913	2.2284	800	0.3838	0.4390
0.6559	2.5070	900	0.3585	0.4425
0.6303	2.7855	1000	0.3425	0.4211
0.6078	3.0641	1100	0.3257	0.4019
0.591	3.3426	1200	0.3149	0.4031
0.5795	3.6212	1300	0.3019	0.3759
0.567	3.8997	1400	0.2990	0.3782

Framework versions

• Transformers 4.43.0.dev0
• Pytorch 2.1.2
• Datasets 2.20.0
• Tokenizers 0.19.1