mms-1b-lozgen-female-model

This model is a fine-tuned version of facebook/mms-1b-all on the LOZGEN - LOZ dataset. It achieves the following results on the evaluation set:

• Loss: 0.6218
• Wer: 0.3955

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.7809	0.8696	100	3.1491	0.9759
2.6957	1.7391	200	2.0011	0.9278
1.1829	2.6087	300	0.8064	0.6421
0.8527	3.4783	400	0.7470	0.5333
0.8376	4.3478	500	0.6996	0.4923
0.7567	5.2174	600	0.6792	0.4686
0.7014	6.0870	700	0.6882	0.4555
0.7242	6.9565	800	0.6557	0.4578
0.6895	7.8261	900	0.6532	0.4409
0.6543	8.6957	1000	0.6477	0.4278
0.6386	9.5652	1100	0.6592	0.4219
0.6145	10.4348	1200	0.6435	0.4257
0.6513	11.3043	1300	0.6570	0.4147
0.5788	12.1739	1400	0.6259	0.4086
0.6061	13.0435	1500	0.6179	0.4006
0.5647	13.9130	1600	0.6186	0.4006
0.5715	14.7826	1700	0.6271	0.3985
0.5502	15.6522	1800	0.6218	0.3962

Framework versions

• Transformers 4.48.0.dev0
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0