480min_mms-1b_Full_FT

This model is a fine-tuned version of facebook/mms-1b-all on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7791
• Wer: 0.4490
• Cer: 0.1380

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 32
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.0493	0.7630	100	0.8570	0.7046	0.2250
0.9068	1.5260	200	0.7564	0.6409	0.2011
0.8394	2.2890	300	0.6806	0.5957	0.1846
0.7225	3.0520	400	0.6884	0.5615	0.1778
0.6612	3.8150	500	0.6458	0.5293	0.1643
0.5954	4.5780	600	0.6354	0.5258	0.1625
0.5308	5.3410	700	0.6337	0.5139	0.1592
0.5339	6.1040	800	0.6460	0.5076	0.1575
0.473	6.8670	900	0.6282	0.5035	0.1548
0.434	7.6299	1000	0.6319	0.5026	0.1550
0.3966	8.3929	1100	0.6413	0.4928	0.1518
0.3856	9.1559	1200	0.6590	0.4920	0.1523
0.3741	9.9189	1300	0.6090	0.4784	0.1473
0.3267	10.6819	1400	0.6900	0.4770	0.1451
0.2927	11.4449	1500	0.6587	0.4624	0.1425
0.275	12.2079	1600	0.6774	0.4711	0.1459
0.2787	12.9709	1700	0.6770	0.4689	0.1427
0.2343	13.7339	1800	0.7046	0.4632	0.1422
0.2193	14.4969	1900	0.7158	0.4573	0.1409
0.2138	15.2599	2000	0.7665	0.4556	0.1400
0.2088	16.0229	2100	0.7361	0.4518	0.1386
0.1919	16.7859	2200	0.7392	0.4514	0.1379
0.1766	17.5489	2300	0.7764	0.4515	0.1400
0.1756	18.3119	2400	0.7922	0.4531	0.1385
0.1785	19.0749	2500	0.7789	0.4462	0.1377
0.1647	19.8379	2600	0.7791	0.4490	0.1380

Framework versions

• Transformers 4.41.1
• Pytorch 2.9.0+cu126
• Datasets 2.21.0
• Tokenizers 0.19.1