khier12/800min_mms-1b_Full_FT_Algerian_Dialect

800min_mms-1b_Full_FT

This model is a fine-tuned version of facebook/mms-1b-all on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7149
• Wer: 0.4113
• Cer: 0.1265

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 32
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 18
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Cer	Validation Loss	Wer
1.0923	0.4578	100	0.2360	0.8702	0.7332
0.9113	0.9156	200	0.2291	0.8857	0.6870
0.7944	1.3734	300	0.1811	0.7011	0.5729
0.7395	1.8312	400	0.1754	0.6814	0.5626
0.6569	2.2890	500	0.1650	0.6441	0.5317
0.6721	2.7468	600	0.1641	0.6141	0.5396
0.559	3.2046	700	0.1540	0.6004	0.5004
0.5732	3.6624	800	0.1528	0.6227	0.4947
0.5754	4.1202	900	0.1497	0.5807	0.4892
0.5125	4.5780	1000	0.1468	0.5681	0.4839
0.5139	5.0358	1100	0.1427	0.5546	0.4725
0.4565	5.5036	1200	0.1431	0.5824	0.4598
0.4513	5.9614	1300	0.1415	0.5827	0.4635
0.4192	6.4192	1400	0.1415	0.5674	0.4626
0.4326	6.8770	1500	0.1388	0.5866	0.4592
0.3627	7.3348	1600	0.1422	0.5689	0.4570
0.3775	7.7926	1700	0.1400	0.5672	0.4513
0.3319	8.2504	1800	0.1374	0.5788	0.4402
0.3236	8.7082	1900	0.1364	0.5807	0.4435
0.3069	9.1720	2000	0.6232	0.4364	0.1348
0.3045	9.6298	2100	0.5903	0.4275	0.1304
0.3076	10.0876	2200	0.5934	0.4321	0.1336
0.2662	10.5454	2300	0.5858	0.4289	0.1306
0.2641	11.0031	2400	0.5754	0.4280	0.1299
0.2456	11.4609	2500	0.6478	0.4272	0.1312
0.2432	11.9187	2600	0.6221	0.4200	0.1283
0.2281	12.3765	2700	0.6166	0.4195	0.1291
0.2275	12.8343	2800	0.6176	0.4181	0.1297
0.2011	13.2921	2900	0.6225	0.4276	0.1347
0.2036	13.7499	3000	0.6234	0.4189	0.1294
0.1782	14.2077	3100	0.6835	0.4276	0.1331
0.178	14.6655	3200	0.6716	0.4226	0.1317
0.1857	15.1233	3300	0.6782	0.4105	0.1266
0.1736	15.5811	3400	0.6545	0.4164	0.1279
0.1724	16.0389	3500	0.6854	0.4101	0.1263
0.1554	16.4967	3600	0.7056	0.4082	0.1262
0.1536	16.9545	3700	0.7061	0.4095	0.1256
0.1542	17.4123	3800	0.7007	0.4139	0.1274
0.1385	17.8701	3900	0.7149	0.4113	0.1265

Framework versions

• Transformers 4.41.1
• Pytorch 2.9.0+cu126
• Datasets 2.21.0
• Tokenizers 0.19.1