SuperSindhi

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1850
• Wer: 0.1513

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
23.6951	0.2283	50	12.4335	1.0
10.0619	0.4566	100	7.2129	1.0
6.0237	0.6849	150	4.3896	1.0
3.7555	0.9132	200	3.4440	1.0
3.2614	1.1416	250	3.2118	1.0
3.1725	1.3699	300	3.1193	0.9995
3.0116	1.5982	350	2.7685	1.0
2.1438	1.8265	400	1.2964	0.7798
1.2369	2.0548	450	0.8161	0.5840
0.8896	2.2831	500	0.6958	0.5295
0.8004	2.5114	550	0.5918	0.4849
0.701	2.7397	600	0.4901	0.4161
0.6333	2.9680	650	0.4689	0.3978
0.4878	3.1963	700	0.4251	0.3819
0.501	3.4247	750	0.4009	0.3507
0.4599	3.6530	800	0.3477	0.3086
0.4426	3.8813	850	0.3219	0.2967
0.429	4.1096	900	0.3077	0.2722
0.3689	4.3379	950	0.2865	0.2609
0.3725	4.5662	1000	0.2880	0.2492
0.3478	4.7945	1050	0.2573	0.2293
0.336	5.0228	1100	0.2824	0.2536
0.2992	5.2511	1150	0.2744	0.2449
0.2775	5.4795	1200	0.2674	0.2464
0.274	5.7078	1250	0.2590	0.2387
0.2651	5.9361	1300	0.2678	0.2412
0.241	6.1644	1350	0.2602	0.2298
0.2539	6.3927	1400	0.2477	0.2246
0.2576	6.6210	1450	0.2346	0.2110
0.2626	6.8493	1500	0.2310	0.2138
0.2556	7.0776	1550	0.2365	0.2058
0.2271	7.3059	1600	0.2253	0.2003
0.1902	7.5342	1650	0.2013	0.1812
0.1633	7.7626	1700	0.1960	0.1706
0.1557	7.9909	1750	0.1920	0.1694
0.1555	8.2192	1800	0.1911	0.1680
0.1514	8.4475	1850	0.2003	0.1640
0.146	8.6758	1900	0.1860	0.1592
0.1269	8.9041	1950	0.1920	0.1569
0.1314	9.1324	2000	0.1872	0.1553
0.1247	9.3607	2050	0.1874	0.1549
0.1262	9.5890	2100	0.1852	0.1515
0.1201	9.8174	2150	0.1850	0.1513

Framework versions

• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0