xho_finetune

This model is a fine-tuned version of Akashpb13/Swahili_xlsr on the ml-superb-subset dataset. It achieves the following results on the evaluation set:

• Loss: 0.5370
• Wer: 53.5109

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 9.6e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 25
• training_steps: 500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
25.5184	0.7692	10	24.2275	100.0
14.5363	1.5385	20	9.8357	100.0
4.5811	2.3077	30	3.8367	100.0
3.4822	3.0769	40	3.3922	100.0
3.2732	3.8462	50	3.2398	100.0
3.1796	4.6154	60	3.1705	100.0
3.1504	5.3846	70	3.1419	100.0
3.1119	6.1538	80	3.1084	100.0
3.0789	6.9231	90	3.0735	100.0
3.0619	7.6923	100	3.0590	100.0
3.0298	8.4615	110	3.0247	100.0
2.9933	9.2308	120	2.9716	100.0
2.9079	10.0	130	2.8647	100.0
2.8414	10.7692	140	2.7931	100.0
2.6939	11.5385	150	2.5932	100.0
2.3274	12.3077	160	2.1000	99.7579
1.7068	13.0769	170	1.4580	93.4625
1.206	13.8462	180	1.1027	83.0508
0.9587	14.6154	190	0.9152	79.4189
0.7806	15.3846	200	0.8122	69.7337
0.7118	16.1538	210	0.7445	69.0073
0.6814	16.9231	220	0.6945	62.9540
0.5709	17.6923	230	0.6787	67.5545
0.5653	18.4615	240	0.6758	62.2276
0.5437	19.2308	250	0.6511	60.7748
0.5092	20.0	260	0.6237	62.7119
0.4239	20.7692	270	0.6000	61.5012
0.4355	21.5385	280	0.5899	59.8063
0.4456	22.3077	290	0.5960	59.3220
0.3986	23.0769	300	0.5764	56.6586
0.3856	23.8462	310	0.5801	55.9322
0.3607	24.6154	320	0.5682	57.6271
0.358	25.3846	330	0.5675	55.9322
0.3452	26.1538	340	0.5630	57.8692
0.3289	26.9231	350	0.5515	57.8692
0.353	27.6923	360	0.5621	57.3850
0.2907	28.4615	370	0.5486	55.2058
0.3237	29.2308	380	0.5445	54.4794
0.3202	30.0	390	0.5384	52.7845
0.2918	30.7692	400	0.5370	55.6901
0.3106	31.5385	410	0.5422	53.7530
0.3105	32.3077	420	0.5438	55.2058
0.2835	33.0769	430	0.5437	55.9322
0.2966	33.8462	440	0.5416	54.7215
0.2719	34.6154	450	0.5394	54.2373
0.2859	35.3846	460	0.5384	53.7530
0.29	36.1538	470	0.5379	53.2688
0.2879	36.9231	480	0.5372	53.5109
0.2871	37.6923	490	0.5370	53.5109
0.3019	38.4615	500	0.5370	53.5109

Framework versions

• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1