wav2vec2-large-mms-1b-somalia

This model is a fine-tuned version of facebook/mms-1b-all on the xtreme_s dataset. It achieves the following results on the evaluation set:

• Loss: 0.5553
• Wer: 0.4330

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.5097	0.0558	100	0.6980	0.4802
0.4771	0.1117	200	0.6315	0.4613
0.4098	0.1675	300	0.6112	0.4519
0.4555	0.2233	400	0.6085	0.4540
0.4319	0.2792	500	0.6041	0.4614
0.4582	0.3350	600	0.5927	0.4526
0.3879	0.3908	700	0.6079	0.4530
0.3919	0.4467	800	0.6135	0.4479
0.4245	0.5025	900	0.5879	0.4461
0.3872	0.5583	1000	0.6088	0.4627
0.3843	0.6142	1100	0.5939	0.4451
0.3842	0.6700	1200	0.6133	0.4631
0.3771	0.7259	1300	0.6026	0.4510
0.4458	0.7817	1400	0.5965	0.4494
0.4114	0.8375	1500	0.5950	0.4472
0.4135	0.8934	1600	0.6106	0.4613
0.3763	0.9492	1700	0.6082	0.4426
0.442	1.0050	1800	0.5922	0.4482
0.381	1.0609	1900	0.5827	0.4419
0.3874	1.1167	2000	0.5816	0.4518
0.4602	1.1725	2100	0.5888	0.4453
0.3709	1.2284	2200	0.5870	0.4447
0.3991	1.2842	2300	0.5749	0.4426
0.3771	1.3400	2400	0.5743	0.4396
0.351	1.3959	2500	0.5956	0.4414
0.358	1.4517	2600	0.5772	0.4368
0.4076	1.5075	2700	0.5931	0.4384
0.4322	1.5634	2800	0.5795	0.4407
0.359	1.6192	2900	0.5814	0.4408
0.3714	1.6750	3000	0.5794	0.4366
0.407	1.7309	3100	0.5673	0.4404
0.3803	1.7867	3200	0.5754	0.4453
0.3412	1.8425	3300	0.5974	0.4470
0.3826	1.8984	3400	0.5847	0.4422
0.3961	1.9542	3500	0.5800	0.4417
0.4035	2.0101	3600	0.5804	0.4395
0.3988	2.0659	3700	0.5626	0.4379
0.3642	2.1217	3800	0.5745	0.4393
0.3543	2.1776	3900	0.5786	0.4414
0.3577	2.2334	4000	0.5736	0.4376
0.3818	2.2892	4100	0.5745	0.4412
0.3538	2.3451	4200	0.5856	0.4370
0.3902	2.4009	4300	0.5708	0.4370
0.3915	2.4567	4400	0.5819	0.4372
0.3918	2.5126	4500	0.5620	0.4366
0.3716	2.5684	4600	0.5801	0.4497
0.3696	2.6242	4700	0.5640	0.4321
0.3532	2.6801	4800	0.5772	0.4418
0.4039	2.7359	4900	0.5660	0.4388
0.3488	2.7917	5000	0.5666	0.4364
0.3482	2.8476	5100	0.5662	0.4428
0.3793	2.9034	5200	0.5620	0.4368
0.3808	2.9592	5300	0.5559	0.4387
0.3403	3.0151	5400	0.5647	0.4387
0.3654	3.0709	5500	0.5515	0.4367
0.3614	3.1267	5600	0.5593	0.4410
0.3399	3.1826	5700	0.5604	0.4353
0.3627	3.2384	5800	0.5559	0.4379
0.3583	3.2942	5900	0.5564	0.4343
0.3897	3.3501	6000	0.5593	0.4362
0.3523	3.4059	6100	0.5578	0.4345
0.3412	3.4618	6200	0.5586	0.4346
0.3877	3.5176	6300	0.5529	0.4346
0.3605	3.5734	6400	0.5577	0.4346
0.3869	3.6293	6500	0.5522	0.4332
0.3906	3.6851	6600	0.5591	0.4312
0.3469	3.7409	6700	0.5585	0.4351
0.3665	3.7968	6800	0.5554	0.4333
0.3851	3.8526	6900	0.5543	0.4346
0.3195	3.9084	7000	0.5558	0.4348
0.346	3.9643	7100	0.5553	0.4330

Framework versions

• Transformers 4.48.0.dev0
• Pytorch 2.1.0+cu118
• Datasets 3.2.0
• Tokenizers 0.21.0