w2v-bert-2.0-Fleurs_AMMI_AFRIVOICE_LRSC-ln-50hrs-v3

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0509
• Wer: 0.1756
• Cer: 0.0552

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 80
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Cer	Validation Loss	Wer
2.0468	0.9996	1423	0.0860	0.4544	0.2791
0.4704	2.0	2847	0.0693	0.3196	0.2248
0.38	2.9996	4270	0.0709	0.2852	0.2316
0.3323	4.0	5694	0.0613	0.2762	0.1987
0.2975	4.9996	7117	0.0574	0.2650	0.1801
0.2638	6.0	8541	0.0683	0.2739	0.1973
0.2332	6.9996	9964	0.0648	0.2773	0.2008
0.2056	8.0	11388	0.0719	0.2810	0.2035
0.1845	8.9996	12811	0.0567	0.3252	0.1815
0.1665	10.0	14235	0.0594	0.3125	0.1836
0.1506	10.9996	15658	0.0585	0.3443	0.1830
0.1362	12.0	17082	0.0585	0.3741	0.1864
0.1233	12.9996	18505	0.0592	0.3922	0.1850
0.112	14.0	19929	0.0618	0.3927	0.1912
0.1014	14.9996	21352	0.0553	0.4532	0.1770
0.0923	16.0	22776	0.0576	0.4438	0.1872
0.0822	16.9996	24199	0.0641	0.4500	0.1991
0.0737	18.0	25623	0.0553	0.5237	0.1776
0.0664	18.9996	27046	0.0611	0.4709	0.1959
0.059	20.0	28470	0.0583	0.5256	0.1926
0.0528	20.9996	29893	0.0554	0.5598	0.1768
0.0469	22.0	31317	0.0594	0.5584	0.1861
0.041	22.9996	32740	0.0600	0.5737	0.1927
0.0364	24.0	34164	0.0563	0.6655	0.1802
0.0322	24.9996	35587	0.0556	0.6785	0.1796
0.0277	26.0	37011	0.0581	0.6534	0.1860
0.0252	26.9996	38434	0.0608	0.6609	0.1937
0.0222	28.0	39858	0.0562	0.7283	0.1796
0.0189	28.9996	41267	0.7351	0.1830	0.0577
0.0173	30.0	42691	0.7113	0.1829	0.0577
0.0154	30.9996	44114	0.7763	0.1769	0.0562
0.0138	32.0	45538	0.7698	0.1846	0.0582
0.0121	32.9996	46961	0.7890	0.1859	0.0583
0.011	34.0	48385	0.8291	0.1814	0.0570
0.01	34.9996	49808	0.8334	0.1794	0.0561
0.0088	36.0	51232	0.8611	0.1852	0.0573
0.0081	36.9996	52655	0.8689	0.1805	0.0566
0.0073	38.0	54079	0.8919	0.1801	0.0569
0.007	38.9996	55502	0.8902	0.1744	0.0548
0.0063	40.0	56926	0.9168	0.1815	0.0566
0.0058	40.9996	58349	0.9345	0.1780	0.0560
0.0051	42.0	59773	0.9237	0.1802	0.0562
0.0049	42.9996	61196	0.9519	0.1761	0.0553
0.0042	44.0	62620	0.9862	0.1750	0.0548
0.0041	44.9996	64043	0.9637	0.1777	0.0552
0.004	46.0	65467	0.9783	0.1768	0.0555
0.0034	46.9996	66890	0.9835	0.1799	0.0560
0.0032	48.0	68314	1.0196	0.1758	0.0550
0.003	48.9996	69737	0.9976	0.1804	0.0563
0.0025	50.0	71161	1.0553	0.1783	0.0557
0.0024	50.9996	72584	1.0059	0.1762	0.0555
0.0023	52.0	74008	1.0422	0.1760	0.0552
0.0021	52.9996	75431	1.0497	0.1758	0.0543
0.0019	54.0	76855	1.0509	0.1756	0.0552

Framework versions

• Transformers 4.46.3
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.20.3