anvitamanne/small-final-model-ishaan

wav2vec2-kaggle-final

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on the None dataset. It achieves the following results on the evaluation set:

• Loss: 467.6487
• Wer: 0.3843
• Cer: 0.1615

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1980.0484	0.17	200	1881.8782	1.0	1.0
1857.5231	0.34	400	1847.1379	1.0	1.0
1333.847	0.52	600	993.8895	0.9733	0.4174
990.9649	0.69	800	772.9453	0.8053	0.2881
822.3509	0.86	1000	699.9114	0.7153	0.2577
853.1906	1.03	1200	588.0330	0.6488	0.2377
681.4223	1.21	1400	595.6282	0.6082	0.2213
680.3315	1.38	1600	533.0568	0.5844	0.2104
612.303	1.55	1800	530.6596	0.5911	0.2142
628.1575	1.72	2000	504.1773	0.5508	0.2025
594.902	1.89	2200	483.2587	0.5422	0.1984
615.5168	2.07	2400	482.7257	0.5196	0.1939
552.6289	2.24	2600	487.7754	0.5168	0.1929
585.4439	2.41	2800	548.5886	0.5305	0.1981
531.7139	2.58	3000	593.6019	0.5795	0.2327
497.7652	2.76	3200	679.6472	0.5966	0.2382
479.2212	2.93	3400	566.4340	0.5237	0.1963
532.7164	3.1	3600	510.3879	0.4811	0.1846
508.853	3.27	3800	499.0350	0.4631	0.1798
502.5701	3.44	4000	557.4500	0.4885	0.1914
466.4071	3.62	4200	504.6231	0.4503	0.1766
521.5373	3.79	4400	816.4122	0.6920	0.2705
521.7708	3.96	4600	542.0289	0.4748	0.1827
434.5645	4.13	4800	483.0985	0.4563	0.1773
478.0779	4.3	5000	573.2537	0.4733	0.1826
490.7463	4.48	5200	448.2722	0.4495	0.1769
420.8497	4.65	5400	575.9213	0.4622	0.1794
444.1535	4.82	5600	536.1183	0.4689	0.1831
417.559	4.99	5800	526.9570	0.4678	0.1793
462.6115	5.17	6000	553.8561	0.5061	0.1966
472.7057	5.34	6200	473.8315	0.4402	0.1738
474.7029	5.51	6400	574.5921	0.5107	0.2021
422.9327	5.68	6600	533.9930	0.4616	0.1770
454.3927	5.85	6800	485.6865	0.4312	0.1717
414.617	6.03	7000	477.8979	0.4185	0.1709
431.8193	6.2	7200	577.0553	0.4742	0.1834
457.5081	6.37	7400	599.4647	0.4889	0.1939
410.7263	6.54	7600	495.1211	0.4386	0.1749
409.3561	6.72	7800	558.3831	0.4671	0.1846
419.5535	6.89	8000	521.3979	0.4745	0.1880
416.9934	7.06	8200	625.8699	0.5109	0.1989
402.1264	7.23	8400	624.1210	0.4906	0.1943
401.3531	7.4	8600	577.6796	0.4601	0.1828
393.8983	7.58	8800	539.1487	0.4512	0.1793
404.0746	7.75	9000	681.1195	0.5745	0.2427
367.9037	7.92	9200	600.4460	0.4890	0.1933
412.4216	8.09	9400	515.9341	0.4482	0.1764
412.5161	8.27	9600	496.5386	0.4204	0.1710
386.1062	8.44	9800	564.3361	0.4805	0.1899
373.9701	8.61	10000	534.6701	0.4569	0.1807
386.8188	8.78	10200	569.4644	0.4458	0.1769
353.7339	8.95	10400	586.9103	0.4924	0.1937
348.9872	9.13	10600	584.7791	0.4717	0.1890
370.6689	9.3	10800	593.6466	0.4604	0.1794
373.6055	9.47	11000	517.7433	0.4176	0.1664
397.119	9.64	11200	517.0852	0.4353	0.1735
348.3093	9.81	11400	542.9993	0.4342	0.1750
392.0189	9.99	11600	518.7840	0.4343	0.1732
331.1382	10.16	11800	520.4122	0.4155	0.1677
333.4658	10.33	12000	562.0241	0.4370	0.1745
386.7848	10.5	12200	523.9895	0.4181	0.1685
334.7904	10.68	12400	508.0610	0.4249	0.1708
371.6553	10.85	12600	569.8074	0.4587	0.1826
356.7725	11.02	12800	512.5233	0.4124	0.1672
336.8186	11.19	13000	526.1908	0.4311	0.1725
361.0486	11.36	13200	513.0122	0.4213	0.1711
317.7533	11.54	13400	475.2107	0.4128	0.1670
410.9716	11.71	13600	449.9915	0.4006	0.1650
355.4503	11.88	13800	455.7285	0.3945	0.1639
356.4254	12.05	14000	497.9339	0.4050	0.1665
344.9908	12.23	14200	483.6688	0.4072	0.1674
347.5455	12.4	14400	480.7558	0.3973	0.1646
348.6302	12.57	14600	491.5127	0.4001	0.1658
324.9071	12.74	14800	499.9131	0.4052	0.1669
344.9909	12.91	15000	487.6642	0.3919	0.1637
271.2929	13.09	15200	490.5817	0.3936	0.1637
340.6752	13.26	15400	488.0888	0.4021	0.1653
332.469	13.43	15600	483.0107	0.4040	0.1677
379.9843	13.6	15800	464.2564	0.3887	0.1624
324.5989	13.78	16000	469.0699	0.3923	0.1626
342.6454	13.95	16200	466.5754	0.3822	0.1605
299.187	14.12	16400	488.1742	0.3911	0.1627
352.0391	14.29	16600	481.3880	0.3872	0.1612
292.35	14.46	16800	471.6863	0.3864	0.1620
333.7441	14.64	17000	469.7429	0.3879	0.1618
318.1604	14.81	17200	468.5455	0.3846	0.1616
284.3848	14.98	17400	467.6487	0.3843	0.1615

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.0+cu118
• Datasets 2.16.0
• Tokenizers 0.15.2