xls-r-300m-model-G

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.7710
• Wer: 1.0
• Cer: 0.8800

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 60
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
32.1048	1.0	47	31.2303	1.0	1.2082
30.3148	2.0	94	25.3574	1.0	1.0
24.0771	3.0	141	14.3321	1.0	1.0
15.5757	4.0	188	8.7981	1.0	1.0
10.6789	5.0	235	6.3534	1.0	1.0
7.5286	6.0	282	5.0039	1.0	1.0
5.5091	7.0	329	4.2043	1.0	1.0
4.4551	8.0	376	3.7495	1.0	1.0
3.9291	9.0	423	3.4757	1.0	1.0
3.5432	10.0	470	3.3164	1.0	1.0
3.3702	11.0	517	3.2356	1.0	1.0
3.2758	12.0	564	3.1968	1.0	1.0
3.2236	13.0	611	3.1779	1.0	1.0
3.1857	14.0	658	3.1688	1.0	1.0
3.1704	15.0	705	3.1606	1.0	1.0
3.1714	16.0	752	3.1593	1.0	1.0
3.1537	17.0	799	3.1548	1.0	1.0
3.1519	18.0	846	3.1532	1.0	1.0
3.1496	19.0	893	3.1535	1.0	1.0
3.1441	20.0	940	3.1499	1.0	1.0
3.1456	21.0	987	3.1489	1.0	1.0
3.1409	22.0	1034	3.1480	1.0	1.0
3.1438	23.0	1081	3.1459	1.0	1.0
3.1382	24.0	1128	3.1447	1.0	1.0
3.1377	25.0	1175	3.1451	1.0	1.0
3.1306	26.0	1222	3.1409	1.0	1.0
3.1381	27.0	1269	3.1407	1.0	1.0
3.1358	28.0	1316	3.1393	1.0	1.0
3.139	29.0	1363	3.1333	1.0	1.0
3.1216	30.0	1410	3.1159	1.0	1.0
3.1206	31.0	1457	3.0811	1.0	1.0
3.081	32.0	1504	3.0663	1.0	1.0
3.0502	33.0	1551	3.1086	1.0	1.0
3.0215	34.0	1598	3.0463	1.0	1.0
2.9894	35.0	1645	2.9205	1.0	1.0
2.9546	36.0	1692	2.9391	1.0	1.0
2.9263	37.0	1739	2.8987	1.0	1.0
2.9039	38.0	1786	2.8631	1.0	1.0
2.8882	39.0	1833	2.8381	1.0	0.9965
2.8664	40.0	1880	2.8295	1.0	0.9936
2.8564	41.0	1927	2.8181	1.0	0.9781
2.8354	42.0	1974	2.8205	1.0	0.9577
2.824	43.0	2021	2.8253	1.0	0.9421
2.8115	44.0	2068	2.8224	1.0	0.9337
2.7896	45.0	2115	2.8131	1.0	0.9047
2.7896	46.0	2162	2.8056	1.0	0.9054
2.7827	47.0	2209	2.7961	1.0	0.8800
2.7782	48.0	2256	2.7957	1.0	0.9012
2.7735	49.0	2303	2.7653	1.0	0.8673
2.764	50.0	2350	2.7609	1.0	0.8709
2.7556	51.0	2397	2.7868	1.0	0.8793
2.749	52.0	2444	2.7823	1.0	0.8687
2.7536	53.0	2491	2.7569	1.0	0.8631
2.7514	54.0	2538	2.7655	1.0	0.8751
2.7421	55.0	2585	2.7617	1.0	0.8652
2.7364	56.0	2632	2.7755	1.0	0.8800
2.732	57.0	2679	2.7698	1.0	0.8744
2.7358	58.0	2726	2.7808	1.0	0.8843
2.7343	59.0	2773	2.7706	1.0	0.8793
2.7345	60.0	2820	2.7710	1.0	0.8800

Framework versions

• Transformers 4.28.0
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.13.3