wav2vec2-xlrs-finetuning

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the arrow dataset. It achieves the following results on the evaluation set:

• Loss: 2.6930
• Accuracy: 0.2803
• F1 Score: 0.1655
• Mse: 1.9782
• Mae: 1.0558
• Mae^m: 1.5570

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 9
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 18
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Score	Mse	Mae	Mae^m
1.7525	0.6116	100	1.7058	0.3497	0.0648	3.0952	1.2231	2.75
1.6944	1.2232	200	1.6854	0.3497	0.0648	3.0952	1.2231	2.75
1.7037	1.8349	300	1.7210	0.2367	0.0479	2.0190	1.1075	2.25
1.7667	2.4465	400	1.6868	0.3646	0.0927	2.6707	1.1224	2.5934
1.6859	3.0581	500	1.6833	0.3497	0.0648	3.0952	1.2231	2.75
1.6991	3.6697	600	1.6827	0.3497	0.0648	3.0952	1.2231	2.75
1.6404	4.2813	700	1.7078	0.3497	0.0648	3.0952	1.2231	2.75
1.7088	4.8930	800	1.6962	0.3497	0.0648	3.0952	1.2231	2.75
1.751	5.5046	900	1.6863	0.3497	0.0648	3.0952	1.2231	2.75
1.7813	6.1162	1000	1.6966	0.3497	0.0648	3.0952	1.2231	2.75
1.712	6.7278	1100	1.6783	0.3497	0.0648	3.0952	1.2231	2.75
1.6503	7.3394	1200	1.6816	0.3497	0.0648	3.0952	1.2231	2.75
1.5967	7.9511	1300	1.6588	0.3497	0.0648	3.0952	1.2231	2.75
1.6538	8.5627	1400	1.6229	0.3497	0.0648	3.0952	1.2231	2.75
1.654	9.1743	1500	1.6118	0.3633	0.0919	2.5347	1.1061	2.2236
1.5913	9.7859	1600	1.5914	0.3660	0.1215	2.1537	1.0218	2.1284
1.5896	10.3976	1700	1.6029	0.3578	0.1333	1.9102	0.9796	1.9420
1.5274	11.0092	1800	1.6201	0.3769	0.1124	2.3429	1.0558	2.1678
1.4581	11.6208	1900	1.5739	0.3687	0.1036	2.3918	1.0748	2.2286
1.4	12.2324	2000	1.6100	0.3442	0.1365	2.0395	1.0190	1.6647
1.5475	12.8440	2100	1.5741	0.3524	0.1431	1.6735	0.9388	1.7980
1.4724	13.4557	2200	1.5782	0.3660	0.1596	1.7537	0.9347	1.6576
1.3411	14.0673	2300	1.5875	0.3810	0.1273	2.0503	0.9891	1.8447
1.4884	14.6789	2400	1.5822	0.3537	0.1388	1.9293	0.9878	1.6427
1.4326	15.2905	2500	1.5996	0.3456	0.1613	1.7320	0.9565	1.5640
1.3412	15.9021	2600	1.6870	0.2952	0.1280	2.0231	1.0653	1.5219
1.3139	16.5138	2700	1.6911	0.3361	0.1439	1.7605	0.9660	1.6466
1.1558	17.1254	2800	1.6576	0.3469	0.1504	1.8340	0.9714	1.6081
1.3099	17.7370	2900	1.6184	0.3156	0.1694	1.7075	0.9701	1.5880
1.3763	18.3486	3000	1.7211	0.3333	0.1505	1.7578	0.9633	1.6180
1.1732	18.9602	3100	1.7625	0.3401	0.1580	1.8082	0.9728	1.6791
1.1137	19.5719	3200	1.8241	0.3252	0.1757	1.7946	0.9864	1.6387
1.128	20.1835	3300	1.8824	0.3156	0.1727	1.7823	0.9932	1.6233
1.0219	20.7951	3400	1.9237	0.3456	0.1630	1.7891	0.9619	1.5458
1.0466	21.4067	3500	1.9450	0.3238	0.1610	1.6993	0.9619	1.6316
1.1768	22.0183	3600	2.0642	0.3034	0.1720	1.9946	1.0503	1.4796
1.1656	22.6300	3700	2.2870	0.2680	0.1696	2.1741	1.1156	1.4731
0.8624	23.2416	3800	2.2612	0.3102	0.1749	1.8435	1.0	1.6763
1.0267	23.8532	3900	2.2753	0.2939	0.1647	2.0150	1.0599	1.5093
0.8059	24.4648	4000	2.2790	0.3197	0.1704	1.7714	0.9796	1.5151
0.8924	25.0765	4100	2.3183	0.2898	0.1755	1.9048	1.0313	1.5539
0.9581	25.6881	4200	2.3930	0.2803	0.1722	2.0082	1.0667	1.5516
0.7513	26.2997	4300	2.5025	0.2789	0.1631	2.0367	1.0735	1.5718
0.8617	26.9113	4400	2.5418	0.2966	0.1686	1.9959	1.0463	1.5909
0.7768	27.5229	4500	2.6084	0.2898	0.1718	1.9551	1.0463	1.5532
0.7316	28.1346	4600	2.6581	0.2844	0.1687	1.8803	1.0286	1.5414
0.7436	28.7462	4700	2.6899	0.2667	0.1611	2.0503	1.0871	1.5357
0.8487	29.3578	4800	2.6930	0.2803	0.1655	1.9782	1.0558	1.5570

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.1+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1