wav2vec2-finetune-XLSR_maroc

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2804
• Wer: 0.3265

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
4.9859	0.83	400	2.0919	0.9982
0.8639	1.66	800	0.4365	0.5925
0.4879	2.49	1200	0.3467	0.5079
0.3732	3.32	1600	0.3267	0.4756
0.314	4.15	2000	0.2835	0.4314
0.274	4.97	2400	0.2915	0.4364
0.2463	5.8	2800	0.3050	0.4277
0.2354	6.63	3200	0.2766	0.4179
0.2101	7.46	3600	0.2896	0.4071
0.1976	8.29	4000	0.2856	0.4099
0.186	9.12	4400	0.2849	0.3987
0.1758	9.95	4800	0.2819	0.4026
0.1667	10.78	5200	0.2869	0.3934
0.1508	11.61	5600	0.2793	0.3851
0.1468	12.44	6000	0.2777	0.3836
0.1322	13.26	6400	0.2731	0.3767
0.1295	14.09	6800	0.2833	0.3741
0.1157	14.92	7200	0.2815	0.3786
0.1147	15.75	7600	0.2684	0.3741
0.1099	16.58	8000	0.2704	0.3677
0.1056	17.41	8400	0.2744	0.3668
0.0983	18.24	8800	0.2675	0.3676
0.0975	19.07	9200	0.2787	0.3588
0.0906	19.9	9600	0.2749	0.3537
0.0862	20.73	10000	0.2875	0.3557
0.0812	21.55	10400	0.2863	0.3482
0.0761	22.38	10800	0.2739	0.3513
0.0738	23.21	11200	0.2878	0.3467
0.0678	24.04	11600	0.2886	0.3399
0.0661	24.87	12000	0.2958	0.3380
0.0623	25.7	12400	0.2779	0.3354
0.0586	26.53	12800	0.2871	0.3333
0.0563	27.36	13200	0.2895	0.3316
0.0554	28.19	13600	0.2846	0.3277
0.0522	29.02	14000	0.2752	0.3276
0.0498	29.84	14400	0.2804	0.3265

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1