wav2vec2-large-xlsr-darija

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1197
• Wer: 0.7182

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.9899	3.3003	500	3.0004	0.9941
1.4295	6.6007	1000	1.1875	0.9684
0.8642	9.9010	1500	0.9675	0.8853
0.4967	13.2013	2000	0.9604	0.8331
0.6898	16.5017	2500	0.9469	0.7793
0.6133	19.8020	3000	0.9454	0.7450
0.3807	23.1023	3500	1.0373	0.7361
0.2648	26.4026	4000	1.0828	0.7222
0.2119	29.7030	4500	1.1177	0.7185

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1