wav2vec2-large-xls-r-300m-Arabic-phoneme-based-MDD-experiment2

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0434
• Per: 0.0207
• Wer: 0.0212

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 8
• eval_batch_size: 6
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 250
• num_epochs: 30.0

Training results

Training Loss	Epoch	Step	Validation Loss	Per	Wer
2.4593	1.0	546	1.6587	0.9769	0.9819
1.7776	2.0	1093	1.5794	0.9995	0.9994
1.733	3.0	1640	1.5264	0.9998	0.9998
1.6734	4.0	2187	1.3536	0.9948	0.9831
1.5266	5.0	2733	0.8784	0.6903	0.7321
1.1705	6.0	3280	0.3408	0.2076	0.2568
0.8237	7.0	3827	0.1968	0.1045	0.1223
0.66	8.0	4374	0.1400	0.0756	0.0851
0.5747	9.0	4920	0.0954	0.0665	0.0659
0.5063	10.0	5467	0.0999	0.0645	0.0650
0.4641	11.0	6014	0.0800	0.0545	0.0572
0.4246	12.0	6561	0.0783	0.0512	0.0506
0.3957	13.0	7107	0.0780	0.0494	0.0516
0.3693	14.0	7654	0.0647	0.0393	0.0411
0.3517	15.0	8201	0.0597	0.0364	0.0370
0.3292	16.0	8748	0.0577	0.0356	0.0361
0.3079	17.0	9294	0.0573	0.0346	0.0362
0.2946	18.0	9841	0.0622	0.0350	0.0389
0.2735	19.0	10388	0.0525	0.0293	0.0300
0.2588	20.0	10935	0.0553	0.0254	0.0283
0.2477	21.0	11481	0.0479	0.0244	0.0259
0.2346	22.0	12028	0.0560	0.0280	0.0283
0.222	23.0	12575	0.0421	0.0242	0.0250
0.208	24.0	13122	0.0508	0.0240	0.0245
0.2002	25.0	13668	0.0435	0.0218	0.0223
0.1912	26.0	14215	0.0477	0.0225	0.0239
0.1823	27.0	14762	0.0414	0.0210	0.0214
0.1745	28.0	15309	0.0434	0.0212	0.0216
0.1703	29.0	15855	0.0427	0.0202	0.0212
0.163	29.96	16380	0.0434	0.0207	0.0212

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 1.18.3
• Tokenizers 0.13.3