fine-tune-wav2vec2-large-xls-r-300m-ssw_224s

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the ml-superb-subset dataset. It achieves the following results on the evaluation set:

• Loss: 0.8167
• Wer: 0.5492

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.4669	2.0997	400	3.1603	1.0
2.5291	4.1995	800	1.2456	0.9651
0.8905	6.2992	1200	0.7689	0.7746
0.5222	8.3990	1600	0.7821	0.7048
0.3768	10.4987	2000	0.7637	0.7238
0.2874	12.5984	2400	0.7030	0.6063
0.2216	14.6982	2800	0.8468	0.6571
0.1954	16.7979	3200	0.7099	0.5841
0.1649	18.8976	3600	0.7696	0.5651
0.1384	20.9974	4000	0.8328	0.5873
0.1208	23.0971	4400	0.7899	0.5651
0.1054	25.1969	4800	0.8310	0.5714
0.095	27.2966	5200	0.8183	0.5302
0.0835	29.3963	5600	0.8167	0.5492

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1