wav2vec2-xls-r-1b-faroese-100h-11k-steps

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1086
• Wer: 24.7332
• Cer: 7.9893

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 3000
• training_steps: 11000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.8973	0.4640	1000	0.4108	52.5770	16.0283
0.5764	0.9281	2000	0.2881	40.4168	12.4518
0.4124	1.3921	3000	0.2481	38.8565	12.0174
0.3191	1.8561	4000	0.2274	36.4959	11.3942
0.2362	2.3202	5000	0.1796	32.1774	10.0837
0.203	2.7842	6000	0.1725	30.8738	9.7667
0.1742	3.2483	7000	0.1468	29.1524	9.2248
0.1367	3.7123	8000	0.1320	27.2448	8.7037
0.1066	4.1763	9000	0.1259	26.2885	8.4553
0.0949	4.6404	10000	0.1145	25.1611	8.0950
0.0869	5.1044	11000	0.1086	24.7332	7.9893

Framework versions

• Transformers 4.46.2
• Pytorch 2.5.0+cu121
• Datasets 3.1.0
• Tokenizers 0.20.3