wav2vec2-balti

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the generator dataset. It achieves the following results on the evaluation set:

• Loss: 0.3484
• Wer: 0.2282

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 300
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
24.9571	0.3971	200	3.1221	1.0
12.3515	0.7942	400	1.1488	0.8080
7.2630	1.1906	600	0.6948	0.6025
5.8629	1.5877	800	0.5714	0.5066
5.5451	1.9849	1000	0.4945	0.4236
4.7129	2.3812	1200	0.4673	0.4094
4.6447	2.7784	1400	0.4284	0.3989
3.8591	3.1747	1600	0.4073	0.3692
3.8259	3.5719	1800	0.3952	0.3406
3.6229	3.9690	2000	0.3744	0.3340
3.1691	4.3654	2200	0.3719	0.3179
3.2988	4.7625	2400	0.3513	0.3078
2.7646	5.1588	2600	0.3599	0.3105
2.7120	5.5560	2800	0.3445	0.2917
2.8528	5.9531	3000	0.3358	0.2842
2.3170	6.3495	3200	0.3455	0.2882
2.6743	6.7466	3400	0.3492	0.2862
2.1990	7.1430	3600	0.3379	0.2769
2.2063	7.5401	3800	0.3429	0.2731
2.3009	7.9372	4000	0.3361	0.2734
1.9231	8.3336	4200	0.3332	0.2639
1.8913	8.7307	4400	0.3247	0.2621
1.6484	9.1271	4600	0.3292	0.2489
1.7733	9.5242	4800	0.3208	0.2592
1.6847	9.9213	5000	0.3327	0.2495
1.6250	10.3177	5200	0.3416	0.2458
1.7336	10.7148	5400	0.3212	0.2456
1.3292	11.1112	5600	0.3407	0.2393
1.4187	11.5083	5800	0.3319	0.2386
1.3104	11.9054	6000	0.3266	0.2378
1.2938	12.3018	6200	0.3373	0.2372
1.0719	12.6989	6400	0.3469	0.2326
1.1590	13.0953	6600	0.3380	0.2297
1.1588	13.4924	6800	0.3409	0.2323
1.0949	13.8896	7000	0.3386	0.2289
1.1241	14.2859	7200	0.3354	0.2284
1.0928	14.6830	7400	0.3484	0.2282

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 5.0.0
• Tokenizers 0.22.2