MahmoodAnaam/MSP-AVSR-E2E

github dev branch commit

MSP-AVSR

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1193
• Wer: 0.1736

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 1000.0
• training_steps: 30000

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.2821	0.0333	1000	1.5690	0.2983
2.1664	0.0667	2000	1.3053	0.2234
2.0936	0.1	3000	1.1193	0.1736
2.2092	0.1333	4000	1.2822	0.2420
2.3215	0.1667	5000	1.4375	0.2676
2.0802	0.2	6000	1.3366	0.2295
2.0530	0.2333	7000	1.2210	0.1985
2.1923	0.2667	8000	1.3655	0.2465
2.1158	0.3	9000	1.4065	0.2484
2.0377	0.3333	10000	1.4430	0.2271
2.0040	0.3667	11000	1.3231	0.2188
1.9140	0.4	12000	1.5497	0.2668
2.1310	0.4333	13000	1.1773	0.1909
2.0774	0.4667	14000	1.1900	0.1888
1.8577	0.5	15000	1.4826	0.2567
1.8542	0.5333	16000	1.3038	0.2196
1.9256	0.5667	17000	1.1100	0.1767
2.0081	0.6	18000	1.3671	0.2258
1.8983	0.6333	19000	1.2541	0.2035
1.8399	0.6667	20000	1.2768	0.2127
1.9230	0.0333	21000	1.3529	0.2279
1.7808	0.0667	22000	1.3944	0.2262
1.8058	0.1	23000	1.2840	0.2044
1.8971	0.1333	24000	1.2563	0.2056
2.0928	0.1667	25000	1.2498	0.2054
1.8353	0.2	26000	1.2187	0.1997
1.8451	0.2333	27000	1.2304	0.1999
2.0854	0.2667	28000	1.2552	0.2079
1.9424	0.3	29000	1.2496	0.2042
1.8996	0.3333	30000	1.2481	0.2046

Framework versions

• Transformers 5.10.2
• Pytorch 2.10.0+rocm7.2.4.git3d3aa833
• Datasets 4.0.0
• Tokenizers 0.22.2