my-seq2seq-model

This model is a fine-tuned version of openai/whisper-tiny on the PolyAI/minds14 dataset. It achieves the following results on the evaluation set:

• Loss: 0.7355
• Wer Ortho: 0.3526
• Wer: 0.3542

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 40
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Ortho	Wer
No log	1.0	57	0.5982	0.3546	0.3536
No log	2.0	114	0.6244	0.3816	0.3838
No log	3.0	171	0.6240	0.3662	0.3629
No log	4.0	228	0.6321	0.3423	0.3426
No log	5.0	285	0.6522	0.3732	0.3715
No log	6.0	342	0.6582	0.3526	0.3524
No log	7.0	399	0.6628	0.3507	0.3506
No log	8.0	456	0.6683	0.3501	0.3499
0.0203	9.0	513	0.6734	0.3443	0.3444
0.0203	10.0	570	0.6804	0.3456	0.3456
0.0203	11.0	627	0.6837	0.3430	0.3432
0.0203	12.0	684	0.6878	0.3468	0.3456
0.0203	13.0	741	0.6939	0.3481	0.3469
0.0203	14.0	798	0.6960	0.3501	0.3487
0.0203	15.0	855	0.6991	0.3481	0.3469
0.0203	16.0	912	0.7017	0.3468	0.3475
0.0203	17.0	969	0.7044	0.3468	0.3475
0.0024	18.0	1026	0.7082	0.3507	0.3506
0.0024	19.0	1083	0.7089	0.3507	0.3506
0.0024	20.0	1140	0.7131	0.3533	0.3536
0.0024	21.0	1197	0.7141	0.3546	0.3549
0.0024	22.0	1254	0.7176	0.3520	0.3518
0.0024	23.0	1311	0.7197	0.3507	0.3506
0.0024	24.0	1368	0.7211	0.3507	0.3506
0.0024	25.0	1425	0.7223	0.3501	0.3499
0.0024	26.0	1482	0.7243	0.3501	0.3499
0.0014	27.0	1539	0.7255	0.3507	0.3506
0.0014	28.0	1596	0.7264	0.3514	0.3512
0.0014	29.0	1653	0.7293	0.3520	0.3524
0.0014	30.0	1710	0.7297	0.3514	0.3518
0.0014	31.0	1767	0.7310	0.3507	0.3518
0.0014	32.0	1824	0.7308	0.3507	0.3518
0.0014	33.0	1881	0.7330	0.3507	0.3518
0.0014	34.0	1938	0.7336	0.3507	0.3518
0.0014	35.0	1995	0.7338	0.3507	0.3518
0.0010	36.0	2052	0.7341	0.3526	0.3542
0.0010	37.0	2109	0.7349	0.3520	0.3530
0.0010	38.0	2166	0.7350	0.3526	0.3542
0.0010	39.0	2223	0.7356	0.3520	0.3530
0.0010	40.0	2280	0.7355	0.3526	0.3542

Framework versions

• Transformers 5.14.1
• Pytorch 2.11.0+cu128
• Datasets 5.0.0
• Tokenizers 0.22.2