whisper-large-v3-ft

This model is a fine-tuned version of openai/whisper-large-v3-turbo on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0821
• Cer: 8.7431

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 2
• eval_batch_size: 8
• seed: 3407
• gradient_accumulation_steps: 4
• total_train_batch_size: 8
• optimizer: Use OptimizerNames.ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 5
• num_epochs: 1
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer
0.2707	0.0216	5	0.3880	23.5651
0.5532	0.0431	10	0.5040	26.8960
0.476	0.0647	15	0.4247	27.7647
0.3431	0.0862	20	0.4034	36.5404
0.6185	0.1078	25	0.3918	24.5546
0.5411	0.1293	30	0.3905	21.0796
0.5324	0.1509	35	0.3742	18.0089
0.4989	0.1724	40	0.3445	54.7885
0.3512	0.1940	45	0.3749	78.1538
0.2636	0.2155	50	0.3357	99.2590
0.2148	0.2371	55	0.3092	311.9277
0.3944	0.2586	60	0.3271	52.0220
0.4783	0.2802	65	0.2863	19.6395
0.3513	0.3017	70	0.3049	35.3302
0.2803	0.3233	75	0.2810	15.4259
0.4088	0.3448	80	0.3059	13.3098
0.5916	0.3664	85	0.3278	13.1588
0.2734	0.3879	90	0.2855	22.7335
0.4415	0.4095	95	0.2608	15.3748
0.2744	0.4310	100	0.2387	23.0424
0.4602	0.4526	105	0.2614	35.7111
0.3697	0.4741	110	0.2264	24.8612
0.2217	0.4957	115	0.2243	27.5673
0.338	0.5172	120	0.2229	47.1952
0.1739	0.5388	125	0.2146	14.3876
0.3325	0.5603	130	0.1969	12.6478
0.1548	0.5819	135	0.1925	19.5884
0.3931	0.6034	140	0.1802	20.8683
0.652	0.625	145	0.1763	10.1926
0.4129	0.6466	150	0.1769	9.4609
0.2063	0.6681	155	0.1580	20.1273
0.2321	0.6897	160	0.1622	28.1875
0.1785	0.7112	165	0.1526	20.4827
0.1649	0.7328	170	0.1367	14.3342
0.4396	0.7543	175	0.1286	11.2378
0.3189	0.7759	180	0.1241	13.1286
0.1618	0.7974	185	0.1190	19.4002
0.2464	0.8190	190	0.1151	14.1878
0.1662	0.8405	195	0.1112	10.1066
0.2494	0.8621	200	0.1047	10.8476
0.3984	0.8836	205	0.1002	12.8220
0.1388	0.9052	210	0.0967	10.2135
0.2207	0.9267	215	0.0929	6.7827
0.2357	0.9483	220	0.0875	6.5620
0.2983	0.9698	225	0.0838	6.7037
0.1648	0.9914	230	0.0821	8.7431

Framework versions

• Transformers 4.51.3
• Pytorch 2.7.0+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1