whisper-medium-pt-nonverbal

This model is a fine-tuned version of openai/whisper-medium on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2991
• Wer: 97.7528

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.5616	0.5747	50	4.9095	101.1236
2.9024	1.1494	100	2.6538	261.7978
1.0425	1.7241	150	0.1202	97.7528
0.1256	2.2989	200	0.1106	96.6292
0.2022	2.8736	250	0.1010	95.5056
0.1121	3.4483	300	0.1808	97.7528
0.1178	4.0230	350	0.1375	95.5056
0.123	4.5977	400	0.1492	98.8764
0.0813	5.1724	450	0.2487	96.6292
0.104	5.7471	500	0.3074	98.8764
0.0635	6.3218	550	0.2173	97.7528
0.1154	6.8966	600	0.3012	97.7528
0.0964	7.4713	650	0.2261	96.6292
0.0661	8.0460	700	0.2979	96.6292
0.0169	8.6207	750	0.3020	96.6292
0.0192	9.1954	800	0.3055	97.7528
0.0888	9.7701	850	0.2718	97.7528
0.0277	10.3448	900	0.3717	100.0
0.0304	10.9195	950	0.3276	100.0
0.0064	11.4943	1000	0.4031	96.6292
0.0226	12.0690	1050	0.4136	97.7528
0.0158	12.6437	1100	0.3901	95.5056
0.0163	13.2184	1150	0.3312	97.7528
0.013	13.7931	1200	0.3925	98.8764
0.001	14.3678	1250	0.4168	100.0
0.016	14.9425	1300	0.4466	98.8764
0.0171	15.5172	1350	0.3616	98.8764
0.0008	16.0920	1400	0.2944	97.7528
0.0102	16.6667	1450	0.2915	96.6292
0.0092	17.2414	1500	0.2665	97.7528
0.0155	17.8161	1550	0.2713	96.6292
0.0004	18.3908	1600	0.2743	97.7528
0.0149	18.9655	1650	0.2850	96.6292
0.0062	19.5402	1700	0.2868	97.7528
0.0129	20.1149	1750	0.3166	97.7528
0.0021	20.6897	1800	0.3289	96.6292
0.0014	21.2644	1850	0.2872	97.7528
0.0061	21.8391	1900	0.2667	97.7528
0.0048	22.4138	1950	0.2936	97.7528
0.0	22.9885	2000	0.2979	96.6292
0.0026	23.5632	2050	0.3010	96.6292
0.0008	24.1379	2100	0.3309	97.7528
0.0	24.7126	2150	0.3464	97.7528
0.0089	25.2874	2200	0.3163	97.7528
0.0	25.8621	2250	0.3137	97.7528
0.0	26.4368	2300	0.3160	97.7528
0.011	27.0115	2350	0.3236	97.7528
0.0036	27.5862	2400	0.3311	97.7528
0.0036	28.1609	2450	0.3433	97.7528
0.0	28.7356	2500	0.3706	97.7528
0.0003	29.3103	2550	0.3322	97.7528
0.0	29.8851	2600	0.3313	97.7528
0.0	30.4598	2650	0.2929	97.7528
0.009	31.0345	2700	0.2978	97.7528
0.0037	31.6092	2750	0.2991	97.7528
0.0068	32.1839	2800	0.2890	97.7528
0.0036	32.7586	2850	0.2914	97.7528
0.0	33.3333	2900	0.2940	97.7528
0.0002	33.9080	2950	0.3004	97.7528
0.0001	34.4828	3000	0.2963	97.7528
0.0	35.0575	3050	0.2930	97.7528
0.0	35.6322	3100	0.2934	97.7528
0.0033	36.2069	3150	0.2972	97.7528
0.0	36.7816	3200	0.2970	97.7528
0.0038	37.3563	3250	0.2972	97.7528
0.0	37.9310	3300	0.2972	97.7528
0.0	38.5057	3350	0.2973	97.7528
0.0039	39.0805	3400	0.2976	97.7528
0.0037	39.6552	3450	0.2976	97.7528
0.0	40.2299	3500	0.2979	97.7528
0.0	40.8046	3550	0.2982	97.7528
0.0	41.3793	3600	0.2983	97.7528
0.0	41.9540	3650	0.2985	97.7528
0.0	42.5287	3700	0.2987	97.7528
0.004	43.1034	3750	0.2987	97.7528
0.0	43.6782	3800	0.2989	97.7528
0.0	44.2529	3850	0.2990	97.7528
0.0042	44.8276	3900	0.2991	97.7528
0.0045	45.4023	3950	0.2991	97.7528
0.0	45.9770	4000	0.2991	97.7528

Framework versions

• Transformers 4.42.3
• Pytorch 2.3.1+cu118
• Datasets 2.20.0
• Tokenizers 0.19.1