Taeyeun72/whisper-small-denoising_all_pure

whisper-kor3_de_all_pure

This model is a fine-tuned version of openai/whisper-small on the whisper-kor3_de_all_pure dataset. It achieves the following results on the evaluation set:

• Loss: 0.4036
• Wer: 24.4454
• Cer: 10.9931

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 4000

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.1711	0.05	100	0.9951	36.7906	18.3518
0.7306	0.09	200	0.7148	32.9732	15.0468
0.5337	0.14	300	0.5650	32.4402	15.4036
0.492	0.18	400	0.5397	32.4546	16.1485
0.4779	0.23	500	0.5196	30.6108	14.1530
0.4501	0.28	600	0.5130	31.3166	14.6783
0.4574	0.32	700	0.4941	32.4978	15.6153
0.4215	0.37	800	0.4920	28.4500	13.0396
0.4574	0.42	900	0.4829	30.4379	14.8704
0.3928	0.46	1000	0.4750	28.1619	12.7024
0.4207	0.51	1100	0.4673	29.3287	13.8550
0.4026	0.55	1200	0.4654	28.7957	13.8393
0.4004	0.6	1300	0.4581	27.9458	14.1726
0.3808	0.65	1400	0.4498	26.7214	12.2319
0.3994	0.69	1500	0.4483	26.7790	12.0751
0.4087	0.74	1600	0.4373	26.3901	12.1496
0.3761	0.78	1700	0.4343	26.6926	11.9771
0.3889	0.83	1800	0.4334	27.3840	12.9651
0.4135	0.88	1900	0.4296	25.9723	12.5417
0.3559	0.92	2000	0.4257	25.3097	11.5537
0.433	0.97	2100	0.4211	25.6410	11.5615
0.2988	1.02	2200	0.4213	25.0648	12.0790
0.221	1.06	2300	0.4241	25.3529	11.2832
0.2657	1.11	2400	0.4218	25.2953	11.4949
0.2538	1.15	2500	0.4205	25.3961	12.1731
0.2398	1.2	2600	0.4202	25.1945	12.0281
0.268	1.25	2700	0.4152	24.8776	11.2165
0.2683	1.29	2800	0.4129	24.9496	11.1734
0.2688	1.34	2900	0.4118	25.0936	11.2793
0.2713	1.39	3000	0.4135	24.5030	11.0754
0.2572	1.43	3100	0.4103	24.7479	11.0793
0.2565	1.48	3200	0.4106	24.5462	11.6360
0.253	1.52	3300	0.4116	24.5462	10.9068
0.2549	1.57	3400	0.4092	24.5895	11.0283
0.2859	1.62	3500	0.4080	24.8920	11.2087
0.236	1.66	3600	0.4062	24.8632	11.1499
0.2422	1.71	3700	0.4051	24.4454	11.0127
0.2441	1.75	3800	0.4050	24.4166	10.9735
0.2694	1.8	3900	0.4041	24.5030	11.0440
0.2311	1.85	4000	0.4036	24.4454	10.9931

Framework versions

• Transformers 4.33.2
• Pytorch 2.1.0+cu121
• Datasets 2.14.5
• Tokenizers 0.13.3