ymoslem/whisper-medium-ga2en-v6.2.3-r

Whisper Medium GA-EN Speech Translation

This model is a fine-tuned version of openai/whisper-small on the IWSLT-2023, FLEURS, BiteSize, SpokenWords, Tatoeba, Wikimedia, and EUbookshop dataset. It achieves the following results on the evaluation set:

• Loss: 1.1622
• Bleu: 35.77
• Chrf: 56.32
• Wer: 58.3971

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.02
• training_steps: 15000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Bleu	Chrf	Validation Loss	Wer
2.6534	0.0138	100	1.43	15.99	2.2446	269.1130
2.4519	0.0276	200	2.13	18.36	2.1941	250.5178
2.2928	0.0414	300	7.14	25.95	2.0086	128.3656
2.233	0.0552	400	5.61	24.25	2.0239	134.0837
2.0406	0.0690	500	5.64	25.65	1.9215	183.8361
2.0273	0.0828	600	13.41	30.96	1.8556	83.7010
1.895	0.0966	700	7.02	26.82	1.8278	158.2170
1.9889	0.1103	800	12.22	31.62	1.7842	99.6398
1.8484	0.1241	900	10.97	30.45	1.7648	91.1751
1.7491	0.1379	1000	10.0	29.42	1.7498	109.0050
1.699	0.1517	1100	12.53	34.87	1.6662	109.9054
1.6959	0.1655	1200	14.54	34.8	1.6287	92.3008
1.6682	0.1793	1300	13.26	33.5	1.5800	103.0617
1.6625	0.1931	1400	19.71	37.33	1.6115	75.9118
1.5462	0.2069	1500	18.3	39.49	1.4993	93.7866
1.3834	0.2207	1600	20.32	40.87	1.4906	79.2436
1.39	0.2345	1700	17.3	38.16	1.4752	93.1562
1.5061	0.2483	1800	20.11	39.69	1.4004	81.0446
1.4125	0.2621	1900	23.82	42.67	1.3854	73.3904
1.3181	0.2759	2000	20.57	40.87	1.3979	78.8384
1.283	0.2897	2100	17.97	40.47	1.3446	88.8789
1.2061	0.3034	2200	25.12	45.42	1.3130	73.5254
1.2091	0.3172	2300	22.12	43.56	1.3274	79.8739
1.1264	0.3310	2400	22.94	45.96	1.2771	78.2080
1.0972	0.3448	2500	24.38	46.04	1.2858	75.4615
1.0822	0.3586	2600	27.39	48.34	1.2376	67.6722
1.0316	0.3724	2700	28.0	47.61	1.2461	68.5277
1.165	0.3862	2800	26.05	48.13	1.1869	71.6794
1.025	0.4	2900	27.14	47.91	1.1716	68.7528
0.8978	0.4138	3000	28.34	49.15	1.1628	65.6461
0.9146	0.4276	3100	25.81	48.42	1.1703	71.7244
0.9764	0.4414	3200	29.63	51.22	1.1526	67.3570
0.9455	0.4552	3300	25.31	49.73	1.1108	72.6249
0.9073	0.4690	3400	27.7	50.85	1.1085	72.7150
0.8596	0.4828	3500	28.34	52.39	1.0927	67.9424
0.8241	0.4966	3600	29.95	51.37	1.1026	65.2859
0.8436	0.5103	3700	27.18	51.45	1.0718	71.2292
0.8318	0.5241	3800	30.71	53.35	1.0678	64.3404
0.8262	0.5379	3900	27.05	51.94	1.0534	71.5894
0.8129	0.5517	4000	27.38	51.97	1.0491	72.1747
0.9036	0.5655	4100	14.43	40.57	1.2250	139.3066
1.0314	0.5793	4200	24.27	46.97	1.2310	75.5966
0.9209	0.5931	4300	23.55	46.04	1.2447	76.4070
0.9204	0.6069	4400	25.87	45.32	1.2891	73.0302
0.9843	0.6207	4500	27.2	46.36	1.2269	71.8145
1.0225	0.6345	4600	26.16	45.72	1.2403	69.6983
0.9773	0.6483	4700	26.37	45.62	1.2464	68.4376
0.9794	0.6621	4800	24.77	47.11	1.2461	72.0846
0.8905	0.6759	4900	24.58	46.35	1.2345	71.2742
0.8305	0.6897	5000	27.28	48.37	1.2239	68.1675
0.9019	0.7034	5100	27.04	50.28	1.1730	70.1486
0.7969	0.7172	5200	26.27	48.07	1.1807	69.0230
0.8036	0.7310	5300	23.04	48.3	1.1632	77.5326
0.8195	0.7448	5400	25.58	50.29	1.1811	76.2269
0.7697	0.7586	5500	23.99	48.91	1.1825	81.4948
0.727	0.7724	5600	23.93	49.23	1.1623	79.5137
0.8002	0.7862	5700	26.29	50.44	1.1503	75.6866
0.6909	0.8	5800	29.27	50.85	1.1338	64.0252
0.7146	0.8138	5900	28.24	50.82	1.1420	66.6367
0.7452	0.8276	6000	31.33	51.92	1.1328	62.4944
0.5989	0.8414	6100	31.1	52.15	1.1455	65.1959
0.6818	0.8552	6200	32.56	52.46	1.1112	62.1342
0.6074	0.8690	6300	33.48	53.32	1.1072	60.6033
0.5942	0.8828	6400	31.39	51.03	1.1462	62.8546
0.6341	0.8966	6500	31.55	52.15	1.1093	62.4043
0.5992	0.9103	6600	33.06	52.52	1.1215	61.4588
0.6156	0.9241	6700	32.38	52.76	1.1031	62.9446
0.6169	0.9379	6800	31.46	52.96	1.1082	64.3404
0.6543	0.9517	6900	33.49	54.02	1.0943	63.1247
0.5017	0.9655	7000	30.95	52.64	1.1141	68.6177
0.5583	0.9793	7100	34.39	54.03	1.1004	61.6839
0.5986	0.9931	7200	33.92	52.85	1.1055	62.4944
0.2443	1.0069	7300	34.86	53.01	1.1442	60.1981
0.254	1.0207	7400	33.92	53.25	1.1458	62.1792
0.2827	1.0345	7500	34.49	53.43	1.1190	60.6484
0.2326	1.0483	7600	35.47	53.53	1.1237	59.2076
0.2017	1.0621	7700	34.65	53.87	1.1179	60.0180
0.2367	1.0759	7800	34.23	53.67	1.1075	60.6484
0.2276	1.0897	7900	34.67	54.51	1.1063	60.3332
0.2087	1.1034	8000	34.44	54.07	1.1090	60.6484
0.2514	1.1172	8100	29.85	51.91	1.1199	69.6083
0.2692	1.1310	8200	28.05	51.94	1.1642	72.1747
0.2784	1.1448	8300	27.26	50.77	1.1262	74.8312
0.2539	1.1586	8400	30.7	53.1	1.1463	65.0158
0.2599	1.1724	8500	31.64	53.71	1.1255	63.2148
0.2419	1.1862	8600	33.2	54.15	1.1223	62.4043
0.2583	1.2	8700	33.98	53.65	1.1304	61.2787
0.239	1.2138	8800	34.68	54.35	1.1371	61.7740
0.2198	1.2276	8900	30.65	52.15	1.1533	72.2647
0.248	1.2414	9000	31.98	53.68	1.1266	65.4210
0.2377	1.2552	9100	30.9	53.6	1.1510	67.9424
0.2183	1.2690	9200	30.35	53.04	1.1565	73.1202
0.1999	1.2828	9300	29.48	53.0	1.1426	74.2909
0.22	1.2966	9400	31.93	53.16	1.1332	66.1414
0.2063	1.3103	9500	32.42	53.79	1.1144	63.3949
0.2054	1.3241	9600	33.64	54.69	1.1146	61.5038
0.2145	1.3379	9700	36.68	55.64	1.1123	57.5867
0.2059	1.3517	9800	36.93	56.15	1.1102	57.5416
0.2001	1.3655	9900	36.4	56.09	1.1143	57.9469
0.1973	1.3793	10000	36.46	55.74	1.1121	58.2620
0.2184	1.3931	10100	1.1377	31.31	53.89	66.9068
0.324	1.4069	10200	1.1906	29.38	51.17	70.2386
0.3419	1.4207	10300	1.2105	32.38	52.69	63.3498
0.2838	1.4345	10400	1.1928	31.53	50.99	63.5299
0.2888	1.4483	10500	1.2374	26.67	49.63	72.6700
0.3196	1.4621	10600	1.2343	26.79	48.63	72.3998
0.3081	1.4759	10700	1.2136	26.4	49.29	74.0207
0.2861	1.4897	10800	1.1898	26.01	50.26	73.2553
0.2951	1.5034	10900	1.2020	28.59	51.08	68.0774
0.2637	1.5172	11000	1.1516	29.92	51.81	66.4566
0.2854	1.5310	11100	1.1791	27.96	50.76	72.7150
0.2959	1.5448	11200	1.1705	30.01	51.37	65.5110
0.2649	1.5586	11300	1.1934	24.87	49.47	79.0185
0.2667	1.5724	11400	1.1785	29.03	50.83	69.2481
0.276	1.5862	11500	1.1887	29.83	52.07	66.8618
0.2588	1.6	11600	1.1778	30.0	52.37	66.9518
0.2669	1.6138	11700	1.1858	31.38	51.74	63.4399
0.2472	1.6276	11800	1.1927	31.14	50.61	64.4755
0.2477	1.6414	11900	1.2052	27.75	51.33	68.1225
0.2327	1.6552	12000	1.1935	29.36	51.86	67.7623
0.2299	1.6690	12100	1.1804	27.2	51.18	76.8122
0.2244	1.6828	12200	1.2038	29.6	52.19	67.4471
0.225	1.6966	12300	1.1762	31.87	53.47	64.1603
0.2219	1.7103	12400	1.1784	31.75	51.76	65.1959
0.2216	1.7241	12500	1.1717	31.69	53.61	62.8546
0.2646	1.7379	12600	1.1463	32.85	51.95	60.0630
0.204	1.7517	12700	1.1438	33.17	54.96	60.1531
0.19	1.7655	12800	1.1733	31.62	53.57	63.1247
0.2	1.7793	12900	1.1616	32.16	53.05	62.1792
0.1813	1.7931	13000	1.1632	30.8	52.56	63.9802
0.1648	1.8069	13100	1.1631	31.47	52.44	62.5844
0.1972	1.8207	13200	1.1591	34.03	54.33	59.6128
0.1648	1.8345	13300	1.1513	33.92	53.52	61.1887
0.1871	1.8483	13400	1.1571	33.33	53.79	62.5844
0.1762	1.8621	13500	1.1422	34.34	54.64	60.7834
0.1984	1.8759	13600	1.1465	33.58	55.31	62.3593
0.1838	1.8897	13700	1.1355	34.84	56.04	59.9730
0.1337	1.9034	13800	1.1309	34.3	55.84	60.5133
0.1692	1.9172	13900	1.1407	33.62	55.48	62.9896
0.1556	1.9310	14000	1.1484	35.42	55.56	59.0725
0.1364	1.9448	14100	1.1440	34.44	55.35	59.9280
0.1401	1.9586	14200	1.1345	33.77	55.22	60.5133
0.1399	1.9724	14300	1.1376	35.04	55.47	59.3877
0.139	1.9862	14400	1.1337	35.58	56.38	58.0369
0.1523	2.0	14500	1.1349	35.24	55.73	59.3877
0.0799	2.0138	14600	1.1575	35.13	56.17	59.7479
0.06	2.0276	14700	1.1582	35.52	56.5	59.4327
0.0583	2.0414	14800	1.1628	35.65	56.55	58.6673
0.0605	2.0552	14900	1.1615	35.85	56.37	58.0819
0.0654	2.0690	15000	1.1622	35.77	56.32	58.3971

Framework versions

• Transformers 4.41.2
• Pytorch 2.2.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1