whisper-small-si-bank-v1

This model is a fine-tuned version of openai/whisper-small on the Bank Dtaset dataset. It achieves the following results on the evaluation set:

• Loss: 0.6361
• Wer Ortho: 69.5652
• Wer: 46.9880

Model description

this model is based on the model = openai/whisper-small

Intended uses & limitations

training dataset has the data domain of bank and only 100 dataset, male voices.

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant_with_warmup
• lr_scheduler_warmup_steps: 50
• training_steps: 200
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Ortho	Wer
No log	0.4	2	2.3675	183.6957	250.3614
No log	0.8	4	2.3667	183.6957	250.3614
No log	1.2	6	2.3484	173.3696	254.6988
No log	1.6	8	2.2913	173.3696	254.4578
No log	2.0	10	2.1797	170.1087	255.1807
No log	2.4	12	2.0472	203.2609	239.5181
No log	2.8	14	1.9689	225.0	236.8675
No log	3.2	16	1.8878	236.4130	219.5181
No log	3.6	18	1.8019	259.2391	249.3976
No log	4.0	20	1.7219	242.3913	240.9639
No log	4.4	22	1.6462	178.2609	230.6024
No log	4.8	24	1.5752	156.5217	229.1566
2.0076	5.2	26	1.5220	163.5870	232.5301
2.0076	5.6	28	1.4893	173.9130	241.4458
2.0076	6.0	30	1.4548	182.6087	235.6627
2.0076	6.4	32	1.4253	132.0652	174.9398
2.0076	6.8	34	1.3838	145.6522	166.0241
2.0076	7.2	36	1.3474	116.8478	127.4699
2.0076	7.6	38	1.3150	132.0652	161.4458
2.0076	8.0	40	1.2866	125.0	163.6145
2.0076	8.4	42	1.2551	119.5652	125.5422
2.0076	8.8	44	1.2083	103.8043	150.3614
2.0076	9.2	46	1.1756	98.3696	112.0482
2.0076	9.6	48	1.1422	92.3913	101.6867
1.1337	10.0	50	1.1150	87.5	100.7229
1.1337	10.4	52	1.0769	85.8696	79.0361
1.1337	10.8	54	1.0328	84.7826	95.6627
1.1337	11.2	56	1.0071	93.4783	101.6867
1.1337	11.6	58	0.9826	100.0	86.2651
1.1337	12.0	60	0.9650	82.6087	82.8916
1.1337	12.4	62	0.9482	84.2391	86.5060
1.1337	12.8	64	0.9144	77.7174	71.5663
1.1337	13.2	66	0.8839	89.1304	75.1807
1.1337	13.6	68	0.8784	98.3696	74.6988
1.1337	14.0	70	0.8669	80.4348	65.7831
1.1337	14.4	72	0.8448	80.9783	69.1566
1.1337	14.8	74	0.8420	78.8043	67.9518
0.4644	15.2	76	0.8321	77.7174	72.5301
0.4644	15.6	78	0.7959	73.3696	57.8313
0.4644	16.0	80	0.7898	77.7174	60.0
0.4644	16.4	82	0.8081	76.0870	64.5783
0.4644	16.8	84	0.7942	76.6304	60.9639
0.4644	17.2	86	0.8025	80.4348	64.8193
0.4644	17.6	88	0.7867	79.8913	69.6386
0.4644	18.0	90	0.7894	81.5217	78.3133
0.4644	18.4	92	0.7815	110.8696	75.4217
0.4644	18.8	94	0.7870	73.3696	57.1084
0.4644	19.2	96	0.7925	73.3696	56.1446
0.4644	19.6	98	0.8006	75.5435	56.8675
0.1663	20.0	100	0.8155	75.5435	60.0
0.1663	20.4	102	0.7785	71.1957	60.4819
0.1663	20.8	104	0.7851	75.0	57.1084
0.1663	21.2	106	0.7998	72.8261	59.7590
0.1663	21.6	108	0.7855	76.0870	61.9277
0.1663	22.0	110	0.7740	71.7391	53.7349
0.1663	22.4	112	0.7820	72.8261	56.1446
0.1663	22.8	114	0.8177	76.0870	71.3253
0.1663	23.2	116	0.7610	72.2826	54.4578
0.1663	23.6	118	0.7477	71.7391	54.2169
0.1663	24.0	120	0.7726	71.1957	55.1807
0.1663	24.4	122	0.7767	72.2826	54.2169
0.1663	24.8	124	0.7470	71.1957	57.1084
0.1064	25.2	126	0.7444	69.0217	52.5301
0.1064	25.6	128	0.7619	72.2826	52.7711
0.1064	26.0	130	0.7792	74.4565	58.0723
0.1064	26.4	132	0.7774	68.4783	49.6386
0.1064	26.8	134	0.7712	73.3696	55.6627
0.1064	27.2	136	0.7601	72.8261	58.0723
0.1064	27.6	138	0.7206	70.1087	58.7952
0.1064	28.0	140	0.7250	76.6304	62.1687
0.1064	28.4	142	0.6878	71.1957	50.8434
0.1064	28.8	144	0.6782	76.0870	61.6867
0.1064	29.2	146	0.7219	76.6304	60.7229
0.1064	29.6	148	0.6799	72.2826	52.7711
0.0432	30.0	150	0.6596	67.3913	48.1928
0.0432	30.4	152	0.6828	69.5652	51.5663
0.0432	30.8	154	0.7441	73.9130	55.6627
0.0432	31.2	156	0.6991	72.8261	52.7711
0.0432	31.6	158	0.6976	69.0217	50.8434
0.0432	32.0	160	0.7152	72.8261	57.5904
0.0432	32.4	162	0.7021	66.8478	56.8675
0.0432	32.8	164	0.6552	67.9348	49.1566
0.0432	33.2	166	0.6351	65.2174	49.3976
0.0432	33.6	168	0.6843	71.7391	60.7229
0.0432	34.0	170	0.6427	69.5652	52.5301
0.0432	34.4	172	0.6155	66.3043	49.3976
0.0432	34.8	174	0.6324	68.4783	57.8313
0.0095	35.2	176	0.6425	65.7609	47.9518
0.0095	35.6	178	0.6384	61.9565	45.3012
0.0095	36.0	180	0.6809	66.3043	46.7470
0.0095	36.4	182	0.7058	72.2826	54.2169
0.0095	36.8	184	0.7125	76.6304	62.1687
0.0095	37.2	186	0.6317	66.3043	45.7831
0.0095	37.6	188	0.6123	58.1522	43.1325
0.0095	38.0	190	0.6409	66.3043	51.3253
0.0095	38.4	192	0.6466	83.6957	58.7952
0.0095	38.8	194	0.6415	64.6739	49.1566
0.0095	39.2	196	0.6324	65.7609	48.4337
0.0095	39.6	198	0.6297	66.8478	46.7470
0.0063	40.0	200	0.6361	69.5652	46.9880

Framework versions

• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.3.1
• Tokenizers 0.21.0