whisper-small-si-bank-v3

This model is a fine-tuned version of openai/whisper-small on the sinhala-bank-speech dataset. It achieves the following results on the evaluation set:

• Loss: 0.7054
• Wer Ortho: 64.1791
• Wer: 47.4041

Model description

The IshanSuga/whisper-small-si-bank-v3 model is an Automatic Speech Recognition (ASR) system fine-tuned on Sinhala speech data, specifically tailored for banking-related conversations. Built upon OpenAI’s Whisper-Small architecture, this model leverages transfer learning to enhance transcription accuracy for Sinhala audio.

Key Features:

Pretrained Foundation: Based on OpenAI’s Whisper-Small, a robust ASR model trained on a large multilingual dataset.

Fine-tuned for Sinhala: Adapted using domain-specific Sinhala banking conversations to improve transcription quality in financial settings.

Intended uses & limitations

Use Cases:

Banking Chatbots & Assistants – Enables automated customer service and inquiry handling.

Speech-to-Text Applications – Converts Sinhala speech into text for documentation and accessibility.

Financial Call Center Analytics – Helps in analyzing and transcribing customer interactions.

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant_with_warmup
• lr_scheduler_warmup_steps: 50
• training_steps: 250
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Ortho	Wer
No log	0.4	2	2.4227	196.5174	222.3476
No log	0.8	4	2.4223	184.0796	225.9594
No log	1.2	6	2.4041	178.1095	230.6998
No log	1.6	8	2.3475	200.0	235.8916
No log	2.0	10	2.2372	204.4776	230.6998
No log	2.4	12	2.1041	226.8657	223.7020
No log	2.8	14	2.0246	243.7811	231.6027
No log	3.2	16	1.9415	251.7413	225.0564
No log	3.6	18	1.8591	261.6915	219.8646
No log	4.0	20	1.7851	254.2289	222.3476
No log	4.4	22	1.7161	180.5970	219.4131
No log	4.8	24	1.6398	179.6020	219.6388
2.0087	5.2	26	1.5743	162.1891	224.1535
2.0087	5.6	28	1.5253	161.1940	229.5711
2.0087	6.0	30	1.4871	148.2587	234.3115
2.0087	6.4	32	1.4461	138.8060	182.3928
2.0087	6.8	34	1.4145	126.8657	156.2077
2.0087	7.2	36	1.3807	114.9254	148.0813
2.0087	7.6	38	1.3431	127.8607	153.0474
2.0087	8.0	40	1.2989	102.4876	144.2438
2.0087	8.4	42	1.2561	96.5174	127.3138
2.0087	8.8	44	1.2180	99.0050	119.1874
2.0087	9.2	46	1.1857	93.0348	116.9300
2.0087	9.6	48	1.1512	94.5274	91.1964
1.1238	10.0	50	1.1099	86.5672	99.5485
1.1238	10.4	52	1.0849	83.5821	113.0926
1.1238	10.8	54	1.0564	83.0846	81.9413
1.1238	11.2	56	1.0203	85.0746	104.7404
1.1238	11.6	58	0.9934	82.5871	97.9684
1.1238	12.0	60	0.9693	84.0796	82.1670
1.1238	12.4	62	0.9399	83.0846	80.8126
1.1238	12.8	64	0.8985	80.0995	71.3318
1.1238	13.2	66	0.8841	78.1095	68.3973
1.1238	13.6	68	0.8707	77.6119	71.1061
1.1238	14.0	70	0.8469	77.1144	69.5260
1.1238	14.4	72	0.8612	77.1144	101.8059
1.1238	14.8	74	0.8603	73.6318	57.7878
0.4443	15.2	76	0.8440	75.1244	57.5621
0.4443	15.6	78	0.8534	72.6368	82.1670
0.4443	16.0	80	0.8377	74.6269	60.0451
0.4443	16.4	82	0.8370	71.1443	67.7201
0.4443	16.8	84	0.8740	71.1443	69.5260
0.4443	17.2	86	0.8431	71.1443	58.4650
0.4443	17.6	88	0.8481	75.6219	62.7540
0.4443	18.0	90	0.8321	75.1244	66.8172
0.4443	18.4	92	0.8490	69.6517	55.3047
0.4443	18.8	94	0.8685	73.1343	66.1400
0.4443	19.2	96	0.8263	72.6368	59.1422
0.4443	19.6	98	0.8659	74.6269	58.9165
0.1597	20.0	100	0.8714	72.6368	60.9481
0.1597	20.4	102	0.8245	74.1294	53.9503
0.1597	20.8	104	0.8476	70.1493	51.4673
0.1597	21.2	106	0.8953	72.6368	57.1106
0.1597	21.6	108	0.8871	70.6468	55.5305
0.1597	22.0	110	0.8335	69.1542	53.0474
0.1597	22.4	112	0.9095	71.6418	80.1354
0.1597	22.8	114	0.9014	97.0149	83.9729
0.1597	23.2	116	0.7924	77.1144	53.9503
0.1597	23.6	118	0.8353	66.6667	48.7585
0.1597	24.0	120	0.9749	69.6517	63.6569
0.1597	24.4	122	0.8675	71.1443	53.9503
0.1597	24.8	124	0.7980	69.6517	50.3386
0.1073	25.2	126	0.8297	69.6517	59.5937
0.1073	25.6	128	0.9054	120.3980	79.0068
0.1073	26.0	130	0.8254	67.6617	51.6930
0.1073	26.4	132	0.8260	67.1642	49.8871
0.1073	26.8	134	0.8305	68.1592	49.6614
0.1073	27.2	136	0.8382	66.6667	54.4018
0.1073	27.6	138	0.8302	68.1592	58.6907
0.1073	28.0	140	0.7815	71.6418	55.9819
0.1073	28.4	142	0.7478	67.6617	48.9842
0.1073	28.8	144	0.7830	76.6169	57.7878
0.1073	29.2	146	0.7150	67.1642	48.7585
0.1073	29.6	148	0.7300	68.6567	48.3070
0.0584	30.0	150	0.7099	70.6468	51.2415
0.0584	30.4	152	0.7285	66.6667	47.6298
0.0584	30.8	154	0.7717	68.6567	60.7223
0.0584	31.2	156	0.8301	67.6617	54.6275
0.0584	31.6	158	0.7430	68.1592	56.4334
0.0584	32.0	160	0.6981	67.6617	47.8555
0.0584	32.4	162	0.7349	66.6667	55.3047
0.0584	32.8	164	0.7662	63.6816	49.6614
0.0584	33.2	166	0.8003	66.6667	55.0790
0.0584	33.6	168	0.7360	63.6816	48.0813
0.0584	34.0	170	0.7166	67.1642	49.8871
0.0584	34.4	172	0.7899	72.6368	57.5621
0.0584	34.8	174	0.7921	81.0945	58.6907
0.0069	35.2	176	0.7251	67.6617	45.3725
0.0069	35.6	178	0.7142	71.1443	51.9187
0.0069	36.0	180	0.7564	74.6269	61.1738
0.0069	36.4	182	0.7012	70.1493	54.4018
0.0069	36.8	184	0.6962	65.1741	43.7923
0.0069	37.2	186	0.7316	66.6667	50.3386
0.0069	37.6	188	0.7482	67.1642	46.9526
0.0069	38.0	190	0.6728	60.6965	40.6321
0.0069	38.4	192	0.6576	63.1841	45.8239
0.0069	38.8	194	0.7134	67.1642	52.1445
0.0069	39.2	196	0.6794	69.1542	50.3386
0.0069	39.6	198	0.6344	67.1642	47.8555
0.0062	40.0	200	0.6440	60.6965	42.6637
0.0062	40.4	202	0.7188	66.1692	49.8871
0.0062	40.8	204	0.7213	64.1791	44.9210
0.0062	41.2	206	0.6740	64.6766	42.2122
0.0062	41.6	208	0.6472	61.6915	41.5350
0.0062	42.0	210	0.6664	60.6965	43.3409
0.0062	42.4	212	0.7102	59.7015	44.2438
0.0062	42.8	214	0.7214	60.6965	47.4041
0.0062	43.2	216	0.6949	63.6816	50.1129
0.0062	43.6	218	0.6944	62.6866	42.4379
0.0062	44.0	220	0.6759	64.1791	44.0181
0.0062	44.4	222	0.6834	65.6716	45.5982
0.0062	44.8	224	0.7508	67.6617	53.2731
0.0042	45.2	226	0.7720	72.6368	67.9458
0.0042	45.6	228	0.7001	65.6716	50.5643
0.0042	46.0	230	0.6872	66.6667	45.3725
0.0042	46.4	232	0.7127	67.6617	46.2754
0.0042	46.8	234	0.7334	65.6716	48.3070
0.0042	47.2	236	0.7485	62.1891	45.3725
0.0042	47.6	238	0.7175	64.1791	49.8871
0.0042	48.0	240	0.7251	63.6816	50.1129
0.0042	48.4	242	0.6949	62.6866	48.3070
0.0042	48.8	244	0.6684	60.6965	44.2438
0.0042	49.2	246	0.6712	62.1891	48.0813
0.0042	49.6	248	0.6939	66.1692	50.1129
0.0034	50.0	250	0.7054	64.1791	47.4041

Framework versions

• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.3.1
• Tokenizers 0.21.0