whisper-small-si-bank-v3

This model is a fine-tuned version of openai/whisper-small on the sinhala-bank-speech dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7054
  • Wer Ortho: 64.1791
  • Wer: 47.4041

Model description

The IshanSuga/whisper-small-si-bank-v3 model is an Automatic Speech Recognition (ASR) system fine-tuned on Sinhala speech data, specifically tailored for banking-related conversations. Built upon OpenAI’s Whisper-Small architecture, this model leverages transfer learning to enhance transcription accuracy for Sinhala audio.

Key Features:

Pretrained Foundation: Based on OpenAI’s Whisper-Small, a robust ASR model trained on a large multilingual dataset.

Fine-tuned for Sinhala: Adapted using domain-specific Sinhala banking conversations to improve transcription quality in financial settings.

Intended uses & limitations

Use Cases:

Banking Chatbots & Assistants – Enables automated customer service and inquiry handling.

Speech-to-Text Applications – Converts Sinhala speech into text for documentation and accessibility.

Financial Call Center Analytics – Helps in analyzing and transcribing customer interactions.

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: constant_with_warmup
  • lr_scheduler_warmup_steps: 50
  • training_steps: 250
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Ortho Wer
No log 0.4 2 2.4227 196.5174 222.3476
No log 0.8 4 2.4223 184.0796 225.9594
No log 1.2 6 2.4041 178.1095 230.6998
No log 1.6 8 2.3475 200.0 235.8916
No log 2.0 10 2.2372 204.4776 230.6998
No log 2.4 12 2.1041 226.8657 223.7020
No log 2.8 14 2.0246 243.7811 231.6027
No log 3.2 16 1.9415 251.7413 225.0564
No log 3.6 18 1.8591 261.6915 219.8646
No log 4.0 20 1.7851 254.2289 222.3476
No log 4.4 22 1.7161 180.5970 219.4131
No log 4.8 24 1.6398 179.6020 219.6388
2.0087 5.2 26 1.5743 162.1891 224.1535
2.0087 5.6 28 1.5253 161.1940 229.5711
2.0087 6.0 30 1.4871 148.2587 234.3115
2.0087 6.4 32 1.4461 138.8060 182.3928
2.0087 6.8 34 1.4145 126.8657 156.2077
2.0087 7.2 36 1.3807 114.9254 148.0813
2.0087 7.6 38 1.3431 127.8607 153.0474
2.0087 8.0 40 1.2989 102.4876 144.2438
2.0087 8.4 42 1.2561 96.5174 127.3138
2.0087 8.8 44 1.2180 99.0050 119.1874
2.0087 9.2 46 1.1857 93.0348 116.9300
2.0087 9.6 48 1.1512 94.5274 91.1964
1.1238 10.0 50 1.1099 86.5672 99.5485
1.1238 10.4 52 1.0849 83.5821 113.0926
1.1238 10.8 54 1.0564 83.0846 81.9413
1.1238 11.2 56 1.0203 85.0746 104.7404
1.1238 11.6 58 0.9934 82.5871 97.9684
1.1238 12.0 60 0.9693 84.0796 82.1670
1.1238 12.4 62 0.9399 83.0846 80.8126
1.1238 12.8 64 0.8985 80.0995 71.3318
1.1238 13.2 66 0.8841 78.1095 68.3973
1.1238 13.6 68 0.8707 77.6119 71.1061
1.1238 14.0 70 0.8469 77.1144 69.5260
1.1238 14.4 72 0.8612 77.1144 101.8059
1.1238 14.8 74 0.8603 73.6318 57.7878
0.4443 15.2 76 0.8440 75.1244 57.5621
0.4443 15.6 78 0.8534 72.6368 82.1670
0.4443 16.0 80 0.8377 74.6269 60.0451
0.4443 16.4 82 0.8370 71.1443 67.7201
0.4443 16.8 84 0.8740 71.1443 69.5260
0.4443 17.2 86 0.8431 71.1443 58.4650
0.4443 17.6 88 0.8481 75.6219 62.7540
0.4443 18.0 90 0.8321 75.1244 66.8172
0.4443 18.4 92 0.8490 69.6517 55.3047
0.4443 18.8 94 0.8685 73.1343 66.1400
0.4443 19.2 96 0.8263 72.6368 59.1422
0.4443 19.6 98 0.8659 74.6269 58.9165
0.1597 20.0 100 0.8714 72.6368 60.9481
0.1597 20.4 102 0.8245 74.1294 53.9503
0.1597 20.8 104 0.8476 70.1493 51.4673
0.1597 21.2 106 0.8953 72.6368 57.1106
0.1597 21.6 108 0.8871 70.6468 55.5305
0.1597 22.0 110 0.8335 69.1542 53.0474
0.1597 22.4 112 0.9095 71.6418 80.1354
0.1597 22.8 114 0.9014 97.0149 83.9729
0.1597 23.2 116 0.7924 77.1144 53.9503
0.1597 23.6 118 0.8353 66.6667 48.7585
0.1597 24.0 120 0.9749 69.6517 63.6569
0.1597 24.4 122 0.8675 71.1443 53.9503
0.1597 24.8 124 0.7980 69.6517 50.3386
0.1073 25.2 126 0.8297 69.6517 59.5937
0.1073 25.6 128 0.9054 120.3980 79.0068
0.1073 26.0 130 0.8254 67.6617 51.6930
0.1073 26.4 132 0.8260 67.1642 49.8871
0.1073 26.8 134 0.8305 68.1592 49.6614
0.1073 27.2 136 0.8382 66.6667 54.4018
0.1073 27.6 138 0.8302 68.1592 58.6907
0.1073 28.0 140 0.7815 71.6418 55.9819
0.1073 28.4 142 0.7478 67.6617 48.9842
0.1073 28.8 144 0.7830 76.6169 57.7878
0.1073 29.2 146 0.7150 67.1642 48.7585
0.1073 29.6 148 0.7300 68.6567 48.3070
0.0584 30.0 150 0.7099 70.6468 51.2415
0.0584 30.4 152 0.7285 66.6667 47.6298
0.0584 30.8 154 0.7717 68.6567 60.7223
0.0584 31.2 156 0.8301 67.6617 54.6275
0.0584 31.6 158 0.7430 68.1592 56.4334
0.0584 32.0 160 0.6981 67.6617 47.8555
0.0584 32.4 162 0.7349 66.6667 55.3047
0.0584 32.8 164 0.7662 63.6816 49.6614
0.0584 33.2 166 0.8003 66.6667 55.0790
0.0584 33.6 168 0.7360 63.6816 48.0813
0.0584 34.0 170 0.7166 67.1642 49.8871
0.0584 34.4 172 0.7899 72.6368 57.5621
0.0584 34.8 174 0.7921 81.0945 58.6907
0.0069 35.2 176 0.7251 67.6617 45.3725
0.0069 35.6 178 0.7142 71.1443 51.9187
0.0069 36.0 180 0.7564 74.6269 61.1738
0.0069 36.4 182 0.7012 70.1493 54.4018
0.0069 36.8 184 0.6962 65.1741 43.7923
0.0069 37.2 186 0.7316 66.6667 50.3386
0.0069 37.6 188 0.7482 67.1642 46.9526
0.0069 38.0 190 0.6728 60.6965 40.6321
0.0069 38.4 192 0.6576 63.1841 45.8239
0.0069 38.8 194 0.7134 67.1642 52.1445
0.0069 39.2 196 0.6794 69.1542 50.3386
0.0069 39.6 198 0.6344 67.1642 47.8555
0.0062 40.0 200 0.6440 60.6965 42.6637
0.0062 40.4 202 0.7188 66.1692 49.8871
0.0062 40.8 204 0.7213 64.1791 44.9210
0.0062 41.2 206 0.6740 64.6766 42.2122
0.0062 41.6 208 0.6472 61.6915 41.5350
0.0062 42.0 210 0.6664 60.6965 43.3409
0.0062 42.4 212 0.7102 59.7015 44.2438
0.0062 42.8 214 0.7214 60.6965 47.4041
0.0062 43.2 216 0.6949 63.6816 50.1129
0.0062 43.6 218 0.6944 62.6866 42.4379
0.0062 44.0 220 0.6759 64.1791 44.0181
0.0062 44.4 222 0.6834 65.6716 45.5982
0.0062 44.8 224 0.7508 67.6617 53.2731
0.0042 45.2 226 0.7720 72.6368 67.9458
0.0042 45.6 228 0.7001 65.6716 50.5643
0.0042 46.0 230 0.6872 66.6667 45.3725
0.0042 46.4 232 0.7127 67.6617 46.2754
0.0042 46.8 234 0.7334 65.6716 48.3070
0.0042 47.2 236 0.7485 62.1891 45.3725
0.0042 47.6 238 0.7175 64.1791 49.8871
0.0042 48.0 240 0.7251 63.6816 50.1129
0.0042 48.4 242 0.6949 62.6866 48.3070
0.0042 48.8 244 0.6684 60.6965 44.2438
0.0042 49.2 246 0.6712 62.1891 48.0813
0.0042 49.6 248 0.6939 66.1692 50.1129
0.0034 50.0 250 0.7054 64.1791 47.4041

Framework versions

  • Transformers 4.48.3
  • Pytorch 2.5.1+cu124
  • Datasets 3.3.1
  • Tokenizers 0.21.0
Downloads last month
92
Safetensors
Model size
242M params
Tensor type
F32
·
Inference Providers NEW
or

Model tree for IshanSuga/whisper-small-si-bank-v3

Finetuned
(2370)
this model

Dataset used to train IshanSuga/whisper-small-si-bank-v3