---
language:
- 'no'
license: apache-2.0
tags:
- audio
- asr
- automatic-speech-recognition
- hf-asr-leaderboard
model-index:
- name: nb-whisper-small-publicbeta-25k
  results: []
---

<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# nb-whisper-small-publicbeta-25k

This model is a fine-tuned version of [openai/whisper-small](https://huggingface.co/openai/whisper-small) on the NbAiLab/ncc_speech2 dataset.

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- lr_scheduler_type: linear
- per_device_train_batch_size: 32
- total_train_batch_size_per_node: 128
- total_train_batch_size: 1024
- total_optimization_steps: 25,000
- starting_optimization_step: None
- finishing_optimization_step: 25,000
- num_train_dataset_workers: 32
- num_hosts: 8
- total_num_training_examples: 25,600,000
- steps_per_epoch: 7313
- num_beams: 5
- weight_decay: 0.01
- adam_beta1: 0.9
- adam_beta2: 0.98
- adam_epsilon: 1e-06
- dropout: True
- bpe_dropout_probability: 0.1
- activation_dropout_probability: 0.1

### Training results

| step  | validation_fleurs_loss | train_loss | validation_fleurs_wer | validation_fleurs_cer | validation_fleurs_exact_wer | validation_fleurs_exact_cer | validation_stortinget_loss | validation_stortinget_wer | validation_stortinget_cer | validation_stortinget_exact_wer | validation_stortinget_exact_cer |
|:-----:|:----------------------:|:----------:|:---------------------:|:---------------------:|:---------------------------:|:---------------------------:|:--------------------------:|:-------------------------:|:-------------------------:|:-------------------------------:|:-------------------------------:|
| 0     | 1.2013                 | 3.1115     | 218.8876              | 174.4279              | 388.7694                    | 278.9901                    | 1.4191                     | 71.3727                   | 46.4810                   | 76.7531                         | 49.0057                         |
| 1000  | 0.5627                 | 1.1938     | 16.3593               | 6.2586                | 20.0717                     | 7.2820                      | 0.4640                     | 20.7725                   | 11.8840                   | 24.4401                         | 12.5992                         |
| 2000  | 0.3961                 | 0.9944     | 11.7192               | 4.0146                | 15.4719                     | 4.9384                      | 0.3737                     | 16.5674                   | 10.1748                   | 20.0976                         | 10.8109                         |
| 3000  | 0.3696                 | 0.9185     | 10.8269               | 4.1576                | 14.7551                     | 5.1220                      | 0.3426                     | 14.9167                   | 9.5103                    | 18.3471                         | 10.1061                         |
| 4000  | 0.3467                 | 0.8298     | 9.7858                | 4.2513                | 13.6201                     | 5.1558                      | 0.3251                     | 14.3438                   | 9.2267                    | 17.7666                         | 9.8219                          |
| 5000  | 0.3266                 | 0.8400     | 10.0833               | 4.2711                | 13.8889                     | 5.2138                      | 0.3110                     | 13.9022                   | 9.1039                    | 17.2299                         | 9.6697                          |
| 6000  | 0.3280                 | 0.7875     | 8.7745                | 3.3636                | 12.6344                     | 4.3295                      | 0.3058                     | 13.5598                   | 8.8853                    | 16.9561                         | 9.4543                          |
| 7000  | 0.3177                 | 0.7937     | 8.5961                | 3.7581                | 12.7539                     | 4.6775                      | 0.2991                     | 13.1425                   | 8.6226                    | 16.4905                         | 9.1878                          |
| 8000  | 0.3383                 | 0.7872     | 8.8935                | 3.8666                | 12.9630                     | 4.7934                      | 0.2917                     | 13.0831                   | 8.6552                    | 16.4486                         | 9.2255                          |
| 9000  | 0.3320                 | 0.7526     | 9.1612                | 4.0738                | 13.0526                     | 5.0495                      | 0.2899                     | 12.8380                   | 8.4996                    | 16.1350                         | 9.0495                          |
| 10000 | 0.3267                 | 0.7547     | 9.5181                | 4.1280                | 13.3513                     | 5.1462                      | 0.2894                     | 12.7106                   | 8.4593                    | 16.0502                         | 9.0189                          |
| 11000 | 0.3358                 | 0.7120     | 9.0125                | 4.1379                | 13.4409                     | 5.1703                      | 0.2889                     | 12.8828                   | 8.5885                    | 16.1915                         | 9.1459                          |
| 12000 | 0.3179                 | 0.7387     | 9.1910                | 4.2563                | 13.5006                     | 5.2331                      | 0.2825                     | 12.6795                   | 8.4383                    | 16.0152                         | 8.9950                          |
| 13000 | 0.3152                 | 0.7295     | 8.7448                | 4.0541                | 12.7539                     | 4.9529                      | 0.2832                     | 12.5267                   | 8.4567                    | 15.8700                         | 9.0105                          |


### Framework versions

- Transformers 4.31.0.dev0
- Datasets 2.13.0
- Tokenizers 0.13.3