Hugging Face's logo

Rolv-Arild
/
xls-r-300m-npsc-seq2seq

Automatic Speech Recognition
Transformers
PyTorch
TensorBoard
speech-encoder-decoder
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
xls-r-300m-npsc-seq2seq / README.md
Rolv-Arild's picture
Rolv-Arild
update model card README.md
0020927
preview code
|
raw
history blame
3.64 kB
metadata
tags:
  - generated_from_trainer
model-index:
  - name: ''
    results: []

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.3350
  • Wer: 0.3441

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 400
  • num_epochs: 20.0
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
2.2943 0.51 400 4.4367 0.9745
2.8816 1.02 800 2.8103 1.2576
2.7842 1.53 1200 2.6832 1.1330
2.313 2.04 1600 2.6678 1.1348
2.2897 2.55 2000 2.5744 1.2102
1.3821 3.06 2400 2.9908 0.9377
1.074 3.57 2800 2.6649 0.8966
1.3643 4.08 3200 1.0064 0.7236
0.8286 4.59 3600 0.6339 0.5454
0.3872 5.1 4000 0.5170 0.4718
0.3654 5.61 4400 0.4386 0.4420
0.2672 6.12 4800 0.5186 0.4679
0.2519 6.63 5200 0.4238 0.4177
0.3293 7.14 5600 0.3584 0.3970
0.314 7.65 6000 0.3325 0.3911
0.1698 8.16 6400 0.3411 0.3855
0.1682 8.67 6800 0.3239 0.3801
0.1325 9.18 7200 0.3474 0.3832
0.1577 9.69 7600 0.3289 0.3839
0.2259 10.2 8000 0.3183 0.3756
0.2473 10.71 8400 0.3132 0.3654
0.1136 11.22 8800 0.3242 0.3670
0.108 11.73 9200 0.3201 0.3601
0.0806 12.24 9600 0.3223 0.3609
0.0896 12.75 10000 0.3228 0.3584
0.1642 13.27 10400 0.3140 0.3547
0.1442 13.77 10800 0.3235 0.3539
0.0802 14.29 11200 0.3175 0.3553
0.0747 14.8 11600 0.3126 0.3512
0.0488 15.31 12000 0.3292 0.3525
0.0469 15.82 12400 0.3231 0.3504
0.1021 16.33 12800 0.3230 0.3502
0.0841 16.84 13200 0.3348 0.3513
0.0502 17.35 13600 0.3318 0.3486
0.059 17.86 14000 0.3359 0.3462
0.0417 18.37 14400 0.3310 0.3467
0.0322 18.88 14800 0.3325 0.3467
0.0531 19.39 15200 0.3357 0.3449
0.0623 19.9 15600 0.3350 0.3441

Framework versions

  • Transformers 4.17.0.dev0
  • Pytorch 1.10.0+cu113
  • Datasets 1.18.1
  • Tokenizers 0.11.0