Hugging Face's logo

asr-africa
/
w2v-bert-2.0-lg-CV-Fleurs-20hrs-v10

Automatic Speech Recognition
Transformers
Safetensors
wav2vec2-bert
Generated from Trainer
Inference Endpoints
Model card Files Community
w2v-bert-2.0-lg-CV-Fleurs-20hrs-v10 / README.md
sulaimank's picture
sulaimank
End of training
db25cd1 verified
preview code
|
raw
history blame contribute delete
3.93 kB
metadata
library_name: transformers
license: mit
base_model: facebook/w2v-bert-2.0
tags:
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: w2v-bert-2.0-lg-CV-Fleurs-20hrs-v10
    results: []

w2v-bert-2.0-lg-CV-Fleurs-20hrs-v10

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5879
  • Wer: 0.3384
  • Cer: 0.0706

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-05
  • train_batch_size: 8
  • eval_batch_size: 4
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 16
  • optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • num_epochs: 100
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Cer
0.6539 0.9992 664 0.3779 0.4182 0.0890
0.2976 2.0 1329 0.3493 0.4013 0.0823
0.2421 2.9992 1993 0.3409 0.3655 0.0774
0.2038 4.0 2658 0.3221 0.3604 0.0739
0.1789 4.9992 3322 0.3093 0.3711 0.0767
0.158 6.0 3987 0.3472 0.3584 0.0740
0.1388 6.9992 4651 0.3467 0.3763 0.0790
0.1245 8.0 5316 0.3358 0.3644 0.0753
0.1087 8.9992 5980 0.3607 0.3468 0.0724
0.0933 10.0 6645 0.3673 0.3454 0.0719
0.0815 10.9992 7309 0.3767 0.3507 0.0728
0.0684 12.0 7974 0.4179 0.3452 0.0728
0.058 12.9992 8638 0.4341 0.3524 0.0743
0.0516 14.0 9303 0.4370 0.3647 0.0744
0.0423 14.9992 9967 0.4761 0.3587 0.0752
0.0385 16.0 10632 0.4817 0.3432 0.0729
0.033 16.9992 11296 0.4761 0.3575 0.0760
0.0316 18.0 11961 0.5045 0.3485 0.0733
0.026 18.9992 12625 0.5423 0.3482 0.0731
0.0235 20.0 13290 0.5298 0.3442 0.0716
0.0213 20.9992 13954 0.5624 0.3332 0.0715
0.0206 22.0 14619 0.5381 0.3428 0.0711
0.0186 22.9992 15283 0.5958 0.3461 0.0734
0.0176 24.0 15948 0.5234 0.3472 0.0727
0.016 24.9992 16612 0.5626 0.3438 0.0726
0.0155 26.0 17277 0.5592 0.3448 0.0737
0.0137 26.9992 17941 0.5726 0.3357 0.0708
0.0127 28.0 18606 0.5851 0.3371 0.0713
0.0116 28.9992 19270 0.6023 0.3422 0.0720
0.0118 30.0 19935 0.5801 0.3402 0.0732
0.011 30.9992 20599 0.5811 0.3430 0.0721
0.01 32.0 21264 0.5863 0.3412 0.0722
0.0101 32.9992 21928 0.5879 0.3384 0.0706

Framework versions

  • Transformers 4.46.3
  • Pytorch 2.1.0+cu118
  • Datasets 3.1.0
  • Tokenizers 0.20.3