Edit model card

xlsr-big-kannn

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0000
  • Wer: 0.0510

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0004
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 132
  • num_epochs: 100
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
2.0631 1.9704 200 0.6852 0.5241
0.3968 3.9409 400 0.0531 0.1099
0.1256 5.9113 600 0.0184 0.0633
0.0844 7.8818 800 0.0339 0.0643
0.0669 9.8522 1000 0.0070 0.0571
0.05 11.8227 1200 0.0029 0.0545
0.0467 13.7931 1400 0.0049 0.0531
0.0369 15.7635 1600 0.0051 0.0593
0.0267 17.7340 1800 0.0016 0.0529
0.0297 19.7044 2000 0.0010 0.0581
0.0276 21.6749 2200 0.0041 0.0579
0.0211 23.6453 2400 0.0020 0.0525
0.0324 25.6158 2600 0.0091 0.0551
0.0223 27.5862 2800 0.0013 0.0539
0.0182 29.5567 3000 0.0026 0.0551
0.0167 31.5271 3200 0.0010 0.0551
0.0173 33.4975 3400 0.0007 0.0518
0.0178 35.4680 3600 0.0012 0.0510
0.0172 37.4384 3800 0.0008 0.0514
0.0138 39.4089 4000 0.0006 0.0504
0.0122 41.3793 4200 0.0002 0.0512
0.0134 43.3498 4400 0.0003 0.0514
0.0129 45.3202 4600 0.0003 0.0512
0.0075 47.2906 4800 0.0001 0.0512
0.0067 49.2611 5000 0.0001 0.0545
0.0083 51.2315 5200 0.0003 0.0527
0.0067 53.2020 5400 0.0001 0.0525
0.0065 55.1724 5600 0.0004 0.0523
0.0073 57.1429 5800 0.0000 0.0504
0.0051 59.1133 6000 0.0001 0.0510
0.0077 61.0837 6200 0.0006 0.0510
0.0069 63.0542 6400 0.0006 0.0510
0.0058 65.0246 6600 0.0001 0.0506
0.0044 66.9951 6800 0.0003 0.0508
0.0046 68.9655 7000 0.0000 0.0506
0.0049 70.9360 7200 0.0000 0.0508
0.0035 72.9064 7400 0.0001 0.0520
0.0022 74.8768 7600 0.0000 0.0527
0.0039 76.8473 7800 0.0000 0.0518
0.0033 78.8177 8000 0.0000 0.0516
0.0028 80.7882 8200 0.0000 0.0506
0.0024 82.7586 8400 0.0000 0.0510
0.0016 84.7291 8600 0.0000 0.0508
0.0017 86.6995 8800 0.0000 0.0506
0.002 88.6700 9000 0.0000 0.0512
0.0021 90.6404 9200 0.0001 0.0510
0.0014 92.6108 9400 0.0001 0.0508
0.0016 94.5813 9600 0.0000 0.0510
0.0011 96.5517 9800 0.0000 0.0510
0.001 98.5222 10000 0.0000 0.0510

Framework versions

  • Transformers 4.45.0.dev0
  • Pytorch 2.1.2
  • Datasets 2.20.0
  • Tokenizers 0.19.1
Downloads last month
0
Safetensors
Model size
315M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for susmitabhatt/xlsr-big-kannn

Finetuned
(215)
this model