Edit model card

w2v-bert-2.0-tamil-gpu-custom_v10

This model is a fine-tuned version of facebook/w2v-bert-2.0 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: inf
  • Wer: 0.4032

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 4.43567e-05
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • num_epochs: 5
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
2.4046 0.24 300 inf 0.3596
0.5204 0.49 600 inf 0.3451
0.4297 0.73 900 inf 0.3272
0.3891 0.97 1200 inf 0.3477
0.6568 1.22 1500 inf 0.3937
0.862 1.46 1800 inf 0.4033
0.9171 1.71 2100 inf 0.4032
0.9643 1.95 2400 inf 0.4032
0.9568 2.19 2700 inf 0.4032
0.8953 2.44 3000 inf 0.4032
0.9372 2.68 3300 inf 0.4032
0.9671 2.92 3600 inf 0.4032
0.9527 3.17 3900 inf 0.4032
0.8851 3.41 4200 inf 0.4032
0.8781 3.65 4500 inf 0.4032
0.8971 3.9 4800 inf 0.4032
0.8623 4.14 5100 inf 0.4032
0.9137 4.38 5400 inf 0.4032
0.8969 4.63 5700 inf 0.4032
0.8769 4.87 6000 inf 0.4032

Framework versions

  • Transformers 4.39.3
  • Pytorch 2.1.2+cu121
  • Datasets 2.18.0
  • Tokenizers 0.15.2
Downloads last month
37
Safetensors
Model size
606M params
Tensor type
F32
·

Finetuned from