RohanHBTU's picture
End of training
43d9f2e verified
metadata
license: apache-2.0
base_model: t5-base
tags:
  - generated_from_trainer
metrics:
  - bleu
model-index:
  - name: t5-base-finetuned-question-to-answer
    results: []

t5-base-finetuned-question-to-answer

This model is a fine-tuned version of t5-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4006
  • Bleu: 54.0167
  • Gen Len: 28.902

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 16
  • eval_batch_size: 4
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 25
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Bleu Gen Len
1.3089 1.0 516 0.8868 35.5598 34.108
1.2622 2.0 1032 0.8313 37.1928 34.906
1.2093 3.0 1548 0.7822 40.5334 31.082
1.1607 4.0 2064 0.7350 41.6835 32.294
1.1269 5.0 2580 0.6991 41.3956 31.084
1.0765 6.0 3096 0.6644 43.152 31.324
1.0551 7.0 3612 0.6305 45.2289 30.064
1.0326 8.0 4128 0.5984 44.9963 30.856
0.9974 9.0 4644 0.5723 45.8182 30.08
0.9847 10.0 5160 0.5474 46.6307 28.812
0.9553 11.0 5676 0.5245 47.3503 30.256
0.9363 12.0 6192 0.5059 48.8164 29.258
0.9218 13.0 6708 0.4872 49.1785 30.37
0.9096 14.0 7224 0.4743 49.7033 29.48
0.8852 15.0 7740 0.4551 50.9333 30.21
0.886 16.0 8256 0.4456 51.7962 28.472
0.8694 17.0 8772 0.4351 51.9603 29.89
0.8785 18.0 9288 0.4250 52.3147 29.17
0.8606 19.0 9804 0.4158 52.5438 28.96
0.8632 20.0 10320 0.4082 53.7264 28.85
0.8549 21.0 10836 0.4037 53.6781 28.446
0.8608 22.0 11352 0.4017 53.8526 29.088
0.8644 23.0 11868 0.3999 53.8358 28.47
0.8589 24.0 12384 0.3987 53.949 28.792
0.8699 25.0 12900 0.4006 54.0167 28.902

Framework versions

  • Transformers 4.36.2
  • Pytorch 2.0.0
  • Datasets 2.1.0
  • Tokenizers 0.15.0