jvalline's picture
Model save
848b540 verified
metadata
license: apache-2.0
base_model: t5-base
tags:
  - generated_from_trainer
metrics:
  - bleu
  - wer
model-index:
  - name: 100_randomization_model
    results: []

100_randomization_model

This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 2.6576
  • Bleu: 0.0001
  • Wer: 0.9576
  • Rougel: 0.119
  • Gen Len: 18.9986

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 2
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Bleu Wer Rougel Gen Len
2.5767 0.16 1000 1.6626 0.0001 0.954 0.1251 18.9985
1.9533 0.32 2000 1.5147 0.0001 0.9524 0.1284 18.9986
1.8318 0.48 3000 1.4392 0.0001 0.9518 0.1297 18.9986
1.7626 0.64 4000 1.3857 0.0001 0.9514 0.1306 18.9986
1.7199 0.8 5000 1.3553 0.0001 0.951 0.1312 18.9988
1.6727 0.96 6000 1.3325 0.0001 0.9507 0.1319 18.9986
1.9628 1.12 7000 1.8528 0.0001 0.9524 0.1293 18.9988
2.9138 1.28 8000 2.6299 0.0001 0.9568 0.1205 18.9986
3.5506 1.44 9000 2.7483 0.0001 0.958 0.1181 18.9987
3.5214 1.6 10000 2.7007 0.0001 0.9578 0.1186 18.9986
3.4669 1.76 11000 2.6699 0.0001 0.9576 0.1189 18.9986
3.4448 1.92 12000 2.6576 0.0001 0.9576 0.119 18.9986

Framework versions

  • Transformers 4.37.1
  • Pytorch 2.3.0.dev20240122+cu121
  • Datasets 2.16.1
  • Tokenizers 0.15.1