Edit model card

flan-t5-small-test

This model is a fine-tuned version of google/flan-t5-small on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4375
  • Rouge1: 55.6701
  • Rouge2: 45.6817
  • Rougel: 52.259
  • Rougelsum: 52.2632
  • Gen Len: 494.295

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 5

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
No log 1.0 100 0.4736 47.5356 37.7662 43.8872 43.933 493.725
No log 2.0 200 0.4582 51.7559 41.8487 48.1235 48.1762 494.835
No log 3.0 300 0.4469 52.8576 43.1039 49.3153 49.36 493.225
No log 4.0 400 0.4395 55.4214 45.3968 51.8613 51.8725 492.5
0.5066 5.0 500 0.4375 55.6701 45.6817 52.259 52.2632 494.295

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.16.1
  • Tokenizers 0.15.1
Downloads last month
2
Safetensors
Model size
77M params
Tensor type
F32
·

Finetuned from