flan-t5-base

This model is a fine-tuned version of google/flan-t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 50.9550

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	7	52.3732
41.583	2.0	14	52.2166
42.8216	3.0	21	52.0628
42.8216	4.0	28	51.9141
41.7186	5.0	35	51.7700
41.7877	6.0	42	51.6325
41.7877	7.0	49	51.5039
40.9173	8.0	56	51.3844
42.0315	9.0	63	51.2752
41.9766	10.0	70	51.1795
41.9766	11.0	77	51.1012
40.8405	12.0	84	51.0391
42.0735	13.0	91	50.9933
42.0735	14.0	98	50.9652
42.2142	15.0	105	50.9550

Framework versions

• PEFT 0.14.0
• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.3.0
• Tokenizers 0.21.0