flan-t5-base

This model is a fine-tuned version of google/flan-t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6975

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.004
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	7	0.4313
0.3215	2.0	14	0.4669
0.3041	3.0	21	0.4709
0.3041	4.0	28	0.5289
0.258	5.0	35	0.5150
0.2452	6.0	42	0.4786
0.2452	7.0	49	0.4757
0.2183	8.0	56	0.5042
0.1969	9.0	63	0.4987
0.1908	10.0	70	0.5354
0.1908	11.0	77	0.5388
0.1623	12.0	84	0.5487
0.1539	13.0	91	0.5738
0.1539	14.0	98	0.6919
0.1321	15.0	105	0.6198
0.1246	16.0	112	0.6241
0.1246	17.0	119	0.6865
0.1108	18.0	126	0.6446
0.0989	19.0	133	0.6521
0.1004	20.0	140	0.6652
0.1004	21.0	147	0.6546
0.0897	22.0	154	0.6439
0.0872	23.0	161	0.6435
0.0872	24.0	168	0.6627
0.0808	25.0	175	0.6754
0.0687	26.0	182	0.6632
0.0687	27.0	189	0.6677
0.0664	28.0	196	0.6867
0.0694	29.0	203	0.6936
0.0581	30.0	210	0.6975

Framework versions

• PEFT 0.14.0
• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.3.0
• Tokenizers 0.21.0