flan-t5-base

This model is a fine-tuned version of google/flan-t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3298

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	7	1.2308
2.3868	2.0	14	0.9288
1.6775	3.0	21	0.7270
1.6775	4.0	28	0.6608
1.2722	5.0	35	0.5970
1.0065	6.0	42	0.5621
1.0065	7.0	49	0.4971
0.8962	8.0	56	0.4758
0.7643	9.0	63	0.4460
0.6697	10.0	70	0.4292
0.6697	11.0	77	0.4073
0.6176	12.0	84	0.3810
0.6032	13.0	91	0.3783
0.6032	14.0	98	0.3697
0.5406	15.0	105	0.3693
0.5269	16.0	112	0.3619
0.5269	17.0	119	0.3523
0.4787	18.0	126	0.3423
0.4701	19.0	133	0.3387
0.4445	20.0	140	0.3350
0.4445	21.0	147	0.3357
0.4365	22.0	154	0.3318
0.4185	23.0	161	0.3312
0.4185	24.0	168	0.3315
0.4152	25.0	175	0.3310
0.3922	26.0	182	0.3308
0.3922	27.0	189	0.3300
0.3979	28.0	196	0.3301
0.3855	29.0	203	0.3300
0.3908	30.0	210	0.3298

Framework versions

• PEFT 0.14.0
• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.3.0
• Tokenizers 0.21.0