flant5-small-instructds

This model is a fine-tuned version of google/flan-t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2120
• Rouge1: 51.7099
• Rouge2: 33.4742
• Rougel: 46.0614
• Rougelsum: 48.5535
• Gen Len: 19.8846

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.2918	1.0	2325	0.2253	51.3973	33.8657	45.7963	48.2799	19.8819
0.2463	2.0	4650	0.2169	51.0323	32.9362	45.4849	47.8474	19.8723
0.2335	3.0	6975	0.2135	51.6195	33.4977	46.0379	48.4988	19.875
0.2264	4.0	9300	0.2124	51.332	33.1704	45.6422	48.0759	19.8970
0.2221	5.0	11625	0.2120	51.7099	33.4742	46.0614	48.5535	19.8846

Framework versions

• Transformers 4.36.1
• Pytorch 2.1.2
• Datasets 2.19.2
• Tokenizers 0.15.2