flant5-small-qds2

This model is a fine-tuned version of google/flan-t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2886
• Rouge1: 41.0332
• Rouge2: 16.6948
• Rougel: 33.9525
• Rougelsum: 36.5379
• Gen Len: 19.9258

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.3825	1.0	2325	0.3021	38.4564	14.4311	31.8204	34.1957	19.9382
0.328	2.0	4650	0.2939	40.0077	15.7308	33.0933	35.6447	19.9354
0.3125	3.0	6975	0.2905	40.1746	15.7039	33.053	35.7212	19.9203
0.3029	4.0	9300	0.2892	40.387	16.3077	33.6442	36.1277	19.9203
0.2981	5.0	11625	0.2886	41.0332	16.6948	33.9525	36.5379	19.9258

Framework versions

• Transformers 4.36.1
• Pytorch 2.1.2
• Datasets 2.19.2
• Tokenizers 0.15.2