T5LegalAbstractiveSummarization

This model is a fine-tuned version of AathifMohammed/t5baseflan on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.3609
• Rouge1: 46.0257
• Rouge2: 19.7049
• Rougel: 27.1994
• Rougelsum: 41.5335
• Gen Len: 269.53

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	32	2.3728	46.5257	20.1572	27.2915	41.9231	268.37
No log	2.0	64	2.3723	46.6356	20.2894	27.4219	42.0149	267.67
No log	3.0	96	2.3711	46.6278	20.272	27.4113	42.0144	268.32
No log	4.0	128	2.3692	46.5747	20.2018	27.4114	41.8886	270.68
No log	5.0	160	2.3671	46.3585	19.8511	27.1805	41.627	271.06
No log	6.0	192	2.3652	46.1954	19.7736	27.1616	41.5316	270.98
No log	7.0	224	2.3636	46.1564	19.6929	27.1374	41.5196	269.82
No log	8.0	256	2.3622	46.1804	19.7607	27.2296	41.6105	270.93
No log	9.0	288	2.3613	45.9359	19.5433	27.1402	41.4038	269.49
No log	10.0	320	2.3609	46.0257	19.7049	27.1994	41.5335	269.53

Framework versions

• PEFT 0.10.1.dev0
• Transformers 4.40.0.dev0
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2