t5-small-finetuned-billsum_4Epochs

This model is a fine-tuned version of t5-small on the billsum dataset. It achieves the following results on the evaluation set:

• Loss: 2.9033
• Rouge1: 21.3676
• Rouge2: 13.1573
• Rougel: 18.9799
• Rougelsum: 19.4314
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
3.2306	1.0	1185	2.9474	21.1715	12.9398	18.8515	19.2733	19.0
3.1802	2.0	2370	2.9134	21.3066	13.1019	18.9418	19.3806	19.0
3.1553	3.0	3555	2.9033	21.3676	13.1573	18.9799	19.4314	19.0
3.1557	4.0	4740	2.9033	21.3676	13.1573	18.9799	19.4314	19.0

Framework versions

• Transformers 4.28.1
• Pytorch 2.0.0+cu118
• Datasets 2.11.0
• Tokenizers 0.13.3