---
library_name: transformers
license: cc
datasets:
- billsum
language:
- en
metrics:
- rouge
pipeline_tag: summarization
---

# Model Card for Model ID

This is the T5 Model on that is trained on the BillSum dataset.</br>
Trained on the Google Colab's T4 GPU </br>

# Training Arguments

training_args = Seq2SeqTrainingArguments(</br>
    "bert-on-the-billsum",</br>
    evaluation_strategy="epoch",</br>
    learning_rate=2e-5,</br>
    per_device_train_batch_size=2,</br>
    per_device_eval_batch_size=2, </br>
    weight_decay=0.01, </br>
    save_total_limit=3, </br>
    num_train_epochs=4, </br>
    predict_with_generate=True, </br>
    fp16=True, </br>
    push_to_hub=True, </br>
)

# Training Results

Epoch	Training_Loss	Validation_Loss	Rouge1	    Rouge2	    Rougel	    Rougelsum	Gen_Len </br>
1	    No log	        1.920353	    0.193900	0.105400	0.168900	0.168800	19.000000  </br>
2	    2.296300	    1.860363	    0.195300	0.105100	0.171400	0.171400	19.000000 </br>
3	    1.927000	    1.834611	    0.195500	0.106800	0.171600	0.171500	19.000000 </br>
4	    1.849100	    1.826394	    0.195000	0.105200	0.171400	0.171300	19.000000 </br>