Sasmita03/lora-samsum-summarizer

LoRA-tuned Flan-T5 for Dialogue Summarization

Flan-T5-base fine-tuned with LoRA on the SAMSum dataset (dialogue → summary).

Training details

• Base model: google/flan-t5-base
• Method: LoRA (r=8, alpha=32, dropout=0.1, targeting q and v attention projections)
• Dataset: SAMSum (800-example subset, 1 epoch)
• Trained on a free Google Colab T4 GPU

Results

Evaluated on 100 held-out validation examples, ROUGE F1 vs. the untrained base model:

Metric	Base model	Fine-tuned	Difference
ROUGE-1	0.4406	0.4749	+0.0343
ROUGE-2	0.2147	0.2237	+0.0090
ROUGE-L	0.3705	0.3887	+0.0182

Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForSeq2SeqLM.from_pretrained("google/flan-t5-base")
model = PeftModel.from_pretrained(base_model, "Sasmita03/lora-samsum-summarizer")
tokenizer = AutoTokenizer.from_pretrained("Sasmita03/lora-samsum-summarizer")