jonleed/clinical-note-bart-lora

Clinical Note BART-LoRA

LoRA adapter (r=8, alpha=64) fine-tuned on top of facebook/bart-base to summarize doctor-patient dialogues into section-wise clinical notes (EHR-style), trained on the MTS-Dialog dataset (MEDIQA-Chat 2023).

Part of the Medical Dialogue Summary project (May 2025): github.com/jonleed/Medical-Dialogue-Summary

This is a mirror of the original mdlam/clinical-note-model upload with an expanded model card.

Model details

Base model	facebook/bart-base (~140M params)
Method	LoRA via PEFT
r / alpha / dropout	8 / 64 / 0.01
Target modules	q_proj, v_proj
Training data	MTS-Dialog training split (~1,700 dialogues)
Decode	beam=4, temperature=0.9, top_p=0.95, no_repeat_ngram_size=4

Results

ROUGE on the MTS-Dialog test set, against reference section summaries:

Source	ROUGE-1	ROUGE-2	ROUGE-L	ROUGE-Lsum
Reported at project completion (May 2025)	—	—	0.43	—
Re-evaluation Aug 2026 (100-example sample, seed=42)	0.290	0.114	0.252	0.259

The re-evaluation is reproducible via compute_metrics.py in the project repo.

Usage

from peft import PeftModel
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

base = AutoModelForSeq2SeqLM.from_pretrained("facebook/bart-base")
model = PeftModel.from_pretrained(base, "jonleed/clinical-note-bart-lora").merge_and_unload()
tokenizer = AutoTokenizer.from_pretrained("facebook/bart-base")

prompt = ("Summarize the following doctor-patient dialogue into a detailed "
          "History of Present Illness clinical note: <dialogue text>")
inputs = tokenizer(prompt, max_length=128, truncation=True, return_tensors="pt")
out = model.generate(**inputs, max_length=256, num_beams=4)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Intended scope

An educational project demonstrating parameter-efficient fine-tuning of compact seq2seq models for clinical summarization (pre-LLM-default era). Not a medical device; outputs are not clinically validated.