skblv/gemma-3-27b-it-lora-pitvqa-phase-step

Gemma 3 27B-it LoRA — PitVQA phase and step

LoRA adapter plus a 17-way linear classification head on google/gemma-3-27b-it, fine-tuned for joint surgical-phase and surgical-step recognition on PitVQA frames.

This is the “Gemma 3 27B fine-tuned” row on the Clinical context / VQA tab of the SDSC × Chicago Booth surgical video understanding leaderboard.

Prompt example

This closed-set example mirrors the leaderboard format, not a text-input API for this checkpoint.

[surgical frame]

What is the current surgical phase and surgical step in this endoscopic pituitary frame?
Choose one phase and one step.

Phase (choose one)
- closure
- nasal sphenoid
- sellar

Step (choose one)
- anterior sphenoidotomy
- debris clearance
- dural sealant
- durotomy
- fat graft placement
- gasket seal construct
- haemostasis
- nasal corridor creation
- nasal packing
- sellotomy
- septum displacement
- sphenoid sinus clearance
- synthetic graft placement
- tumour excision

Files

• adapter_model.safetensors / adapter_config.json — LoRA (r=128, alpha=256) on q/k/v/o_proj and out_proj
• classifier.pt — linear head over the 5376-d hidden state (17 phase/step labels)
• model_config.json — class vocabulary and training hyperparameters

You need the adapter and the classification head; this is not a generative JSON adapter.

Training

• 10 epochs, lr 5e-6, seed 42
• full PitVQA train split training frames / 24,767 validation frames

Evaluation

Full PitVQA validation split (95% bootstrap CI):

Metric	Value
Exact match	64.6% (64.0–65.2)
Micro-averaged F1	74.6% (74.2–75.1)

References

• Skobelev, K., Fithian, E., Baranovski, Y., et al. A Comparative Study in Surgical AI: Potential and Limitations of Data, Compute, and Scaling. arXiv:2603.27341, 2026.
• Dataset: He, R., Xu, M., Das, A., et al. PitVQA: Image-grounded Text Embedding LLM for Visual Question Answering in Pituitary Surgery. MICCAI 2024.

Limitations

Research baseline only. Not a medical device.