orena-frame-weights

LoRA adapters for the FRAME track of the ORena SAVE FOCUS Challenge (MICCAI 2026).

Setting (all checkpoints)

• Base models: Qwen/Qwen3.5-9B (9B) and Qwen/Qwen3.8-27B (27B), served as LoRA (merge_and_unload into base for vLLM inference).
• LoRA config: rank=64, alpha=128, dropout=0.05, target_modules=all-linear (LLM + vision tower + aligner), freeze_llm/vit/aligner=false.
  • 9B: 205.06M trainable (2.13%) · 27B: 499M trainable (1.79%).
• Training: ms-swift swift sft, --num_train_epochs 15, --add_non_thinking_prefix true (Qwen3.5 non-thinking), max_length 4096, max_pixels 1048576 (native ~921K, no downscale), learning_rate 1e-4, cosine, optim adamw_torch_fused, bf16.
  • 9B: per-device batch 8 × 8 GPUs = eff 64 → 3225 steps.
  • 27B: per-device batch 16 × 8 GPUs = eff 128 → 1620 steps.
• Prompt: system ("You are a surgical assistant … Be precise and concise.") + <image> + question, non-thinking prefix <think>\n\n</think>\n\n appended at inference. (fo_definitions NOT injected in these runs.)

Eval (official focus.evaluation.Evaluator, vLLM inference + Qwen3.5-4B judge)

Micro accuracy (raw correct / total), object_recognition / aggregation groups.

ckpt	overall	micro rec	micro agg	number
9b-400	61.46	68.40	53.07	43.65
9b-600	57.32	61.33	52.47	44.56
9b-800	61.67	68.49	53.43	43.98
9b-1000	64.26	71.82	55.12	46.56
9b-1200	63.80	71.38	54.63	46.18
9b-1400	66.98	75.36	56.86	47.56
9b-1600	66.65	74.25	57.46	47.99
9b-1800	66.92	75.07	57.07	47.90
9b-2000	68.49	76.62	58.66	50.00
9b-2200	68.55	76.35	59.12	50.24
9b-2400	68.95	76.94	59.29	50.57
9b-2600	69.40	77.05	60.14	51.48
9b-2800	69.20	76.85	59.96	51.29
9b-3000	69.36	77.00	60.14	51.53
9b-3200	69.32	77.05	59.96	51.19
9b-3225	69.40	77.11	60.07	51.34
9c-1000	66.04	73.57	56.93	48.04
9c-1100	66.61	74.28	57.35	49.81
9c-1200	67.56	74.51	59.15	50.43
9c-1300	67.97	75.21	59.22	50.14
9c-1400	68.13	75.39	59.36	50.91
9c-1500	68.49	75.77	59.68	51.00
9c-1600	68.34	75.68	59.47	50.67
9c-1620	68.41	75.65	59.65	50.81

Best: 9b-3225 (overall 69.40% / rec 77.11% / agg 60.07%). 9B > 27B by 1pp. Weak spots: number (51%, systematic under-count) and aggregation (~60%).

Usage

from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor

base = "Qwen/Qwen3.5-9B"  # or Qwen/Qwen3.8-27B
model = AutoModelForImageTextToText.from_pretrained(base, torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(model, "<repo>/9b-3225")
proc = AutoProcessor.from_pretrained(base)

For vLLM: merge_and_unload() the LoRA into the base first, then serve the merged weights (vLLM's PunicaWrapper does not support vision-tower LoRA).