evalstate/Qwen2.5-VL-3B-TimeSpot-daylight-LoRA

Qwen2.5-VL-3B · TimeSpot daylight-phase LoRA

A LoRA adapter fine-tuned on the TimeSpot benchmark (paper) for daylight-phase classification (6 classes: Sunrise, Morning, Midday, Afternoon, Sunset, Night) — the paper's headline temporal weakness.

Base model

Qwen/Qwen2.5-VL-3B-Instruct (the paper's own SFT base).

Training recipe (paper Appendix H, single-task)

4-bit NFQ (bitsandbytes, double-quant, bf16 compute) · LoRA r=16/α=16, dropout 0 · targets q,k,v,o,gate,up,down_proj · paged_adamw_8bit · lr 2e-4 linear · effective batch 4 · wd 0.01 · seed 3407 · gradient checkpointing on · 200-image train subset · 1 epoch (2 planned; epoch 2 was cut by a host-RAM limit on a10g-small — see sft_results.json).

Epoch	Train loss	Subset eval (100 img)
1 (best)	0.163	52.0%

Results (150-image eval subset, same narrow prompt, deterministic scorer)

Run	Daylight-phase acc
Narrow zero-shot (this base, no adapter)	21.3%
Narrow fine-tuned (this adapter)	56.0%

Fine-tune delta: +34.7 points. The adapter more than doubles daylight-phase accuracy, though the model collapses to the majority class (Afternoon) rather than learning physically grounded daylight reasoning — see the report's failure analysis. Full metrics, predictions, and the deterministic scorer are in evalstate/timespot-daylight-eval.

How to load

import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
base = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2.5-VL-3B-Instruct", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "evalstate/Qwen2.5-VL-3B-TimeSpot-daylight-LoRA")
processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct")

Reproducibility

Training script: code/train_sft.py. Eval + scoring scripts are in the same dataset repo. Part of a reproducible TimeSpot evaluation (see the accompanying report for protocol-sensitivity analysis).