3d-ckpts-v3 — Thinking-in-Space GRPO checkpoints (Qwen2.5-VL-3B)

Qwen2.5-VL-3B-Instruct fine-tuned with GRPO (verl) to answer ScanQA-style 3D spatial questions by actively exploring ScanNet scenes: the policy calls a render_view_tool (camera teleport via coordinates), inspects the returned views, then answers. Reward: LLM-judge accuracy (Qwen3-30B, CORRECT/INCORRECT) + format + small per-call tool bonus + geometric grounding (final view must contain the answer object's 3D box, occlusion-checked raycast). Max 6 assistant turns. No raw ScanNet data is included here.

Checkpoints & performance

folder	recipe	train step	acc, official val (n=256)	acc, tiny val (n=64)
v9.1c-s120	v9.1: coords tool + geometric grounding (reproduction of the reference recipe)	120	0.195	0.266
v9.3-s32	v9.1 + any-frame grounding + coverage bonus (0.2) + closer-final-view prompt	32	—	0.234
v9.4b-s56	v9.4 with UNGATED distinct bonus, lighter repeat penalty (0.02)	56	—	0.188
v9.4-s16	v9.3 + anti-repeat: distinct-viewpoint-only tool bonus + near-duplicate render penalty	16	—	0.125

• Official val = eval_official_val_v2 (n=256, held-out ScanQA-style, pure judge accuracy, no bonuses). Historical bar to beat: 0.176 (best previous healthy run) — v9.1c-s120 clears it at 0.195.
• Tiny val (n=64) is the during-training monitor; noisy, use for relative comparison only.
• All runs: 1 sample/question judge scoring, temperature 0.7 rollouts, KL 0.005, train set train_filtered_v9 (16,777 QA / 562 ScanNet scenes).

Usage

Standard HF Qwen2.5-VL loading; each folder is a complete model.

from transformers import AutoModelForVision2Seq, AutoProcessor
m = AutoModelForVision2Seq.from_pretrained("Icey444/3d-ckpts-v3", subfolder="v9.1c-s120")
p = AutoProcessor.from_pretrained("Icey444/3d-ckpts-v3", subfolder="v9.1c-s120")

The interactive 3D behavior requires the render-tool harness from the training repo (multi-turn tool loop over ScanNet meshes); as plain VLMs these behave like Qwen2.5-VL with improved spatial question answering.