kingzcx/Vision-OPD-Qwen3.5-9B-Option-Tail

Vision-OPD Qwen3.5-9B Option-Tail

This checkpoint is a Qwen3.5-9B Vision-OPD variant trained with option-space on-policy self-distillation on the Vision-OPD-6K dataset.

The student processes the original image while an EMA teacher processes the privileged region-focused image. Unlike the vocabulary Top-K objective in the original Vision-OPD setup, this variant distills the probability distribution over the legal answer-option tokens and a residual tail bucket containing all non-option probability mass. Training responses are restricted to one option token.

Training configuration

• Base model: Qwen/Qwen3.5-9B
• Retained samples after prompt-length filtering: 6,215
• Maximum prompt length: 5,120 tokens
• Maximum response length: 1 token
• Training epochs: 1
• Optimizer steps: 64
• Global training batch size: 96
• Rollouts per prompt: 8
• Learning rate: 2e-6
• Distillation loss: JSD (alpha=0.5)
• Teacher regularization: EMA
• Teacher update rate: 0.05
• Teacher image key: bbox_images

Training completed all 64 steps. No benchmark evaluation is included with this checkpoint.

Loading

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "kingzcx/Vision-OPD-Qwen3.5-9B-Option-Tail"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

For the intended multiple-choice behavior, prompt the model to answer directly with the option letter.

Citation

@article{yuan2026vision,
  title={Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation},
  author={Yuan, Qianhao and Lou, Jie and Yu, Xing and Lin, Hongyu and Sun, Le and Han, Xianpei and Lu, Yaojie},
  journal={arXiv preprint arXiv:2605.18740},
  year={2026}
}