PanoVideoReasoner

PanoVideoReasoner is a panoramic video reasoning model introduced in Think in 360°: Benchmarking and Bootstrapping Panoramic Video Reasoning. It builds on Qwen3.5-9B and learns to connect observations across viewing directions and time.

The model is trained on PanoVideoCoT through Panoramic CoT SFT followed by Evidence-Privileged On-Policy Self-Distillation (EP-OPSD). PanoVideoCoT contains 13,465 question–answer–CoT pairs from 2,063 panoramic videos.

Quick start

Install a recent Transformers version with Qwen3.5 support, PyTorch, and the video-loading dependencies:

pip install -U "transformers>=5.15.0" accelerate torch torchvision pillow av

import torch
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

model_path = "./PanoVideoReasoner"
processor = AutoProcessor.from_pretrained(model_path)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_path,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "video", "path": "./panoramic_video.mp4"},
            {
                "type": "text",
                "text": (
                    "This is a 360-degree panoramic video in equirectangular "
                    "projection (ERP). Describe how the people move through "
                    "the scene, following them across viewing directions."
                ),
            },
        ],
    }
]
inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
    fps=1,
    max_frames=128,
).to(model.device)

with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=8192,
        do_sample=True,
        temperature=1.0,
        top_p=0.95,
        top_k=20,
    )
answer = outputs[:, inputs["input_ids"].shape[1]:]
print(processor.batch_decode(answer, skip_special_tokens=True)[0])

The example shows how to load the model and ask a video question. For the input protocol used in the paper, uniformly sample the complete ERP video at a target rate of 1 FPS, with at most 128 frames, and resize the long edge to 1,280 pixels. The training input budgets are 409,600 pixels per frame and 52,428,800 pixels per video. Keep the original ERP representation and its complete viewing sphere.