XinNUS/Generalized-CycleGRPO-Temporal-4B

Generalized CycleGRPO — Temporal 4B

Official temporal model for Generalized CycleGRPO for Fine-Grained Multimodal Understanding and Grounding. This Qwen3-VL-based model couples interval-conditioned captioning with caption-conditioned temporal localization through a region–text–region reconstruction cycle.

Project page · Code · Video implementation

Intended use

• Describe a specified interval while viewing the complete source video.
• Localize an event described in natural language.
• Research on distinctive video descriptions and temporal grounding.

This is the Qwen-initialized temporal model used for the Qwen comparison on TimeLens and the main DOVE captioning result. It is distinct from the OMTG-initialized temporal model and from the ECCV image-mask model.

Reported results

Benchmark	Metrics	Result (%)
Charades-STA	R1@0.3 / R1@0.5 / R1@0.7 / tIoU	74.8 / 58.6 / 30.7 / 51.9
DOVE	TEC / Temp-Cov / Clip-Complete	33.3 / 33.4 / 14.6

Numbers follow the paper's evaluation protocols. Minimal inference examples are not substitutes for the official benchmark preprocessing and scoring.

Quickstart

Use Transformers with Qwen3-VL support and install qwen-vl-utils plus a working video decoder. The weights are stored in bfloat16.

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from qwen_vl_utils import process_vision_info

model_id = "XinNUS/Generalized-CycleGRPO-Temporal-4B"
model = AutoModelForImageTextToText.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
).eval()
processor = AutoProcessor.from_pretrained(model_id)

messages = [{"role": "user", "content": [
    {"type": "video", "video": "example.mp4", "fps": 2.0},
    {"type": "text", "text": (
        "Please provide a detailed description of what happens in this video "
        "during 12.0 - 18.0 seconds. Describe only the specified occurrence."
    )},
]}]
text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
images, videos, video_kwargs = process_vision_info(
    messages, return_video_kwargs=True
)
inputs = processor(
    text=[text], images=images, videos=videos,
    padding=True, return_tensors="pt", **video_kwargs
).to(model.device)
with torch.inference_mode():
    output = model.generate(
        **inputs, max_new_tokens=256, do_sample=False, use_cache=True
    )
print(processor.batch_decode(
    output[:, inputs.input_ids.shape[1]:], skip_special_tokens=True
)[0])

Replace the video and interval with your own input. Long videos may require a suitable frame/pixel budget to fit GPU memory; the complete source video must remain the input for DOVE-style interval-conditioned captioning.

Limitations and attribution

Reconstructability does not guarantee factual completeness, and descriptions can omit details or confuse repeated events. Do not use generated descriptions as verified evidence without human review. This model inherits the Qwen3-VL architecture and tokenizer. Temporal data and task design draw on OMTG. See NOTICE and LICENSE for attribution and license terms.

Authors

Xin Zhang, Haochen Wang, Beibei Lin, Zhaoxiang Zhang, Robby T. Tan, Jason Li, and Dacheng Tao.