TEMPURA-InternVL3-8B

TEMPURA checkpoint built on OpenGVLab/InternVL3-8B-hf.

TEMPURA (Temporal Event Masked Prediction and Understanding for Reasoning in Action, COLM 2026) teaches video-language models to reason about causal event structure and to describe untrimmed videos with fine-grained, timestamp-aligned events. This model performs

• dense video captioning – partition a video into consecutive, non-overlapping events with timestamps and detailed descriptions;

• masked event prediction – reason step by step about what happens in a hidden segment of the video;

• video temporal grounding / highlight detection – locate the moments that match a natural-language query.

• Project page: https://andy-cheng.github.io/TEMPURA/

• Paper: https://arxiv.org/abs/2505.01583

• Code, inference and benchmark evaluation: https://github.com/Andy-Cheng/TEMPURA

• Dataset: andaba/TEMPURA-VER

Input format

The video is given as a sequence of frames sampled at 0.5 fps, each with its timestamp (seconds, two decimals) drawn on the top-left corner. The helper functions in the GitHub repository take care of this.

Usage

git clone https://github.com/Andy-Cheng/TEMPURA.git && cd TEMPURA
bash scripts/install/install.sh && source .venv/bin/activate
python -m src.inference.dense_video_captioning_demo --model_path andaba/TEMPURA-InternVL3-8B --video test_videos_demo/hotdog.mp4 --fps 0.5

from src.inference.model_utils import load_model, build_messages, generate
from src.inference.video_utils import load_video_for_model
from src.inference import prompts

processor, model, family = load_model("andaba/TEMPURA-InternVL3-8B")
frames, timestamps = load_video_for_model("video.mp4", sample_fps=0.5, add_timestamp=True)
messages = build_messages(frames, prompts.DVC, family, min_pixels=336 * 336, max_pixels=336 * 336)
print(generate(processor, model, family, messages, max_new_tokens=2048))

Benchmark evaluation (Charades-STA temporal grounding, QVHighlights highlight detection):

bash scripts/eval/eval_charades.sh andaba/TEMPURA-InternVL3-8B results/charades/TEMPURA-InternVL3-8B internvl 0
bash scripts/eval/eval_qvhighlights.sh andaba/TEMPURA-InternVL3-8B results/qvhighlights/TEMPURA-InternVL3-8B internvl 0

Results obtained with this pipeline are listed in the repository README.

License

These weights are a fine-tune of InternVL3-8B-hf, which is released under the Qwen License because it builds on Qwen2.5 (the InternVL code is MIT-licensed); the fine-tuned weights follow the same license (a copy is included as LICENSE).

Citation

@inproceedings{
cheng2026tempura,
title={{TEMPURA}: Temporal Event Masked Prediction and Understanding for Reasoning in Action},
author={Cheng, Jen-Hao and Peng, Yi-Hao and Zhou, Huapeng and Wang, Vivian and Wang, Huayu and Huang, Hsiang-Wei and Chai, Wenhao and Liu, Hou-I and Chen, Kuang-Ming and Yang, Cheng-Yen and Chen, Yi-Ling and Vineet, Vibhav and Cai, Qin and Hwang, Jenq-Neng},
booktitle={Third Conference on Language Modeling},
year={2026}
}