When and What: Diffusion-Grounded VideoLLM with Entity Aware
  Segmentation for Long Video Understanding
Authors: name: Pengcheng Fang; name: Yuxia Chen; name: Rui Guo
Published: 2025-08-21 15:12:14

Abstract: Understanding videos requires more than answering open ended questions, it
demands the ability to pinpoint when events occur and how entities interact
across time. While recent Video LLMs have achieved remarkable progress in
holistic reasoning, they remain coarse in temporal perception: timestamps are
encoded only implicitly, frame level features are weak in capturing continuity,
and language vision alignment often drifts from the entities of interest. In
this paper, we present Grounded VideoDiT, a Video LLM designed to overcome
these limitations by introducing three key innovations. First, a Diffusion
Temporal Latent (DTL) encoder enhances boundary sensitivity and maintains
temporal consistency. Second, object grounded representations explicitly bind
query entities to localized visual evidence, strengthening alignment. Third, a
mixed token scheme with discrete temporal tokens provides explicit timestamp
modeling, enabling fine grained temporal reasoning. Together, these designs
equip Grounded VideoDiT with robust grounding capabilities, as validated by
state of the art results on Charades STA, NExT GQA, and multiple VideoQA
benchmarks.