Mavors: Multi-granularity Video Representation for Multimodal Large
  Language Model
Authors: name: Yang Shi; name: Jiaheng Liu; name: Yushuo Guan; name: Zhenhua Wu; name: Yuanxing Zhang; name: Zihao Wang; name: Weihong Lin; name: Jingyun Hua; name: Zekun Wang; name: Xinlong Chen; name: Bohan Zeng; name: Wentao Zhang; name: Fuzheng Zhang; name: Wenjing Yang; name: Di Zhang
Published: 2025-04-14 10:14:44

Abstract: Long-context video understanding in multimodal large language models (MLLMs)
faces a critical challenge: balancing computational efficiency with the
retention of fine-grained spatio-temporal patterns. Existing approaches (e.g.,
sparse sampling, dense sampling with low resolution, and token compression)
suffer from significant information loss in temporal dynamics, spatial details,
or subtle interactions, particularly in videos with complex motion or varying
resolutions. To address this, we propose Mavors, a novel framework
that introduces Multi-granularity
video representation for holistic
long-video modeling. Specifically, Mavors directly encodes raw video content
into latent representations through two core components: 1) an Intra-chunk
Vision Encoder (IVE) that preserves high-resolution spatial features via 3D
convolutions and Vision Transformers, and 2) an Inter-chunk Feature Aggregator
(IFA) that establishes temporal coherence across chunks using transformer-based
dependency modeling with chunk-level rotary position encodings. Moreover, the
framework unifies image and video understanding by treating images as
single-frame videos via sub-image decomposition. Experiments across diverse
benchmarks demonstrate Mavors' superiority in maintaining both spatial fidelity
and temporal continuity, significantly outperforming existing methods in tasks
requiring fine-grained spatio-temporal reasoning.