OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming
  Video Contexts
Authors: name: Yuxuan Wang; name: Yueqian Wang; name: Bo Chen; name: Tong Wu; name: Dongyan Zhao; name: Zilong Zheng
Published: 2025-03-29 02:46:58

Abstract: The rapid advancement of multi-modal language models (MLLMs) like GPT-4o has
propelled the development of Omni language models, designed to process and
proactively respond to continuous streams of multi-modal data. Despite their
potential, evaluating their real-world interactive capabilities in streaming
video contexts remains a formidable challenge. In this work, we introduce
OmniMMI, a comprehensive multi-modal interaction benchmark tailored for
OmniLLMs in streaming video contexts. OmniMMI encompasses over 1,121 videos and
2,290 questions, addressing two critical yet underexplored challenges in
existing video benchmarks: streaming video understanding and proactive
reasoning, across six distinct subtasks. Moreover, we propose a novel
framework, Multi-modal Multiplexing Modeling (M4), designed to enable an
inference-efficient streaming model that can see, listen while generating.