UniMIC: Token-Based Multimodal Interactive Coding for Human-AI
  Collaboration
Authors: name: Qi Mao; name: Tinghan Yang; name: Jiahao Li; name: Bin Li; name: Libiao Jin; name: Yan Lu
Published: 2025-09-26 16:46:12

Abstract: The rapid progress of Large Multimodal Models (LMMs) and cloud-based AI
agents is transforming human-AI collaboration into bidirectional, multimodal
interaction. However, existing codecs remain optimized for unimodal, one-way
communication, resulting in repeated degradation under conventional
compress-transmit-reconstruct pipelines. To address this limitation, we propose
UniMIC, a Unified token-based Multimodal Interactive Coding framework that
bridges edge devices and cloud AI agents. Instead of transmitting raw pixels or
plain text, UniMIC employs compact tokenized representations as the
communication medium, enabling efficient low-bitrate transmission while
maintaining compatibility with LMMs. To further enhance compression,
lightweight Transformer-based entropy models with scenario-specific
designs-generic, masked, and text-conditioned-effectively minimize inter-token
redundancy. Extensive experiments on text-to-image generation, text-guided
inpainting, outpainting, and visual question answering show that UniMIC
achieves substantial bitrate savings and remains robust even at ultra-low
bitrates (<0.05bpp), without compromising downstream task performance. These
results establish UniMIC as a practical and forward-looking paradigm for
next-generation multimodal interactive communication.