X-Streamer: Unified Human World Modeling with Audiovisual Interaction
Authors: name: You Xie, name: Tianpei Gu, name: Zenan Li, name: Chenxu Zhang, name: Guoxian Song, name: Xiaochen Zhao, name: Chao Liang, name: Jianwen Jiang, name: Hongyi Xu, name: Linjie Luo

We introduce X-Streamer, an end-to-end multimodal human world modeling
framework for building digital human agents capable of infinite interactions
across text, speech, and video within a single unified architecture. Starting
from a single portrait, X-Streamer enables real-time, open-ended video calls
driven by streaming multimodal inputs. At its core is a Thinker-Actor
dual-transformer architecture that unifies multimodal understanding and
generation, turning a static portrait into persistent and intelligent
audiovisual interactions. The Thinker module perceives and reasons over
streaming user inputs, while its hidden states are translated by the Actor into
synchronized multimodal streams in real time. Concretely, the Thinker leverages
a pretrained large language-speech model, while the Actor employs a chunk-wise
autoregressive diffusion model that cross-attends to the Thinker's hidden
states to produce time-aligned multimodal responses with interleaved discrete
text and audio tokens and continuous video latents. To ensure long-horizon
stability, we design inter- and intra-chunk attentions with time-aligned
multimodal positional embeddings for fine-grained cross-modality alignment and
context retention, further reinforced by chunk-wise diffusion forcing and
global identity referencing. X-Streamer runs in real time on two A100 GPUs,
sustaining hours-long consistent video chat experiences from arbitrary
portraits and paving the way toward unified world modeling of interactive
digital humans.