StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context
  Modeling
Authors: name: Meng Wei; name: Chenyang Wan; name: Xiqian Yu; name: Tai Wang; name: Yuqiang Yang; name: Xiaohan Mao; name: Chenming Zhu; name: Wenzhe Cai; name: Hanqing Wang; name: Yilun Chen; name: Xihui Liu; name: Jiangmiao Pang
Published: 2025-07-07 17:49:41

Abstract: Vision-and-Language Navigation (VLN) in real-world settings requires agents
to process continuous visual streams and generate actions with low latency
grounded in language instructions. While Video-based Large Language Models
(Video-LLMs) have driven recent progress, current VLN methods based on
Video-LLM often face trade-offs among fine-grained visual understanding,
long-term context modeling and computational efficiency. We introduce
StreamVLN, a streaming VLN framework that employs a hybrid slow-fast context
modeling strategy to support multi-modal reasoning over interleaved vision,
language and action inputs. The fast-streaming dialogue context facilitates
responsive action generation through a sliding-window of active dialogues,
while the slow-updating memory context compresses historical visual states
using a 3D-aware token pruning strategy. With this slow-fast design, StreamVLN
achieves coherent multi-turn dialogue through efficient KV cache reuse,
supporting long video streams with bounded context size and inference cost.
Experiments on VLN-CE benchmarks demonstrate state-of-the-art performance with
stable low latency, ensuring robustness and efficiency in real-world
deployment. The project page is:
https://streamvln.github.io/{https://streamvln.github.io/}.