Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal
  Learning
Authors: name: Sheng Chen; name: Peiyu He; name: Jiaxin Hu; name: Ziyang Liu; name: Yansheng Wang; name: Tao Xu; name: Chi Zhang; name: Chongchong Zhang; name: Chao An; name: Shiyu Cai; name: Duo Cao; name: Kangping Chen; name: Shuai Chu; name: Tianwei Chu; name: Mingdi Dan; name: Min Du; name: Weiwei Fang; name: Pengyou Fu; name: Junkai Hu; name: Xiaowei Jiang; name: Zhaodi Jiang; name: Fuxuan Li; name: Jun Li; name: Minghui Li; name: Mingyao Li; name: Yanchang Li; name: Zhibin Li; name: Guangming Liu; name: Kairui Liu; name: Lihao Liu; name: Weizhi Liu; name: Xiaoshun Liu; name: Yufei Liu; name: Yunfei Liu; name: Qiang Lu; name: Yuanfei Luo; name: Xiang Lv; name: Hongying Ma; name: Sai Ma; name: Lingxian Mi; name: Sha Sa; name: Hongxiang Shu; name: Lei Tian; name: Chengzhi Wang; name: Jiayu Wang; name: Kaijie Wang; name: Qingyi Wang; name: Renwen Wang; name: Tao Wang; name: Wei Wang; name: Xirui Wang; name: Chao Wei; name: Xuguang Wei; name: Zijun Xia; name: Zhaohao Xiao; name: Tingshuai Yan; name: Liyan Yang; name: Yifan Yang; name: Zhikai Yang; name: Zhong Yin; name: Li Yuan; name: Liuchun Yuan; name: Chi Zhang; name: Jinyang Zhang; name: Junhui Zhang; name: Linge Zhang; name: Zhenyi Zhang; name: Zheyu Zhang; name: Dongjie Zhu; name: Hang Li; name: Yangang Zhang
Published: 2025-06-06 16:08:47

Abstract: Modern robot navigation systems encounter difficulties in diverse and complex
indoor environments. Traditional approaches rely on multiple modules with small
models or rule-based systems and thus lack adaptability to new environments. To
address this, we developed Astra, a comprehensive dual-model architecture,
Astra-Global and Astra-Local, for mobile robot navigation. Astra-Global, a
multimodal LLM, processes vision and language inputs to perform self and goal
localization using a hybrid topological-semantic graph as the global map, and
outperforms traditional visual place recognition methods. Astra-Local, a
multitask network, handles local path planning and odometry estimation. Its 4D
spatial-temporal encoder, trained through self-supervised learning, generates
robust 4D features for downstream tasks. The planning head utilizes flow
matching and a novel masked ESDF loss to minimize collision risks for
generating local trajectories, and the odometry head integrates multi-sensor
inputs via a transformer encoder to predict the relative pose of the robot.
Deployed on real in-house mobile robots, Astra achieves high end-to-end mission
success rate across diverse indoor environments.