Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal
  Learning
Authors: name: Sheng Chen, name: Peiyu He, name: Jiaxin Hu, name: Ziyang Liu, name: Yansheng Wang, name: Tao Xu, name: Chi Zhang, name: Chongchong Zhang, name: Chao An, name: Shiyu Cai, name: Duo Cao, name: Kangping Chen, name: Shuai Chu, name: Tianwei Chu, name: Mingdi Dan, name: Min Du, name: Weiwei Fang, name: Pengyou Fu, name: Junkai Hu, name: Xiaowei Jiang, name: Zhaodi Jiang, name: Fuxuan Li, name: Jun Li, name: Minghui Li, name: Mingyao Li, name: Yanchang Li, name: Zhibin Li, name: Guangming Liu, name: Kairui Liu, name: Lihao Liu, name: Weizhi Liu, name: Xiaoshun Liu, name: Yufei Liu, name: Yunfei Liu, name: Qiang Lu, name: Yuanfei Luo, name: Xiang Lv, name: Hongying Ma, name: Sai Ma, name: Lingxian Mi, name: Sha Sa, name: Hongxiang Shu, name: Lei Tian, name: Chengzhi Wang, name: Jiayu Wang, name: Kaijie Wang, name: Qingyi Wang, name: Renwen Wang, name: Tao Wang, name: Wei Wang, name: Xirui Wang, name: Chao Wei, name: Xuguang Wei, name: Zijun Xia, name: Zhaohao Xiao, name: Tingshuai Yan, name: Liyan Yang, name: Yifan Yang, name: Zhikai Yang, name: Zhong Yin, name: Li Yuan, name: Liuchun Yuan, name: Chi Zhang, name: Jinyang Zhang, name: Junhui Zhang, name: Linge Zhang, name: Zhenyi Zhang, name: Zheyu Zhang, name: Dongjie Zhu, name: Hang Li, name: Yangang Zhang

Modern robot navigation systems encounter difficulties in diverse and complex
indoor environments. Traditional approaches rely on multiple modules with small
models or rule-based systems and thus lack adaptability to new environments. To
address this, we developed Astra, a comprehensive dual-model architecture,
Astra-Global and Astra-Local, for mobile robot navigation. Astra-Global, a
multimodal LLM, processes vision and language inputs to perform self and goal
localization using a hybrid topological-semantic graph as the global map, and
outperforms traditional visual place recognition methods. Astra-Local, a
multitask network, handles local path planning and odometry estimation. Its 4D
spatial-temporal encoder, trained through self-supervised learning, generates
robust 4D features for downstream tasks. The planning head utilizes flow
matching and a novel masked ESDF loss to minimize collision risks for
generating local trajectories, and the odometry head integrates multi-sensor
inputs via a transformer encoder to predict the relative pose of the robot.
Deployed on real in-house mobile robots, Astra achieves high end-to-end mission
success rate across diverse indoor environments.