CANVAS: Commonsense-Aware Navigation System for Intuitive Human-Robot
  Interaction
Authors: name: Suhwan Choi; name: Yongjun Cho; name: Minchan Kim; name: Jaeyoon Jung; name: Myunchul Joe; name: Yubeen Park; name: Minseo Kim; name: Sungwoong Kim; name: Sungjae Lee; name: Hwiseong Park; name: Jiwan Chung; name: Youngjae Yu
Published: 2024-10-02 06:34:45

Abstract: Real-life robot navigation involves more than just reaching a destination; it
requires optimizing movements while addressing scenario-specific goals. An
intuitive way for humans to express these goals is through abstract cues like
verbal commands or rough sketches. Such human guidance may lack details or be
noisy. Nonetheless, we expect robots to navigate as intended. For robots to
interpret and execute these abstract instructions in line with human
expectations, they must share a common understanding of basic navigation
concepts with humans. To this end, we introduce CANVAS, a novel framework that
combines visual and linguistic instructions for commonsense-aware navigation.
Its success is driven by imitation learning, enabling the robot to learn from
human navigation behavior. We present COMMAND, a comprehensive dataset with
human-annotated navigation results, spanning over 48 hours and 219 km, designed
to train commonsense-aware navigation systems in simulated environments. Our
experiments show that CANVAS outperforms the strong rule-based system ROS
NavStack across all environments, demonstrating superior performance with noisy
instructions. Notably, in the orchard environment, where ROS NavStack records a
0% total success rate, CANVAS achieves a total success rate of 67%. CANVAS also
closely aligns with human demonstrations and commonsense constraints, even in
unseen environments. Furthermore, real-world deployment of CANVAS showcases
impressive Sim2Real transfer with a total success rate of 69%, highlighting the
potential of learning from human demonstrations in simulated environments for
real-world applications.