GOAT: GO to Any Thing
Authors: name: Matthew Chang; name: Theophile Gervet; name: Mukul Khanna; name: Sriram Yenamandra; name: Dhruv Shah; name: So Yeon Min; name: Kavit Shah; name: Chris Paxton; name: Saurabh Gupta; name: Dhruv Batra; name: Roozbeh Mottaghi; name: Jitendra Malik; name: Devendra Singh Chaplot
Published: 2023-11-10 23:31:51

Abstract: In deployment scenarios such as homes and warehouses, mobile robots are
expected to autonomously navigate for extended periods, seamlessly executing
tasks articulated in terms that are intuitively understandable by human
operators. We present GO To Any Thing (GOAT), a universal navigation system
capable of tackling these requirements with three key features: a) Multimodal:
it can tackle goals specified via category labels, target images, and language
descriptions, b) Lifelong: it benefits from its past experience in the same
environment, and c) Platform Agnostic: it can be quickly deployed on robots
with different embodiments. GOAT is made possible through a modular system
design and a continually augmented instance-aware semantic memory that keeps
track of the appearance of objects from different viewpoints in addition to
category-level semantics. This enables GOAT to distinguish between different
instances of the same category to enable navigation to targets specified by
images and language descriptions. In experimental comparisons spanning over 90
hours in 9 different homes consisting of 675 goals selected across 200+
different object instances, we find GOAT achieves an overall success rate of
83%, surpassing previous methods and ablations by 32% (absolute improvement).
GOAT improves with experience in the environment, from a 60% success rate at
the first goal to a 90% success after exploration. In addition, we demonstrate
that GOAT can readily be applied to downstream tasks such as pick and place and
social navigation.