Title: Moving Out: Physically-grounded Human-AI Collaboration
Primary area: reinforcement learning
Keywords: Human-AI Collaboration, Physically-grounded

Abstract: The ability to adapt to physical actions and constraints in an environment is crucial for embodied agents (e.g., robots) to effectively collaborate with humans.
Such physically grounded human-AI collaboration must account for the increased complexity of the continuous state-action space and constrained dynamics caused by physical constraints.
In this paper, we introduce Moving Out, a new human-AI collaboration benchmark that resembles a wide range of collaboration modes affected by physical attributes and constraints, such as moving heavy items together and maintaining consistent actions to move a big item around a corner.
Using Moving Out, we designed two tasks and collected human-human interaction data to evaluate models' abilities to adapt to diverse human behaviors and unseen physical attributes.
To address the challenges in physical environments, we propose a novel method, BASS (Behavior Augmentation, Simulation, and Selection), to enhance the diversity of agents and their understanding of the outcome of actions.
Our experiments show that BASS outperforms state-of-the-art models in AI-AI and human-AI collaboration.

TL;DR: We introduce Moving Out and BASS for stronger human-AI collaboration under physical constraints.