RoboFactory: Exploring Embodied Agent Collaboration with Compositional
  Constraints
Authors: name: Yiran Qin, name: Li Kang, name: Xiufeng Song, name: Zhenfei Yin, name: Xiaohong Liu, name: Xihui Liu, name: Ruimao Zhang, name: Lei Bai

Designing effective embodied multi-agent systems is critical for solving
complex real-world tasks across domains. Due to the complexity of multi-agent
embodied systems, existing methods fail to automatically generate safe and
efficient training data for such systems. To this end, we propose the concept
of compositional constraints for embodied multi-agent systems, addressing the
challenges arising from collaboration among embodied agents. We design various
interfaces tailored to different types of constraints, enabling seamless
interaction with the physical world. Leveraging compositional constraints and
specifically designed interfaces, we develop an automated data collection
framework for embodied multi-agent systems and introduce the first benchmark
for embodied multi-agent manipulation, RoboFactory. Based on RoboFactory
benchmark, we adapt and evaluate the method of imitation learning and analyzed
its performance in different difficulty agent tasks. Furthermore, we explore
the architectures and training strategies for multi-agent imitation learning,
aiming to build safe and efficient embodied multi-agent systems.