Decentralized Aerial Manipulation of a Cable-Suspended Load using
  Multi-Agent Reinforcement Learning
Authors: name: Jack Zeng, name: Andreu Matoses Gimenez, name: Eugene Vinitsky, name: Javier Alonso-Mora, name: Sihao Sun

This paper presents the first decentralized method to enable real-world 6-DoF
manipulation of a cable-suspended load using a team of Micro-Aerial Vehicles
(MAVs). Our method leverages multi-agent reinforcement learning (MARL) to train
an outer-loop control policy for each MAV. Unlike state-of-the-art controllers
that utilize a centralized scheme, our policy does not require global states,
inter-MAV communications, nor neighboring MAV information. Instead, agents
communicate implicitly through load pose observations alone, which enables high
scalability and flexibility. It also significantly reduces computing costs
during inference time, enabling onboard deployment of the policy. In addition,
we introduce a new action space design for the MAVs using linear acceleration
and body rates. This choice, combined with a robust low-level controller,
enables reliable sim-to-real transfer despite significant uncertainties caused
by cable tension during dynamic 3D motion. We validate our method in various
real-world experiments, including full-pose control under load model
uncertainties, showing setpoint tracking performance comparable to the
state-of-the-art centralized method. We also demonstrate cooperation amongst
agents with heterogeneous control policies, and robustness to the complete
in-flight loss of one MAV. Videos of experiments:
https://autonomousrobots.nl/paper_websites/aerial-manipulation-marl