Learning Decentralized Multi-Biped Control for Payload Transport
Authors: Bikram Pandit
Ashutosh Gupta
Mohitvishnu S. Gadde
Addison Johnson
Aayam Kumar Shrestha
Helei Duan
Jeremy Dao
Alan Fern
Venue: CoRL 2024 (2024)
Keywords: Multi-robot Transport
Bipedal locomotion
Reinforcement Learning
TL;DR: A straightforward yet highly effective decentralized approach for collaborative payload transport using multiple bipedal robots is proposed, demonstrating strong generalization in simulations and successful sim-to-real transfer.

Abstract: Payload transport over flat terrain via multi-wheel robot carriers is well-understood, highly effective, and configurable. In this paper, our goal is to provide similar effectiveness and configurability for transport over rough terrain that is more suitable for legs rather than wheels. For this purpose, we consider multi-biped robot carriers, where wheels are replaced by multiple bipedal robots attached to the carrier. Our main contribution is to design a decentralized controller for such systems that can be effectively applied to varying numbers and configurations of rigidly attached bipedal robots without retraining. We present a reinforcement learning approach for training the controller in simulation that supports transfer to the real world. Our experiments in simulation provide quantitative metrics showing the effectiveness of the approach over a wide variety of simulated transport scenarios. In addition, we demonstrate the controller in the real-world for systems composed of two and three Cassie robots. To our knowledge, this is the first example of a scalable multi-biped payload transport system.

Learning Decentralized Multi-Biped Control for Payload Transport
Bikram Pandit, Ashutosh Gupta, Mohitvishnu S. Gadde,
Addison Johnson,
Aayam Kumar Shrestha, Helei Duan, Jeremy Dao, Alan Fern
Collaborative Robotics and Intelligent Systems (CoRIS) Institute, Oregon State University (panditb, guptaash, gaddem, johnsadd, shrestaa, duanh, daoje, afern (oreen)oregonsttttt wwwpeggnnstttteeee
Project webpage: deembe.github.io
Abstract: Payload transport flat terrain via multi-wheel robot carriers is well-understood, highly effective, and configurable. In this paper, our goal is to provide similar effectiveness and configurability for transport over rough terrain that is more suitable for legs rather than wheels. For this purpose, we consider multi-biped robot carriers, where wheels are replaced by multiple bipedal robots attached to the carrier. Our main contribution is design a decentralized con- troller for such systems that can be effectively applied varying numbers and configurations of rigidly attached bipedal robots without retraining. We present reinforcement learning approach for training the controller in simulation that sup- ports transfer to the real world. Our experiments in simulation provide quantitative metrics showing the effectiveness of the approach over a wide variety of simulated transport scenarios. In addition, we demonstrate the controller in the real-world for systems composed of two and three Cassie robots. To our knowledge, this is the first example of scalable multi-biped payload transport system.
Keywords: Multi-robot Transport, Bipedal locomotion, Reinforcement Learning
1 Introduction
Multi-wheel carriers are the most common way to transport payloads across well-structured terrain due their ease of control, effectiveness, and configurable wheel arrangements [1]. Further, re- searchers have also considered wheeled transport that can be reconfigured on the fly via multiple co- operating wheeled robots [2]. While these transport systems excel in structured environments, they struggle in unstructured and challenging terrain, which limits their general applicabilityy This work aims to overcome such limitations while retaining the control and reconfigurability of multi-wheel systems. To achieve this, we propose the multiibippd transport problem, where wheels or wheeled robots are replaced with bipedal robots that can be arbitrarily attached carriers. This setup allows for transport over various terrains, potentially extending to more challenging environments [3, 4, 5].
Multi-biped transport presents significant challenges. Legged robots are generally unstable and highly dynamic, making them much harder to control compared to wheeled robots. Maintaining stability while transporting payloads and coordinating multiple legged-robots adds further complex- ity to the control problem [6]. However, the use multiple bipeds offers distinct advantages over alternatives such as large quadrupeds, providing greater flexibility in payload transport. This ap- proach allows the number and arrangement of robots to be tailored based the weight and size of the payload. Such versatility opens up potential applications in various areas including construction, moving systems, warehouses, agriculture, and delivery systems.
Prior works on multi-legged transport systems have primarily focused on model-based approaches that rely on accurate modeling of robot interaction and explicit inter-robot communication, which limit their reconfigurabilitt and practicality [7, 8]. While learning- based approaches have emerged as promising alternatives [9, 10. 11, 12, 13, 14, 15, 16, 17, 18] for various control problems, existing methods multi-robot system often assume centralized control with strong assumptions on system observability, leading to limited scalability, robustness, and vulnerability to point of failure [19].
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

A
Carrier control point (Og)
B
Training in simulation
deCMBC
cummunde
π
C
Generalization to other carrier/payload
Robot state
setpoints
D
A) Illustrates the Decentralized Multi Multi-Bipedal Controller (decMBC), which is a single controller deployable to any number of robots for carrier transport without requiring inter-robor communication. B) Shows the distribution of multi-rohot configurations used in the training curriculum for decMBC in simulation. C) Highlights unseen carrier and payload scenarios which the decMBC policy was evaluated for in the simula- tion. D) Showcases sim-to-real deployment where each rohot running copy decMBC policy on hardware.
Figure 1:
To overcome these challenges, propose architecture and training approach for decentralized Multi-Bipeds Controller (decMBC). This controller supports high-level motion control of carrier that is attached to any arbitrary configuration of bipedal robots. Importantly, to maintain maximum reconfigurability, the decMBC is decentralized in the sense that each biped can be controlled using only information about its local state and its relative position with respect to a desired reference point on the carrier. addition, the decMBC can be applied, without retraining, to any number of bipeds that are attached carrier in arbitrary stable configurations. The proposed decMBC is trained using decentralized reinforcement learning (RL) over distribution robot-carrier configurations carefully selected provide generalization to arbitrary biped configurations and facilitate sim-to- real transfer. The approach is easy implement and can be used with wide range of existing single-agent control architectures and RL algorithms, making it highly adaptable and accessible.
We validate the effectiveness and flexibility our approach through extensive experiments across varying numbers of robot and carrier configurations. Our quantitative results simulation highlight the system's ability maintain stability, coordinate effectively, and adapt different payload trans- port tasks. Furthermore, and three Cassie robots. To the best of our knowledge, this is the first multi-leg transport system to demonstrate this level of reconfigurability in simulation and the real world.
demonstrate sim-to-real transfer to multi-biped systems involving two
Related Work
Multi-robot transport has attracted significant research attention due to its potential overcome the limitations of single-robot systems, especially in handling large, heavy payloads and navigating diverse real-world environments. Several works have focused on wheeled robots for collaborative transport which developed a composite connector for non-holonomic mobile robots, but their work primarily focused on well-structured environments [2]. Moreover, Eoh and Park [20] propose a cooperative object transport controller using deep reinforcement learning which uses simple non- holonomic wheeled robots learn the collaborative pushing of an object to its goal position. While their approach shows promise, it assumes that the robots can sense the positions of the object and the goal with respect to each robot. This makes scaling the approach for arbitrary robot-carrier configurations difficult.

---

Similarly, other works have considered predictive control and decentralized approaches. Fawcett et al. [21] proposed a predictive control approach for robust locomotion of three holonomic con- straint quadrupeds, but it did not allow for formation changes or sealing to more agents. Zhang et al. [22] introduced a decentralized control approach using DQN controllers, but they strongly assumed that each robot could perfectly sense the state of the payload and every robot. Lin al. [23] develop an end-to-end decentralized deep reinforcement learning policy to learn multi-robot control directly from LiDAR measurements. They employ centralized learning and decentralized execution to de- ploy on holonomic UGVs. However, their approach relies on the assumption that the team-level action should be a concatenation of all individual actions, which limits its flexibility.
On the other hand, researchers have also explored multi-legged transport systems. Kim al. [7, 8] proposed layered control strategies and control barrier functions for holonomically connected quadrupeds, but their approaches were not generalized for any number of robots and relied on strong assumptions. Ji et al. [19] demonstrated reinforcement learning strategy for four quadrupedal robots executing a collaborative task, but their approach was limited to a fixed configuration and number of agents, and the results are shown only in simulation. Other notable works include Horyna et al. [24], who developed a collaborative transport control for two multi-copters using a master- slave configuration, and Pei et al. [25], who demonstrated multi-robot system for collaboratively transporting objects in a net spanning across all agents. These approaches faced challenges in scaling with the number of agents and had restrictions on the size of the net and topology of team formation.
In this work, we focus on the most primitive legged base i.e., bipedal robots. For bipedal robots, Dao et al. [26] investigate the potential of reinforcement learning and sim-to-real transfer for bipedal lo- comotion under dynamic loads, relying solely on proprioceptive feedback. While they demonstrated effectiveness across a wide range of external forces, their approach faced limitations. Specifically, their method was designed only for single robot and was constrained by weight, making scalability to multiple robots handling heavier loads challenging.
Building upon these prior works, our proposed decentralized Multi-Biped Controller (decMBC) aims to overcome the limitations of existing approaches by enabling decentralized control and gener- alization arbitrary carrier configurations of legged robots. Unlike the works that focus on wheeled robots, our decMBC has the potential to handle the complexity and dynamics of legged robots in challenging terrain. Furthermore, our decentralized architecture enhances the scalability and robust- ness of the system, making it more practical for real-world deployment.
3 Problem Formulation
We consider multi- biped transport problems where a set of N identical bipedal robots are rigidly attached to the base of a payload carrier. We model the carrier base B C R² as rigidly connected planar surface with a finite area. Each carrier base is assumed to contain the origin, which is referred to as the carrier control point Oc [see Figure 1 (A)]. An N robot attachment configuration for a base a set of points P = {pi.... ......... ......... }. such that pr E B, which indicates where each robot is attached. In this work, we assume ball joint attachments where each robot has free movements in all three rotational directions relative the attachment but constrains movements in linear directions. However, our approach is easily adapted to other types of joints and non-planar rigid platform bases.
Our goal is to control the height and linear and angular velocity of the carrier control point via the combined motor actions of the robots. To support flexible applications, we consider a decentral- ized controller architecture that avoids inter-robot communication and flexible varying num- bers of robots N and configurations P (p1, ,Pr....,,,,,, The decentralized Multi-Biped Controller (decMBC) is defined by single-biped controller π. At each time-step, this controller is indepen- dently applied to each robot E {I....,,, N to compute robot actions of = T(] ct), where of is the local observation and c is the carrier command.
In our work, the command O E 1 specifies the target linear xy velocity, angular yaw velocity, and height of the control point. The local observation of contains: 1) the local proprioceptive state sf of the robot, including the joints/motor positions and velocities, base orientation, and angular velocity, 2) the attachment position p. of the robot relative to the control point, and 3) the relative
3

---

yaw orientation 0 E IR of the robot base with respect to the control point orientation axis. Each of our actions of specifies the target PI) motor set points of robot r, which includes 10 motors (5 in each leg) for the Cassie biped used in our experiments. Note that π can be applied to any number of robots in any configuration provided that each robot can obtain its relative position pr and orientation 0r. The challenge is learn such a decentralized control structure that is effective and robust.
4 Learning Approach
We formulate learning the decMBC as reinforcement learning (RL) problem, as it involves solving a complex physical control task without supervision for low-level actions. Our approach employs Independent Proximal Policy Optimization (IPPO) [27], where the shared decMBC controller π is trained using data collected from multiple robots operating in parallel, without inter-robot commu- nication. Below we describe the architecture of decMBC, training environment and curriculum, reward function, and details of IPPO training procedure.
4.1 Neural Network Architecture
Following prior work on bipedal locomotion [28], represent the decMBC using recurrent neural network architecture, which allows for iistory-dependent action selection via the recurrent internal memory. Specifically, our network consists of two Long Short-Term Memory (LSTM) [29] layers, each containing 64-dimensional hidden states. These layers process the decMBC input at each time step to form the internal hidden state. This state is then processed by linear layer that produces a 10-dimensional action vector, specifying the 5 motor set points for each of Cassie's ' legs.
4.2 Episode Generation and Curriculum
Our IPPO training approach uses training data generated by executing training episodes involving one or more robots. During each episode, transitions from each robot are aggregated into a single replay buffer that is used to optimize a single actor and critic using the PPO objective [14]. Details of the PPO hyperparameters are in the Appendix E. The single actor at the end of training is taken to be the decMBC controller. Below, we describe the episode generation process, training configurations, and curriculum used to train the MBC.
Episode Generation. At the start of each training episode, we generate a random robot configu- ration according to the current curriculum stage (see below), including randomized weights on the rigid bars connecting robots. During an episode, the decMBC used to control the N robots, given a randomized sequence of carrier commands. Each command is executed for random number of time steps in the interval [100,450]. In addition, depending on the curriculum stage, random pertur- bation forces (0 to 50 N) are applied to the robot's pelvis and the carrier. Each episode ends after termination condition that checks whether a robot or the carrier has fallen, which is detailed in the Appendix C.
500 time-steps (10 seconds)
Training Configurations. Our current simulation time scales super-linearly as the number of robots grows. For this reason, we limit the number of robots used during training to N S 3. Empirically, have found that using training configurations with N < 3 leads to strong performance for N > 3. We generate random configurations for N E (1, as follows: For N = single Cassie robot is used with a small carrier, where the attachment point is located at the center of the carrier. For N = 2, two Cassie robots are used with their attachment points being endpoints of a rigid bar. The carrier control point located anywhere within a meter away from either of the robot's positions. For N = 3, an arbitrary triangular frame is formed by connecting three rigid bars. Here, the attachment points for three Cassie robots are the vertices of the triangular frame. In this configuration, the carrier control point can be randomized anywhere enclosed in the triangular frame [Figure (B)). Further details are given in Appendix A.
Training Curriculum. We initialize the decMBC controller and critic randomly and train it via IPPO using a 4 stage curriculum, where each stage is defined by the configurations and disturbances considered. Stage involves training with one robot (N = 1) with the objective of learning robust locomotion gaits operating for carrier commands at its attachment point. Stage 2 involves training with the addition of perturbation force and torsion force on the carrier such that robot is resistant to the disturbance forces. Stage 3 involves training with the addition of two-robot and three-robor configurations without any perturbation and torsion forces. This stage ensures each robot Iearns to

---

operate on the carrier control point randomly placed away from its attachment point. Stage 4 involves applying perturbation and torsion forces on the robot's base and carrier in addition to randomizing the mass of the rigid bar connecting the robots. This ensures each robot experiences disturbance forces and varying masses while operating with respect to the carrier control point.
4.3 Reward Design
Our reward function is designed to produce stable locomotion behavior while ensuring the robots collaboratively follow the commanded motion of the carrier control point. For each robot i in N- robot configuration, where i E {l...., N), the reward r; is composed of two components: the local robot reward rf and global reward و where r; = rf - - at each timestep throughout the episode. The local reward rf is designed to produce stable locomotion behavior of each robot, including maintaining consistent base height, penalizing jerky motion, regularizing proper foot alignment and the stepping frequency, with single-contact reward as proposed by Marum al. [30]. Further- more, the local reward minimizes torque of the motors to penalize excessive energy expenditure and minimizes the force exerted at the attachment point at the carrier during standing, ensuring that robots do not pull or push against each other but hold the carrier steady. The global reward r fo- cuses on the performance of the combined action of all robots in following carrier commands. It aims to minimize the deviation between the commanded and actual x-velocity, y-velocity, angular yaw velocity, and height of the carrier control point. We detail the reward components in Appendix B.
5 Experiments and Results
We conduct experiments to empirically evaluate the decMBC on the Cassie bipedal robot platform. The primary objective is to assess the effectiveness and adaptability to coordinate an arbitrary num- ber of robots, maintain stability, and adapt to diverse transport scenarios. We also discuss the scala- bility and generalizability of our approach across a range of practical settings including real-world transfer to multi-biped systems with two and three Cassie robots.
Experimental Procedure and Metries. We compute metrics such as drift, failure rate, and power consumption across various configurations to evaluate our decMBC policy. We also measure these metrics to compare our approach against other approaches. For each of these metrics across various carrier configurations, we run 1000 episodes for a maximum of 20 seconds, each with a fixed carrier command such as hold still, move forward, move sideways, and turn in-place. Drift measures the displacement of the carrier in the x/y direction and its orientational drift from its expected orientation and considers episodes that last for at least 1 second. We do not apply any perturbation forces for measuring drift. A negative and positive sign in drift indicates that the carrier is behind ahead of the expected global position for the desired carrier command. Failure rate is calculated as the percentage of episodes that terminate before reaching 20 seconds. This is evaluated under varying perturbation forces and payload masses fixed on the top of the carrier (except dynamic payload on Table 2). We apply ranges of perturbation forces and change payload masses in the carrier to measure the failure rate. Additionally, we calculate the Power consumption by all robots combined for each configuration.
5.1 Evaluating Scalability and Reconfigurability
Varying Numbers of Robots: We first evaluate the performance of the decMBC for varying num- bers of robots. For this purpose, we vary the number of robots from 2 to 10 using the same solid rectangular carrier. The robots are placed in a manner that ensures they are evenly distributed to maintain static stability. This typically involves positioning robots at the corners and, when nec- essary, placing additional robots in central position to maintain balance (see Figure 2). In order to provide reference point for strong performance, icy (1-R*) trained with its floating base as the control point. We consider achieving multi-biped performance that is comparable to 1-R* as a practical gauge of success.
evaluate the metrics for single robot pol-
Table gives results for the reference 1-R* and decMBC policies across the various control metrics. We first see that there is minimal performance variation across the range of 2 to 10 robots, showing that the decMBC is generalizing beyond its training scenarios of 1 to 3 robots. One exception is for N 2 which performs significantly worse the turn in in-place test, though still achieves relatively
5

---

small errors on other commands. This could be because of compromised stability arising from limited lateral support provided by two robots for a rectangular carrier. However, for > 2, the maximum orientation drift observed - -7.47, which is relatively small given the commanded 300 rotation. Overall, we see that the multi-robot performances are generally comparable to 1-R*.
Figure 3 presents failure rates for increasing carrier perturbation forces and payload masses as the number of robots varies, along with the power usage. The results show a sharp decrease in failure rate as the number of robots increases, achieving nearly zero failures. The cost for the robustness is increased power consumption. Notably, the power consumption scales approximately linearly with the number of robots, suggesting that the robots cooperate effectively without exerting excessive counter-forces on each other.
Figure 2: Configuration across varying numbers of robots for rectangular carrier with payload placed at the center. From left to right, we have 1-R*, and number of robots ranging from 2 to 10.
Hold still
V.=Imis V, -1, wmll
Move torward
Vg-0,
Moye sideways
Vy 1.m.255mk cm0
V,=0,
Vymti 2118%
ia-place
Air
Ay
A0
Ax
Ay
A6
Ax
Ay
A8
Ai
Ag
A0
T-R.
0.07
0.51
1,48
0,37
0.24
0.56
0.09
0.03
-185
0.16
0.28
1.29
1.64
0,93
0.91
0.03
00211
0.44
0.19
0.12
-12.98
0.06
0.21
0,26
-154
0,78
-0.55
0.13
00.00
-0.46
0.16
0.21
-6.00
Number of robots
4
0.06
0.08
-1.66
1,54
0.91
0.51
0.20
0032
0.40
0.08
0.38
6,71
0.08
0.13
-1.99
-1.56
0.72
.0.44
0.31
-0.44
0.57
-0.03
0.11
-7.04
4.79
-1.60
0.77
-0.42
0.27
-0.42
0.46
0.04
0.29
-6.88
0.06
0.10
4078
-1.63
0.64
-0.50
-0.59
0.51
0.12
-7.47
0.03
0.10
40.00
-164
0.71
0.38
0.27
0..88
0.58
0.05
0.22
-5.99
9
0.04
0.11
-0.65
-1,59
0.74
00442
0.27
-0.63
0.93
0.02
0.12
6,76
10
0.03
0.13
-0.34
-1.63
0.72
-015
0.32
-0.67
0.71
0.02
0.10
-6.28
Table 1: Positional (Ax-y in m) & orientational (A0 in 9) drift observed after 20 seconds for each command
with no perturbation force and payload mass of 20kg.
Failure rate with disturbances
Failure rate with payload mass
Power consumption
x10*
Perturbation (N)
100
Payload mass (kg)
100
20
60
80
Falure rale (%)
75
20
30
70
60
Power (W)
Falura cate
40
50
50
25
Number rebots
Number of robots
Number of rooots
Figure 3: (Left) Failure rate for varying number of rohots carrying payload mass of 20kg experiencing various perturbation forces from 0 to 100N. (Center) Failure rate for varying numbers of robots for different payload masses from 20 to 80kg averaged over all perturbation forces. (Right) Power consumed per second by all robots combined and averaged over all perturbation forces and payload masses.
Varying carrier configuration and payload: In real-world scenarios, payload transport tasks often involve diverse carrier shapes and sizes to accommodate various objects and environments. We demonstrate the adaptability of our approach to such practical settings by evaluating the decMBC's performance on three distinct carrier configurations [see Figure (C)), each representing a different real-world use case. The first configuration, Sacks, involves four robots attached to a circular carrier with five sacks of varying weights, totaling 35 kg, randomly fixed on top. The second configuration, Log, uses a long plank
carrier, with three robots attached collinearly, carrying a cylinder that is fixed on the carrier, simulating log weighing 20 kg. The final configuration, Dynamic, shows four robots attached to a carrier that is a container holding rolling ball with total mass of 30 kg.
We observe from Table 2 that the controller maintains near- zero drift across all commands and con- figurations, with the exception of orientation drift during the turn in-place command for dynamic loads. In this case, the drift 77.995", representing an approximately 25% error from the com- manded 300". This discrepancy may be attributed to the unpredictable motion of the rolling ball, which alters the momentum of the carrier. Nevertheless, given that the decMBC policy has never seen these types of payloads during training, the failure rate seems reasonably low.

---

V2-0, va=
Hold atill
Move forward V,-lm/s, V,=0,
V:-0,
Move sideways
1,,0.55ms und
Tum in-place V240,
1,00,
Meun Failuure Rate (%)
Configurations
Ax
Ay
A6
Az
Ay
A8
Ar
Ay
A6
Az
Ay
A0
Sacks
0.11
0.09
-1.60
-149
1.30
-0.60
-0.07
0.54
0.96
0.06
0.55
-13.15
14.71
Log
0.23
0.32
1.12
-1.14
0.50
0.93
0.49
-0.70
1.36
-0.02
0.05
-7.86
8,58
Dynamic
0.15
0.06
-0.57
-3866
0.44
-0.25
0.07
-0.54
-0.24
-0.07
-0.20
-77.95
17.90
Table 2:
Positional (Ax-y in m), orientational (A0 in 9) drift with no perturbation forces observed after 20 seconds & mean failure (%) averaged over all commands, and perturbation forces.
5.2 Comparison with Centralized and Specialized policies
In this section, we compare our decentralized and adaptable decMBC with two types of straightfor- ward but inflexible systems: centralized policies and specialized policies. The aim is to demonstrate that our approach, despite its flexibility and generalizability. does not significantly compromise per- formance compared to these more rigid alternatives. A centralized policy is one where all robots' states are used produce actions for each robot from single controller. This approach relies on a comprehensive view of the entire system to make coordinated decisions. A specialized policy, on the other hand, is designed for specific carrier configuration with fixed number of robots. It is trained to perform optimally under these specific conditions but lacks the flexibility to adapt to different setups.
From our observations in Table 3 and Figure 4, the drift values for our approach are reasonably close to the centralized policy. Interestingly, our approach outperforms the centralized policy for two- robot configuration on failure rate, but the performance worsens the three-robot configuration. We can also observe that the power consumption of our approach is slightly higher than that of the centralized approach. This difference could be attributed to the centralized controller's ability to produce better coordination due to its full observation of the system state.
Interestingly, from the observations in Table and Figure 5, our decentralized approach outperforms the specialized policy for Rectangle and T-shape configurations in terms of failure rate, with cost of slightly more power consumption. Our argument for this result is that the decentralized policy, through its adaptive nature, may have generalized to alternative variations of similar configurations, making it more robust to unexpected disturbances.
While our decentralized approach shows comparable or better performance in many scenarios, cen- tralized policies theoretically have higher potential capacity due to their access to global information. However, our attempts at training centralized controllers have not yielded substantial improvements, possibly because our decentralized architecture is sufficiently rich for near-optimal performance un- der rigid attachments, or because the higher dimensionality of the centralized approach makes it more challenging to train effectively. For instance, training a centralized controller for a 3-robot configuration took about 1000 hours to reach peak reward, compared to 245 hours for our decentral- ized policy. This suggests that while centralized policies may have higher theoretical capacity, our decentralized approach offers a favorable trade-off between performance and training efficiency.
Hold V,=0,
Move forward V,almis, V,=0, wm0
Move sideways Vg=0,
Turn in-place V,=0, V, -0, 15%
V,,00
Number of Robots
Policy
Ar
Ay
A0
Aæ
Ay
A0
Ar
Ay
A0
Ax
Ay
A0
Centralized
0.35
-0.29
0.55
-1.68
0.43
0.93
0.44
-1.45
0.24
-0.18
0.05
-3.74
Ours
0.07
0.18
1.15
-1.03
0.97
1.00
0.71
-0.73
-1.14
-0.20
-0.07
-478
Centralized
0.02
0.06
-0.09
-1.53
0.08
0.85
0.19
-0.68
0.20
-0.09
-0.66
-1.98
3
Ours
0.17
0.17
0.33
-1.66
0.57
-0.14
-0.01
-0.44
-1.01
0.08
-0.07
-6.38
Table 3: Positional (Ax-y in m) & orientational (A9 in 9) drift observed after 20 seconds for each command with no perturbation force, comparing Centralized and Our approach.
5.3 Sim-to-real Transfer
We evaluated our policy on real hardware consisting of two and three bipedal robots, Cassie, attached the endpoints of an I-shaped and T-shaped carrier made from aluminum extrusion with a weight holder. We also added payload the weight holder and used it as carrier control point [Figure (D)). We deployed the decMBC policy each of the robots, running independently, providing
7

---

Policies:
Central
Ours
2 Cassie in-line
Cassie in triangle
x10*
Power consumption
15
20
Failure rate (%)
10
rato
1.
Power (W)
1.0
30
0.0
70
40
60
80
100
20
40
60
80
100
Number of Robots
Figure 4: (Left, Center) Failure rate for 2-Cassie in-line and 3-Cassie in triangular configuration experiencing various perturbation forces from 0 100N. (Right) Power consumed per second by all robots combined and averaged over all perturbation forces and commands Centralized and Our policy.
Hold still V,=0,
Move forward V,=lmh, V,=0),
Move sideways k,500.25ms,
Turn in-place 1, 1=15%
Policy
Ay
A8
Ay
A8
Ay
A0
Ay
Specialized
1.29
0.40
1.34
-115
-.56
1.39
0.12
1.22
0.01
-3.89
Rectangle
Configurations
Ours
0.08
-1.66
1.54
0.91
-0.51
-0.32
0.40
0.38
-6.71
Specialized
-0.69
0.31
-1.33
2.188
0.37
-3.23
-0.22
.0.69
-1.72
0.10
0.09
-10.39
L-shape
Ours
0.12
-0.04
0.37
-0.95
0.63
-0.45
0.43
-1.22
2.46
-0.85
-0.59
-18.65
Specialized
1.37
0.57
-3.89
-1.38
-0.42
-2.97
0.33
-0.38
-1.05
-0.32
-0.11
-20.28
T-shape
Ours
0.23
0.03
-0.60
-1.00
0.74
-0.28
0.60
-1.41
2.41
-0.61
40.58
-16.23
Table 4: Positional (Ax-y in m) & orientational (A0 in ") drift observed after 20 seconds for each command with perturbation force, comparing Specialized and Our approach.
Rectangle
L-shape
Tishape
Foilure rate in Rectangle
Fallure rate in L-shape
Failure rote in T-shape
x10²
Power consumption
10.0
7.5
Perjer OWO
5-
01
0.0
40
80
1.00
20
60
100
40
80
Rectongle
Lahape
shapee
Force (N)
Force (N)
Force
Configurations
Figure 5: (First, Sccond, Third) Failure rate for three configurations experiencing various perturbation forces from 0 to 10ON. (Fourth) Power consumed per second by all robots combined averaged over all perturbation forces and commands for three configurations.
their relative positions with respect to the carrier control point. We use a joystick to broadcast the same carrier command values to all the robots using the UDP protocol. We commanded x- x-velocity, y-velocity, angular yaw velocity, and height to maneuver the payload from one point another. We observed that successfully able maneuver different payloads according various commands for extended period of operation.
Limitations
Despite the promising results, there are clear limitations address in future work. First, the decMBC is not explored on diverse terrain types beyond flat surfaces, eliminating the understanding of its performance in more challenging environments. Second, real-world experiments demon- strated to up to three Cassie robots, despite the simulation results showing the scalability of up to 10 robots. Third, the controller relies solely on proprioceptive sensing and relative robot positions without incorporating additional sensing modalities like camera perception which could enhance its capabilities in complex environments. Fourth, there are several practical challenges associated with multi-biped systems that require further consideration, including higher costs, increased points of failure, perception difficulties due robot proximity, and the complexity of deployment and main- tenance. These factors highlight important trade-offs between system flexibility and operational policy is trained with a wide range of relative positions of each
practicality. Finally, although robot with respect to the carrier point, which should cover most real-world scenarios, our policy does not guarantee it may work for positions beyond the range we have set during training.

---

Acknowledgments
This material is based work supported by the National Science Foundation under Grants 2321851 and IIS- 1524360 and USDA 2021-67021-3532 back, which improved the quality of this paper. In addition to the authors, we also appreciate the support from our colleagues, Pranay Dugar and Bart van Marum, for their assistance with robot operation and repair.
We thank all the reviewers for their helpful feed-
References
[1] R. Tallamraju, P. Verma, V. Sripada, S. Agrawal, and K. Karlapalem. Energy Conscious Over- actuated Multi-Agent Payload Transport Robot: Simulations and Preliminary Physical Valida- tion. In 2019 28th IEEE International Conference Robot and Human Interactive Commu- nication (RO-MAN), pages 1-7, Oct. 2019. doi: 111.111111000010000.... 2019 9.896442. URL https: htops://iieexpporrtt ieee. wttp://cwaaa///66/////e/eeeeeeee 1944-9437.
ISSN:
[2] Q. Liu, Z. Nie, Z. Gong, and X.-J Liu. An Omnidirectional Transportation System With High Terrain Adaptability and Flexible Configurations Using Multiple Nonholonomic Mo- bile Robots. IEEE Robotics and Automation Letters, 8()):6600-6677 Sept. 2023. ISSN 2tw.11000/RRA.223.3330 abstraatt/oocumnnt/tee ters.
2377-3766.
URL https: htipe:///eeeppooreee ieee. .eee.org/ org/ Conference Name: IEEE Robotics and Automation Let-
[3] W. Yu, D. Jain, A. Escontrela, A. Iscen, Xu, E. Coumans, S. Ha, J. Tan, and T. Zhang. Learning to Walk on Complex Terrains with Vision. Proceedings of the 5th Conference on Robot Learning, pages 1291-1302. PMLR, Jan. 2022. URL https: //proceedings. mlr. press//1644/yu22.aaaa. html. ISSN: 2640-3498.
[4] H. Duan, B. Pandit, M. S. Gadde, B. van Marum, J. Dao, C. Kim, and A. Fern. Learning -aseed Bipedal Locomotion for Challenging Terrain, 2023. URL https://arxiv. org/ /t//b///220/ 2309 14594. Publisher: arXiv Version Number:
[5] T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning robust per- ceptive locomotion for quadrupedal robots in the wild. Science Robotics, 7(62)eabb2222, Jan. 2022. doi:10. 120////crroooccc.bbb22222.. URL https http://wwwssccc science. wwwienc.oor//ooo...............e org/doi/10. 1126/ scirobotics abk2822. abk2222
Publisher: American Association for the Advancement of Science.
[6] H. Zhu, S. Yang, W. Wang, X. He, and N. Ding. Cooperative transportation of tether- suspended payload via quadruped robots based on deep reinforcement learning. In 2023 IEEE International Conference on Robotics and Biomimetics (ROBIO), pages 1-6, Dec. 2023. 1oi: 110::0011199000055851...2222 10354488. URL https: https://ieeexplloee/ ieee. Destractt/dccumet//OOott
[7] Kim, R. T. Fawcett, V. R. Kamidi, A. D. Ames, and K. A. Hamed. Layered Control for Co- operative Locomotion of Two Quadrupedal Robots: Centralized and Distributed Approaches. IEEE Transactions Roboties, 39(6):4728-44) 47448, Dec. 2023. ISSN 1941-0468. doi: 10. URL /ttps::/ieeexpplore www.eeeeeeeeeee..eee www.///astaee//toceeeeeeoooo 10281391. Conference Name: IEEE Transactions on Robotics.
(109/TRO.2023.31199
[8] Kim, J. Lee, and D. Ames. Safety-Critical Coordination for Cooperative Legged Loco- motion via Control Barrier Functions. In 2023 IEFE/RSJ International Conference on Intelli- gent Robots and Systems (IROS), pages 2368-2375, Oct. 2023. doi: 10. 109//0555552.2023, 10341987. URL https://ieeexpleee ieee httg///cmmentt114110009 ISSN: 2153- 0866.
[9] G. B. Margolis and P. Agrawal. Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior. In Proceedings of The 6th Conference on Robot Learning, pages

---

22-31. PMLR, Mar. 2023, URL URL.ttps::/proceedngs htt.:/edddinn........rsrsssseees httppp/ss/////////rrlllaaa2.. html. ISSN: 2640-3498.
[10] Siekmann, K. Green, J. Warila, A. Ferm, and Hurst. Blind Bipedal Stair Traversal via Sim- to-Real Reinforcement Learning. In Robotics: Science and Systems XVII. Robotics: Science and Systems Foundation, July 2021. ISBN 978-0-9923747--7888 061. URL http:://wwww.ooooo...ooooeeeeeea
15077//SSS. pdf.
[11] Y. Li, S. Li, V. Sitzmann, Agrawal, and A. Torralba. 3D Neural Scene Representations for Visuomotor Control. In Proceedings of the 5th Conference Robot Learning, pages 112-123. PMLR, Jan. 2022. URL https:// httpp://pprocesdings mlr /ttppp/caaaaaaaa....eeeeeee///a. html. ISSN: 2640-3498.
[12] W. Yuan, C. Paxton, K. Desingh, and D. Fox. SORNet: Spatial Object-Centric Represen- tations for Sequential Manipulation. In Proceedings of the Sth Conference on Robot Learn- ing, pages 148-157. PMLR, Jan. 2022. URL yuan22a html. ISSN: 2640-3498.
mlr wal..press/ wwwpppersssaaa
[13] A. Agarwal, A. Kumar, J. Malik, and D. Pathak. Legged Locomotion in Challenging Terrains using Egocentric Vision. In Proceedings The 6th Conference on Robot Learming, pages 403- 415. PMLR, Mar. 2023. URL https: https://prooeeedinge mlr. press/ pptps//////aaeeeeeeeeeee v225/agaaraa223aa html. ISSN: 2640-3498.
[14] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal Policy Optimization Algorithms, Aug. 2017. URL [es].
06347. arXiv:1707.06347
[15] Chen, A. Murali, and A. Gupta. Hardware Conditioned Policies for Multi-Robot Trans- fer Learning. Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL https: https://proceedding /ttp:///ccceeeee.ee/ee/e nwwrips hccpapper/2118/haatt
[16] K. A. Hamed, V. R. Kamidi, A. Pandala, W.-L. Ma, and A. D. Ames. Distributed Feedback Controllers for Stable Cooperative Locomotion of Quadrupedal Robots: A Virtual Constraint Approach. In 2020 American Control Conference (ACC), pages 5314-5321, July 2020. doi: 000999//AACC5554220009944444 9147673. ISSN: 2378-5861.
URL https://ieeexplorts ieee. org/document/////oooooooooo
[17] Haarojja, B. Moran, G. Lever. S. H. Huang, Tirumala, J. Humplik, M. Wulfmeier, Tun- yasuvunakool, N. Y. Siegel, R. Hafner, M. Bloesch, K. Hartikainen, A. Byravan, L. Hasen- clever, Y. Tassa, F. Sadeghi, N. Batchelor, F. Casarini, S. Saliceti, C. Game, N. Sreendra, K. Patel, M. Gwira, A. Huber, N. Hurley, F. Nori, R. Hadsell, and N. Heess. Learning agile soccer skills for a bipedal robot with deep reinforcement learning. Science Robotics, 9(9)); eadi8022, Apr. 2024. doi: 10i:10.1126/saiie wtt.sscrooootcc.aaiii000. doi/10. 1126/scirobotice. adi8022. Publisher: American Association for the Advance- ment of Science.
URL https: htwps:///ww... www..science. srience.org/
[18] D. Tirumala, M. Wulfmeeer, Moran, S. Huang, J. Humplik, Lever. T. Haarojja, I.. Hasenclever, A. Byravan, N. Batchelor, N. Sreendra, K. Patel, M. Gwira, F. Nori, M. A. Riedmiller, and N. Heess. Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning, May 2024. URL http:://ww..emmmniccchho....s. www.stiismoooileaaee.eeeee org/paper/ Sraplagg--oootttoocc----o--r--e- 4499@aafeee400000d10bocccoo22220
[19] Y. Ji, B. Zhang, and K. Sreenath. Reinforcement Learning for Collaborative Quadrupedal Ma- nipulation of Payload over Challenging Terrain. In 202/ IEEE 17th International Conference on Automation Science and Engineering (CASE), pages 899-904, Aug. 2021. doi: doi:10.11 109/ L.SE49939.2021.95511.1 URL https://ieeexplon ieee. org/ http///ocmmnt/eeee 95551481.
ISSN: 2161-8089.
10

---

[20] G. Eoh and T.-H. Park. Cooperative Object Transportation Using Curriculum-Baas Deep Reinforcement Learning. Sensors, 21(14):4780, Jan. 2021. ISSN 1424-8220. doi:10.3390/ s21 144480. URL https:/ htts::/www.mmoee com/ 1424-8220/21114447778 Number: 14 Pub-
lisher: Multidisciplinary Digital Publishing Institute.
[21] R. T. Fawcett, L. Amanzadeh, J. Kim, A. D. Ames, and K. A. Hamed. Distributed Data-Driven Predictive Control for Multi-Agent Collaborative Legged Locomotion. In 2023 IEEE Inter- national Conference on Robotics and Automation (ICRA), pages 9924-9930, May 2023. doi: 10.1109/1CR/48881. .0233. 10160914. URL httpt:://tieeeppllleeee . org/abstratt/ document/10160014
[22] L. Zhang, Y. Sun, A. Barth, O. Ma. Decentralized Control Multi-Robot System in Cooperative Object Transportation Using Deep Reinforcement Learning. IEEE Access, 8:
184109-184119, 2020. ISSN 2169-3536. 010100010099ACCEESS.22203333333 2020.332222220022223 doi:10.1109/ACCEEE
URL https:
//ieeexplore iece. htt.////sstttt///oooot////////// cess.
Conference Name: IEEE Ac-
[23] Lin, X. Yang. Zheng, and H. Cheng. End-to-end Decentralized Multi-robot Navi- gation in Unknown Complex Environments via Deep Reinforcement Learning. In 2019 IEEE International Conference Mechatronics and Automation (ICMA), pages 2493-2500, Aug. 2019. doi: 10.1109/CMM.2219..888 document/888 6208.
URL https: https://eeeexplore ieee org/
ISSN: 2152-744X.
[24] J. Horyna, T. Baca, and M. Saska. Autonomous Collaborative Transport of Beam- Pay- load by Pair of Multi-rotor Helicopters. In 2021 International Conference Unmanned Aircraft Systems (ICUAS), pages 1139-1147, June 2021. doi: 10: 1000100018 9476789. URL
ieee. oewe..ege/eeeueeeee /ttg//dccmmett99777...... ISSN: 2575-7296.
[25] L. Pei, J. Lin, Z. Han, L. Quan, Y. Cao. C. Xu, and F. Gao. Collaborative Planning for Catch- ing and Transporting Objects in Unstructured Environments. IEEE Robotics and Automa- tion Letters, 9(2):1098-1108 100, Feb. 2024. ISSN 2377-3766. 101.0110//RRR..22...... URL https:: /atess//eeeeeppppo ieee. orgg/documet//10255.. Roboties and Automation Letters.
Conference Name: IEEE
[26] J. Dao, K. Green, H. Duan, A. Fern, and J. Hurst, Sim-to-Real Learning for Bipedal Loco- motion Under Unsensed Dynamic Loads. In 2022 International Conference on Robotics and Automation (ICRA), pages 10449-10455, May 2022. doi: 110.000/CCCAA449922220098118 URL httphttss//// httt:::://eeeeppooeee ieee. .rg . wtgg/occment//881785
[27] C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, and Y. Wu. The Surprising Effective- ness of PPO in Cooperative Multi-Agent Games. Advances in Neural Information Processing
Systems, 35:24611 15:24411-24624, Dec. 2022. URL https: nips wc/pappr_fiilee.e Dasictt-Datttts__aee |ppp://02222aall/t...ccccee32322 Benchmarks html.
[28] Siekmann, S. Valluri, J. Dao, Bermillo, H. Duan, Fern, and Hurst. Learning Memory-
Based Control for Human-Scale Bipedal Locomotion. In Robotics: Science and Systems XVI. Robotics: Science and Systems Foundation, July 2020. ISBN 978-0-99223447-6-1. doi: 10.
15607/RSS.20000 2020 pdf.
URL http:://www..ooorccppooeeeeeeeee http://diiinnnoorrsss/s///////oe
[29] S. Hochreiter and J. Schmidhuber. Long Short-Term Memory. Neural Computation, 9(8): 1735-1780, Nov. 1997. ISSN 0899-7667. //doi.org/10. 1162/neco...oo 1997.9 1733.
URL https:
[30] B. van Marum, A. Shrestha, H. Duan, Dugar, Dao, and A. Fern. Revisiting Reward Design and Evaluation for Robust Humanoid Standing and Walking, Apr. 2024. URL http: // //axiiv. org/ abs//404. 19173. arXiv:2404.99173 [cs].

---

Appendix
A Implementation details
In this section, we describe the implementation details of our decMBC approach, focusing on the configuration generation, and randomization of parameters used during training.
For generating N-robot configuration during training, where N {1, 2, 3), choose relative po- sition p; Vi E {1... N] with respect to the carrier control point which is 2D polar coordinate indicated by (Rq, θ Figure shows an example of robot configurations during training. In Fig- ure 6a (N = 1), the carrier control point is located at the base of the robot, therefore relative position (R) is always zero. Figure 6b (N = 2), we randomize carrier control point location up to meter from any of the robots, and similarly, in Figure 6e (N = 3), we randomize carrier control point to be located anywhere enclosed within the triangular frame, which produces range of rela- tive position (R,, θ2). Furthermore, in each of these configurations, randomize the mass of the rigid bar connection shown in Table 5.
The ranges of relative positions and masses seen during training generalize for various carrier con- figurations with an arbitrary number of robots and payloads as observed in the experiments.
Carriar
θ
point
Robot 1
Robot 1
Ra
Svv soneutin suied posuo
R
e
Control point
Robot 3
Robot 2
Robot
(a) 1-robot configuration
(b) 2-robot configuration
(c) 3-rohot configuration
Figure 6: Training configuration using 1, 2 and 3 robots
Parameter
Ranges
X velocity (V,)
[-0.5, 2.0] m/s
Y velocity (V2)
[-0.3, 0.3] m/s
Angular velocity
[-x/8, m/8) rod/s
lleight (h)
[0.5, 0.8) m
[0, 3.5) m
e,
[-x, æt] rad
Commands Duration
[100, 450) timesteps
Force Duration
[50, 200) timestepp
[0, 10] ky (1-robot configuration)
Rigid bar
[0. kg (2-robot configuration)
[0, 15] ky (3-robot configuration)
Table 5: Parameters ranges for training
12

---

B Reward details
Below, outline the reward terms used in implementation. The reward components di- vided into two categories: Local rewards rf designed to produce stable locomotion for each robot, described in the local robot frame. The Global rewards r are designed for following carrier commands with the combined action of all robots in the control point frame.
Category
Reward Term
Definition
Weighting
0.0
if o
Foet airtime
1.0
Σ Erean 1,air 0.35| Lid,
(n) 2) + 0.5 (n² == 1)
if
Feet contact
0.1
(n) == 1)
exp(-10 |seeti - feet) exp(()
if 0 clsc
Feet stance X
0.02
carp--54 [ Seet og 0.385
if
Feet stance Y
0.02
cxp(()
else
+
Feet orientation
exp(-30 Exea, Uol fawp bosetpu))
0.15
Local Robot
Relative yaw orientation
10//T
0.5
iE (1
cxp(-0.2 00.2 emp(()
[[joint formaay |)
if 0and N
Joint force
0.1
else
Base Height
bbase
h]
0.2
Base acceleration
capl 0.01 Σ Ibase ryall
0.1
Action difference
capl 8 Ela af-1l)
0.1
Torque
capl
0.05
G
X,Y velocity
exp(-2 Vay))
0.15, 0.1
Global Carrier
Orientation
-el(Oerpy w), cap(- 30 qa(O w\)
2.0, 0.15
Table 6: Description of reward terms
Notations used in Table for the reward description.
Number of robots in the system
C Carrier command consisting of linear velocity, angular velocity, and base height.
G = (Va, Vy,w,
G = means a hold still command
t fair = air time for a particular feet
T T boolean variable indicating touchdown for particular feet
n number of feet in contact with the ground
base Robot pelvis global pose
feet' Pose of each foot in robot base frame
qa(-) Quaternion distance function
0 Yaw orientation of each robot relative the Carrier
joint force Force sensor reading on the ball joint connection
= Action taken by the policy
Motor torques
13

---

Oc Control point on the carrier
Current linear velocities of the control point
C
Termination condition
An episode is terminated if any of the following conditions are met:
The carrier's pitch or roll Ocrp is outside the range 1-30",30)
The robot's pelvis pitch or roll base, is outside the range ,-30", 30%]
The robot's 's knee collides with the ground or with each other.
The relative yaw orientation 0j of each robot is outside the range [-300, 30%)
The robot's pelvis height base is outside the range [0.5, 1.0]
The episode duration exceeds 500 timesteps (10 seconds).
D Dynamics Randomization
To facilitate transfer from simulation to the real robot, we employ dynamics randomization through- out all stages the curriculum during training. This involves randomizing various physical param- eters of the simulated environment, such as joint damping, link masses, center of mass positions, encoder noise, ground friction, terrain variations, and adding noise to states and policy rate. Table lists the dynamics randomization ranges used.
Element
Range
Joint Damping
[-50, 250] %
Body Mass
[-25, 25] %
CoM Position
[-1, %
Friction
[-20, 20] %
Encoder Noise
[-0.05, 0.05] red
Ground Slope
1-0.05, 0.05] rad
Table 7: Dynamics randomization ranges
E PPO Hyperparameters
Table 8 lists the hyperparameters used for the PPO training.
Hyperparameter
Value
Learning Rate (Actor & Critic)
3e-4
PPO Clip Range
0.2
Number of Epochs
5
Batch Size (Episodes)
32
Discount Factor (Gamma)
0.95
GAE Lambda
1.0
Value Function Coefficient
0.5
Entropy Coefficient
0.01
Max Grad Norm
0.05
Buffer Size
60000
Table PPO Hyperparameters
14


Reviews:
[{"review_id": "rhLYtG1EGe", "reviewer": "Reviewer_vVS6", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper presents a decentralized controller designed for varying numbers and configurations of rigidly attached bipedal robots, specifically within a payload transport scenario, without necessitating retraining. The authors provide simulation-based quantitative metrics to demonstrate the effectiveness of their approach across a diverse array of transport scenarios. Furthermore, the controller's capabilities are validated in real-world experiments involving systems composed of two and three Cassie robots. Notably, this work represents the first instance of a scalable multi-biped payload transport system.", "summary_of_recommendation": "This paper presents the first decentralized controller designed for varying numbers and configurations of rigidly attached bipedal robots, specifically within a payload transport scenario, without necessitating retraining. The authors provide simulation-based quantitative metrics to demonstrate the effectiveness of their approach across a diverse array of transport scenarios. Furthermore, the controller's capabilities are validated in real-world experiments involving systems composed of two and three Cassie robots. Notably, this work represents the first instance of a scalable multi-biped payload transport system. However, the paper suffers from the absence of ablation study/ insufficient baselines.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "Please refer to the above section.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pAqR2UNOqN", "reviewer": "Reviewer_wdBL", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes a decentralized controller for a variable number of bipedal robots to collaboratively manipulate payloads, with sim-to-real transfer.", "summary_of_recommendation": "The paper is well written, thorough, and novel, with high relevance to robotics. It would be enhanced by additional discussion of the results and the practical applications of its problem setting.", "technical_quality": 4, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "The comparison with centralized and specialized policies seems to mostly suggest that the learning dynamics of the decentralized policy are better than those of the centralized policy or specialist policies. But from a capacity standpoint, it seems like the centralized and specialist policies should have strictly higher potential than the decentralized one, because they have access to more information about the system and task respectively. I wonder what would happen if you pretrained the policies using the decMBC approach before continuing training with a centralized or specialized architecture. Currently the results suggest that decentralization is actually better and it would be great to attribute the cause. Can you discuss the results of section 5.2 a bit and whether I am thinking about this reasonably?\n\nCan you add more discussion of the practical applications as mentioned above?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aI3zID7Ok9", "reviewer": "Reviewer_eduL", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper designs a decentralized controller for multi-biped robot carriers, which can be applied to varying number of robots and various configurations. The controller is trained through a decentralized multi-agent reinforcement learning method, independent PPO (IPPO). The proposed method are evaluated on extensive settings in simulation, and achieve good performance with regards to drift, failure rate, power consumption for payload transport, reaching flexible and robust policy compared to baselines. The proposed framework is also validated on real robots.", "summary_of_recommendation": "This paper presents a solid work on multi-biped payload transport with multi-agent reinforcement learning. Extensive experiments are implemented to show its effectiveness in both simulation and on real robots.", "technical_quality": 4, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "* It looks like the the robots are distributed with symmetry under payloads as the number of agents increases in Figure 2. Would the robustness of the policy be affected when this symmetry is disturbed?\n* Another potential benefit of applying decentralized multi-agent policy to robotic scenarios is better sim-to-real transfer, as indicated in the following work. It would be beneficial to discuss this in the results or consider it in the future work.\n    * Dynamic Handover: Throw and Catch with Bimanual Hands. Huang et al., CoRL 2023.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Strengths:
- First scalable decentralized controller for multi-biped payload transport, showing effectiveness in both simulation and real-world experiments
- Robust generalization to out-of-distribution robot numbers and payload shapes without retraining
- Well-written paper with thorough experimental results and honest discussion of limitations 
- Novel training curriculum design for multi-robot tasks

Weaknesses:
- Experiments limited to flat terrain, misaligned with motivation of bipedal robots for rough terrain
- Lack of detailed ablation studies on multi-stage training process impacts reproducibility
- Limited comparison to other multi-robot control method baselines
- Insufficient discussion of practical real-world applications and advantages over alternatives like large quadrupeds
Decision: Accept