Learning Decentralized Multi-Biped Control for Payload Transport

Authors: bikram  pandit
ashutosh  gupta
mohitvishnu s gadde
addison  johnson
kumar  shrestha
helei  duan
jeremy  dao
alan  fern
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-07 09:20:14

Abstract: Payload transport over flat terrain via multi-wheel robot carriers is well-understood, highly effective, and configurable. In this paper, our goal is to provide similar effectiveness and configurability for transport over rough terrain that is more suitable for legs rather than wheels. For this purpose, we consider multi-biped robot carriers, where wheels are replaced by multiple bipedal robots attached to the carrier. Our main contribution is to design a decentralized controller for such systems that can be effectively applied to varying numbers and configurations of rigidly attached bipedal robots without retraining. We present a reinforcement learning approach for training the controller in simulation that supports transfer to the real world. Our experiments in simulation provide quantitative metrics showing the effectiveness of the approach over a wide variety of simulated transport scenarios. In addition, we demonstrate the controller in the real-world for systems composed of two and three Cassie robots. To our knowledge, this is the first example of a scalable multi-biped payload transport system.

Main paper: 
 introduction: 
 Multi-wheel carriers are the most common way to transport payloads across well-structured terrain due to their ease of control, effectiveness, and configurable wheel arrangements [1] . Further, researchers have also considered wheeled transport that can be reconfigured on the fly via multiple cooperating wheeled robots [2] . While these transport systems excel in structured environments, they struggle in unstructured and challenging terrain, which limits their general applicability. This work aims to overcome such limitations while retaining the control and reconfigurability of multi-wheel systems. To achieve this, we propose the multi-biped transport problem, where wheels or wheeled robots are replaced with bipedal robots that can be arbitrarily attached to carriers. This setup allows for transport over various terrains, potentially extending to more challenging environments [3, 4, 5] .Multi-biped transport presents significant challenges. Legged robots are generally unstable and highly dynamic, making them much harder to control compared to wheeled robots. Maintaining stability while transporting payloads and coordinating multiple legged-robots adds further complexity to the control problem [6] . However, the use of multiple bipeds offers distinct advantages over alternatives such as large quadrupeds, providing greater flexibility in payload transport. This approach allows the number and arrangement of robots to be tailored based on the weight and size of the payload. Such versatility opens up potential applications in various areas including construction, moving systems, warehouses, agriculture, and delivery systems.Prior works on multi-legged transport systems have primarily focused on model-based approaches that rely on accurate modeling of robot interaction and explicit inter-robot communication, which limit their reconfigurability and practicality [7, 8] . While learning-based approaches have emerged as promising alternatives [9, 10, 11, 12, 13, 14, 15, 16, 17, 18] for various control problems, existing methods on multi-robot system often assume centralized control with strong assumptions on system observability, leading to limited scalability, robustness, and vulnerability to point of failure [19] . C) Highlights unseen carrier and payload scenarios which the decMBC policy was evaluated for in the simulation. D) Showcases sim-to-real deployment where each robot running a copy decMBC policy on hardware.To overcome these challenges, we propose an architecture and training approach for a decentralized Multi-Biped Controller (decMBC). This controller supports high-level motion control of a carrier that is attached to any arbitrary configuration of bipedal robots. Importantly, to maintain maximum reconfigurability, the decMBC is decentralized in the sense that each biped can be controlled using only information about its local state and its relative position with respect to a desired reference point on the carrier. In addition, the decMBC can be applied, without retraining, to any number of bipeds that are attached to a carrier in arbitrary stable configurations. The proposed decMBC is trained using decentralized reinforcement learning (RL) over a distribution of robot-carrier configurations carefully selected to provide generalization to arbitrary biped configurations and facilitate sim-toreal transfer. The approach is easy to implement and can be used with a wide range of existing single-agent control architectures and RL algorithms, making it highly adaptable and accessible.We validate the effectiveness and flexibility of our approach through extensive experiments across varying numbers of robot and carrier configurations. Our quantitative results in simulation highlight the system's ability to maintain stability, coordinate effectively, and adapt to different payload transport tasks. Furthermore, we demonstrate sim-to-real transfer to multi-biped systems involving two and three Cassie robots. To the best of our knowledge, this is the first multi-leg transport system to demonstrate this level of reconfigurability in simulation and the real world. 
related work: 
 Multi-robot transport has attracted significant research attention due to its potential to overcome the limitations of single-robot systems, especially in handling large, heavy payloads and navigating diverse real-world environments. Several works have focused on wheeled robots for collaborative transport which developed a composite connector for non-holonomic mobile robots, but their work primarily focused on well-structured environments [2] . Moreover, Eoh and Park [20] propose a cooperative object transport controller using deep reinforcement learning which uses simple nonholonomic wheeled robots to learn the collaborative pushing of an object to its goal position. While their approach shows promise, it assumes that the robots can sense the positions of the object and the goal with respect to each robot. This makes scaling the approach for arbitrary robot-carrier configurations difficult.Similarly, other works have considered predictive control and decentralized approaches. Fawcett et al. [21] proposed a predictive control approach for robust locomotion of three holonomic constraint quadrupeds, but it did not allow for formation changes or scaling to more agents. Zhang et al. [22] introduced a decentralized control approach using DQN controllers, but they strongly assumed that each robot could perfectly sense the state of the payload and every robot. Lin et al. [23] develop an end-to-end decentralized deep reinforcement learning policy to learn multi-robot control directly from LiDAR measurements. They employ centralized learning and decentralized execution to deploy on holonomic UGVs. However, their approach relies on the assumption that the team-level action should be a concatenation of all individual actions, which limits its flexibility.On the other hand, researchers have also explored multi-legged transport systems. Kim et al. [7, 8] proposed layered control strategies and control barrier functions for holonomically connected quadrupeds, but their approaches were not generalized for any number of robots and relied on strong assumptions. Ji et al. [19] demonstrated a reinforcement learning strategy for four quadrupedal robots executing a collaborative task, but their approach was limited to a fixed configuration and number of agents, and the results are shown only in simulation. Other notable works include Horyna et al. [24] , who developed a collaborative transport control for two multi-copters using a masterslave configuration, and Pei et al. [25] , who demonstrated a multi-robot system for collaboratively transporting objects in a net spanning across all agents. These approaches faced challenges in scaling with the number of agents and had restrictions on the size of the net and topology of team formation.In this work, we focus on the most primitive legged base i.e., bipedal robots. For bipedal robots, Dao et al. [26] investigate the potential of reinforcement learning and sim-to-real transfer for bipedal locomotion under dynamic loads, relying solely on proprioceptive feedback. While they demonstrated effectiveness across a wide range of external forces, their approach faced limitations. Specifically, their method was designed only for a single robot and was constrained by weight, making scalability to multiple robots or handling heavier loads challenging.Building upon these prior works, our proposed decentralized Multi-Biped Controller (decMBC) aims to overcome the limitations of existing approaches by enabling decentralized control and generalization to arbitrary carrier configurations of legged robots. Unlike the works that focus on wheeled robots, our decMBC has the potential to handle the complexity and dynamics of legged robots in challenging terrain. Furthermore, our decentralized architecture enhances the scalability and robustness of the system, making it more practical for real-world deployment. 
problem formulation: 
 We consider multi-biped transport problems where a set of N identical bipedal robots are rigidly attached to the base of a payload carrier. We model the carrier base B ⊂ R 2 as a rigidly connected planar surface with a finite area. Each carrier base is assumed to contain the origin, which is referred to as the carrier control point O C [see Figure 1 (A) ]. An N robot attachment configuration for a base B is a set of points P = {p 1 , . . . , p N }, such that p r ∈ B, which indicates where each robot is attached. In this work, we assume ball joint attachments where each robot has free movements in all three rotational directions relative to the attachment but constrains movements in linear directions. However, our approach is easily adapted to other types of joints and non-planar rigid platform bases.Our goal is to control the height and linear and angular velocity of the carrier control point via the combined motor actions of the robots. To support flexible applications, we consider a decentralized controller architecture that avoids inter-robot communication and is flexible to varying numbers of robots N and configurations P = {p 1 , . . . , p N }. The decentralized Multi-Biped Controller (decMBC) is defined by a single-biped controller π. At each time-step, t, this controller is independently applied to each robot r ∈ {1, . . . , N } to compute robot actions a r t = π(o r t , c t ), where o r t is the local observation and c t is the carrier command.In our work, the command c t ∈ R 4 specifies the target linear xy velocity, angular yaw velocity, and height of the control point. The local observation o r t contains: 1) the local proprioceptive state s r t of the robot, including the joints/motor positions and velocities, base orientation, and angular velocity, 2) the attachment position p r of the robot relative to the control point, and 3) the relative yaw orientation θ r t ∈ R of the robot base with respect to the control point orientation axis. Each of our actions a r t specifies the target PD motor set points of robot r, which includes 10 motors (5 in each leg) for the Cassie biped used in our experiments. Note that π can be applied to any number of robots in any configuration provided that each robot can obtain its relative position p r and orientation θ r t . The challenge is to learn such a decentralized control structure that is effective and robust. 
learning approach: 
 We formulate learning the decMBC as a reinforcement learning (RL) problem, as it involves solving a complex physical control task without supervision for low-level actions. Our approach employs Independent Proximal Policy Optimization (IPPO) [27] , where the shared decMBC controller π is trained using data collected from multiple robots operating in parallel, without inter-robot communication. Below we describe the architecture of decMBC, training environment and curriculum, reward function, and details of IPPO training procedure.Following prior work on bipedal locomotion [28] , we represent the decMBC using a recurrent neural network architecture, which allows for history-dependent action selection via the recurrent internal memory. Specifically, our network consists of two Long Short-Term Memory (LSTM) [29] layers, each containing 64-dimensional hidden states. These layers process the decMBC input at each time step to form the internal hidden state. This state is then processed by a linear layer that produces a 10-dimensional action vector, specifying the 5 motor set points for each of Cassie's legs.Our IPPO training approach uses training data generated by executing training episodes involving one or more robots. During each episode, transitions from each robot are aggregated into a single replay buffer that is used to optimize a single actor and critic using the PPO objective [14] . Details of the PPO hyperparameters are in the Appendix E. The single actor at the end of training is taken to be the decMBC controller. Below, we describe the episode generation process, training configurations, and curriculum used to train the MBC. Episode Generation. At the start of each training episode, we generate a random robot configuration according to the current curriculum stage (see below), including randomized weights on the rigid bars connecting robots. During an episode, the decMBC is used to control the N robots, given a randomized sequence of carrier commands. Each command is executed for a random number of time steps in the interval [100,450]. In addition, depending on the curriculum stage, random perturbation forces (0 to 50 N) are applied to the robot's pelvis and the carrier. Each episode ends after 500 time-steps (10 seconds) or a termination condition that checks whether a robot or the carrier has fallen, which is detailed in the Appendix C. Training Configurations. Our current simulation time scales super-linearly as the number of robots grows. For this reason, we limit the number of robots used during training to N ≤ 3. Empirically, we have found that using training configurations with N ≤ 3 leads to strong performance for N > 3.We generate random configurations for N ∈ {1, 2, 3} as follows: For N = 1, a single Cassie robot is used with a small carrier, where the attachment point is located at the center of the carrier. For N = 2, two Cassie robots are used with their attachment points being endpoints of a rigid bar. The carrier control point is located anywhere within a meter away from either of the robot's positions. For N = 3, an arbitrary triangular frame is formed by connecting three rigid bars. Here, the attachment points for three Cassie robots are the vertices of the triangular frame. In this configuration, the carrier control point can be randomized anywhere enclosed in the triangular frame [Figure 1 (B) ].Further details are given in Appendix A.Training Curriculum. We initialize the decMBC controller and critic randomly and train it via IPPO using a 4 stage curriculum, where each stage is defined by the configurations and disturbances considered. Stage 1 involves training with one robot (N = 1) with the objective of learning robust locomotion gaits operating for carrier commands at its attachment point. Stage 2 involves training with the addition of perturbation force and torsion force on the carrier such that a robot is resistant to the disturbance forces. Stage 3 involves training with the addition of two-robot and three-robot configurations without any perturbation and torsion forces. This stage ensures each robot learns to operate on the carrier control point randomly placed away from its attachment point. Stage 4 involves applying perturbation and torsion forces on the robot's base and carrier in addition to randomizing the mass of the rigid bar connecting the robots. This ensures each robot experiences disturbance forces and varying masses while operating with respect to the carrier control point.Our reward function is designed to produce stable locomotion behavior while ensuring the robots collaboratively follow the commanded motion of the carrier control point. For each robot i in Nrobot configuration, where i ∈ {1, . . . , N }, the reward r i is composed of two components: the local robot reward r L i and global reward r G , where r i = r L i + r G at each timestep throughout the episode. The local reward r L i is designed to produce stable locomotion behavior of each robot, including maintaining consistent base height, penalizing jerky motion, regularizing proper foot alignment and the stepping frequency, with single-contact reward as proposed by van Marum et al. [30] . Furthermore, the local reward minimizes torque of the motors to penalize excessive energy expenditure and minimizes the force exerted at the attachment point at the carrier during standing, ensuring that robots do not pull or push against each other but hold the carrier steady. The global reward r G focuses on the performance of the combined action of all robots in following carrier commands. It aims to minimize the deviation between the commanded and actual x-velocity, y-velocity, angular yaw velocity, and height of the carrier control point. We detail the reward components in Appendix B. 
experiments and results: 
 We conduct experiments to empirically evaluate the decMBC on the Cassie bipedal robot platform. The primary objective is to assess the effectiveness and adaptability to coordinate an arbitrary number of robots, maintain stability, and adapt to diverse transport scenarios. We also discuss the scalability and generalizability of our approach across a range of practical settings including real-world transfer to multi-biped systems with two and three Cassie robots.Experimental Procedure and Metrics. We compute metrics such as drift, failure rate, and power consumption across various configurations to evaluate our decMBC policy. We also measure these metrics to compare our approach against other approaches. For each of these metrics across various carrier configurations, we run 1000 episodes for a maximum of 20 seconds, each with a fixed carrier command such as hold still, move forward, move sideways, and turn in-place. Drift measures the displacement of the carrier in the x/y direction and its orientational drift from its expected orientation and considers episodes that last for at least 1 second. We do not apply any perturbation forces for measuring drift. A negative and positive sign in drift indicates that the carrier is behind or ahead of the expected global position for the desired carrier command. Failure rate is calculated as the percentage of episodes that terminate before reaching 20 seconds. This is evaluated under varying perturbation forces and payload masses fixed on the top of the carrier (except dynamic payload on Table 2 ). We apply ranges of perturbation forces and change payload masses in the carrier to measure the failure rate. Additionally, we calculate the Power consumption by all robots combined for each configuration.Varying Numbers of Robots: We first evaluate the performance of the decMBC for varying numbers of robots. For this purpose, we vary the number of robots from 2 to 10 using the same solid rectangular carrier. The robots are placed in a manner that ensures they are evenly distributed to maintain static stability. This typically involves positioning robots at the corners and, when necessary, placing additional robots in a central position to maintain balance (see Figure 2 ). In order to provide a reference point for strong performance, we evaluate the metrics for a single robot policy (1-R*) trained with its floating base as the control point. We consider achieving multi-biped performance that is comparable to 1-R* as a practical gauge of success.Table 1 gives results for the reference 1-R* and decMBC policies across the various control metrics. We first see that there is minimal performance variation across the range of 2 to 10 robots, showing that the decMBC is generalizing beyond its training scenarios of 1 to 3 robots. One exception is for N = 2 which performs significantly worse on the turn in-place test, though still achieves relatively small errors on other commands. This could be because of compromised stability arising from limited lateral support provided by two robots for a rectangular carrier. However, for N > 2, the maximum orientation drift observed is -7.47°, which is relatively small given the commanded 300°r otation. Overall, we see that the multi-robot performances are generally comparable to 1-R*.Figure 3 presents failure rates for increasing carrier perturbation forces and payload masses as the number of robots varies, along with the power usage. The results show a sharp decrease in failure rate as the number of robots increases, achieving nearly zero failures. The cost for the robustness is increased power consumption. Notably, the power consumption scales approximately linearly with the number of robots, suggesting that the robots cooperate effectively without exerting excessive counter-forces on each other. Varying carrier configuration and payload: In real-world scenarios, payload transport tasks often involve diverse carrier shapes and sizes to accommodate various objects and environments. We demonstrate the adaptability of our approach to such practical settings by evaluating the decMBC's performance on three distinct carrier configurations [see Figure 1 (C)] , each representing a different real-world use case. The first configuration, Sacks, involves four robots attached to a circular carrier with five sacks of varying weights, totaling 35 kg, randomly fixed on top. The second configuration, Log, uses a long plank as a carrier, with three robots attached collinearly, carrying a cylinder that is fixed on the carrier, simulating a log weighing 20 kg. The final configuration, Dynamic, shows four robots attached to a carrier that is a container holding a rolling ball with a total mass of 30 kg.We observe from Table 2 that the controller maintains near-zero drift across all commands and configurations, with the exception of orientation drift during the turn in-place command for dynamic loads. In this case, the drift is -77.95 °, representing an approximately 25% error from the commanded 300 °. This discrepancy may be attributed to the unpredictable motion of the rolling ball, which alters the momentum of the carrier. Nevertheless, given that the decMBC policy has never seen these types of payloads during training, the failure rate seems reasonably low. seconds & mean failure (%) averaged over all commands, and perturbation forces.In this section, we compare our decentralized and adaptable decMBC with two types of straightforward but inflexible systems: centralized policies and specialized policies. The aim is to demonstrate that our approach, despite its flexibility and generalizability, does not significantly compromise performance compared to these more rigid alternatives. A centralized policy is one where all robots' states are used to produce actions for each robot from a single controller. This approach relies on a comprehensive view of the entire system to make coordinated decisions. A specialized policy, on the other hand, is designed for a specific carrier configuration with a fixed number of robots. It is trained to perform optimally under these specific conditions but lacks the flexibility to adapt to different setups.From our observations in Table 3 and Figure 4 , the drift values for our approach are reasonably close to the centralized policy. Interestingly, our approach outperforms the centralized policy for tworobot configuration on failure rate, but the performance worsens on the three-robot configuration.We can also observe that the power consumption of our approach is slightly higher than that of the centralized approach. This difference could be attributed to the centralized controller's ability to produce better coordination due to its full observation of the system state.Interestingly, from the observations in Table 4 and Figure 5 , our decentralized approach outperforms the specialized policy for Rectangle and T-shape configurations in terms of failure rate, with a cost of slightly more power consumption. Our argument for this result is that the decentralized policy, through its adaptive nature, may have generalized to alternative variations of similar configurations, making it more robust to unexpected disturbances.While our decentralized approach shows comparable or better performance in many scenarios, centralized policies theoretically have higher potential capacity due to their access to global information. However, our attempts at training centralized controllers have not yielded substantial improvements, possibly because our decentralized architecture is sufficiently rich for near-optimal performance under rigid attachments, or because the higher dimensionality of the centralized approach makes it more challenging to train effectively. For instance, training a centralized controller for a 3-robot configuration took about 1000 hours to reach peak reward, compared to 245 hours for our decentralized policy. This suggests that while centralized policies may have higher theoretical capacity, our decentralized approach offers a favorable trade-off between performance and training efficiency.Vx=0, Vy=0, ω=0Vx=1m/s, Vy=0, ω=0Vx=0, Vy=0.25m/s, ω=0Turn in-place We evaluated our policy on real hardware consisting of two and three bipedal robots, Cassie, attached at the endpoints of an I-shaped and T-shaped carrier made from aluminum extrusion with a weight holder. We also added a payload on the weight holder and used it as a carrier control point [Figure 1 (D)]. We deployed the decMBC policy on each of the robots, running independently, providing their relative positions with respect to the carrier control point. We use a joystick to broadcast the same carrier command values to all the robots using the UDP protocol. We commanded x-velocity, y-velocity, angular yaw velocity, and height to maneuver the payload from one point to another. We observed that it was successfully able to maneuver different payloads according to various user commands for an extended period of operation. 
limitations: 
 Despite the promising results, there are clear limitations to address in future work. First, the decMBC is not explored on diverse terrain types beyond flat surfaces, eliminating the understanding of its performance in more challenging environments. Second, real-world experiments are demonstrated to up to three Cassie robots, despite the simulation results showing the scalability of up to 10 robots. Third, the controller relies solely on proprioceptive sensing and relative robot positions without incorporating additional sensing modalities like camera perception which could enhance its capabilities in complex environments. Fourth, there are several practical challenges associated with multi-biped systems that require further consideration, including higher costs, increased points of failure, perception difficulties due to robot proximity, and the complexity of deployment and maintenance. These factors highlight important trade-offs between system flexibility and operational practicality. Finally, although our policy is trained with a wide range of relative positions of each robot with respect to the carrier point, which should cover most real-world scenarios, our policy does not guarantee it may work for positions beyond the range we have set during training.Below, we outline the reward terms used in our implementation. The reward components are divided into two categories: Local rewards r L i are designed to produce stable locomotion for each robot, described in the local robot frame. The Global rewards r G are designed for following carrier commands with the combined action of all robots in the control point frame. 6 for the reward description.• N = Number of robots in the system • c t = Carrier command consisting of linear velocity, angular velocity, and base height. c t = (V x , V y , ω, h) • c t = 
 
 Appendix:  


Decision: Strengths:
- First scalable decentralized controller for multi-biped payload transport, showing effectiveness in both simulation and real-world experiments
- Robust generalization to out-of-distribution robot numbers and payload shapes without retraining
- Well-written paper with thorough experimental results and honest discussion of limitations 
- Novel training curriculum design for multi-robot tasks

Weaknesses:
- Experiments limited to flat terrain, misaligned with motivation of bipedal robots for rough terrain
- Lack of detailed ablation studies on multi-stage training process impacts reproducibility
- Limited comparison to other multi-robot control method baselines
- Insufficient discussion of practical real-world applications and advantages over alternatives like large quadrupeds

Reviews: review: title: title: Interesting paper
paper_summary: paper_summary: This paper presents a decentralized controller designed for varying numbers and configurations of rigidly attached bipedal robots, specifically within a payload transport scenario, without necessitating retraining. The authors provide simulation-based quantitative metrics to demonstrate the effectiveness of their approach across a diverse array of transport scenarios. Furthermore, the controller's capabilities are validated in real-world experiments involving systems composed of two and three Cassie robots. Notably, this work represents the first instance of a scalable multi-biped payload transport system.
main_review: main_review: Advantages:

1. This paper introduces the first decentralized controller designed for scalable bipedal robots. The experimental results, both in simulation and with real robots, robustly support the effectiveness of the proposed framework.

2. The paper presents thorough experimental results, demonstrating the framework's capability to 
   a. control multiple robots and generalize effectively to out-of-distribution (OOD) numbers of robots.
   b. The trained policy adapts well to different payload shapes.
   c. Superior performance is evident in both simulated and real-world scenarios.

3. The paper is well-written, with a logical and fluent flow of ideas.

4. The authors clearly and honestly present the limitations of their work, potentially inspiring further research.

Disadvantages:

1. The paper's experiments are confined to flat surfaces. Given the inherent flexibility of bipedal robots, the motivation and experimental scope are somewhat misaligned. Testing on diverse terrain types would better align with the stated advantages of bipedal robots.

2. The framework employs a multi-stage training process, yet the paper lacks detailed ablation studies to elucidate the significance of each training stage. This omission raises concerns about reproducibility. Including such studies would help readers understand how each training method impacts the model's performance and generalization to different shapes and robot numbers.

3. The paper compares the proposed method with only a limited number of baselines. Given the numerous multi-robot control methods available, it would be beneficial for the authors to include more baselines for a comprehensive comparison.
questions: questions: Please refer to the above section.
limitations: limitations: 3
review_summary: review_summary: This paper presents the first decentralized controller designed for varying numbers and configurations of rigidly attached bipedal robots, specifically within a payload transport scenario, without necessitating retraining. The authors provide simulation-based quantitative metrics to demonstrate the effectiveness of their approach across a diverse array of transport scenarios. Furthermore, the controller's capabilities are validated in real-world experiments involving systems composed of two and three Cassie robots. Notably, this work represents the first instance of a scalable multi-biped payload transport system. However, the paper suffers from the absence of ablation study/ insufficient baselines.
title: title: Review
paper_summary: paper_summary: This paper proposes a decentralized controller for a variable number of bipedal robots to collaboratively manipulate payloads, with sim-to-real transfer.
main_review: main_review: Strengths:
- The sim-to-real deployment constrains the multi-agent controller to making practical design choices for deployment, which was not the case in prior works
- The decentralized architecture and training scheme allow the approach to scale to varied numbers of robots and robot configurations without retraining
- Evaluation shows the expected trends of decreasing failure rate, decreasing energy consumption, and increasing payload capacity with number of robots
- Limitations are well addressed
- Overall, evaluations are complete and thorough, and the paper is well-written and novel

Weaknesses:
- It would be worth more discussion on the real-world practical applications of the approach. What kinds of application tasks and scenarios would it be desirable for multiple bipeds to collaboratively manipulate a payload? Why is this better for practical applications than just having e.g. a big quadruped? Since this paper only demonstrates the capability on flat ground, it might be hard for the reader to envision how such systems can evolve in the future, which would potentially make the paper more engaging. There would also be many practical drawbacks to a collaborative bipedal system such as high cost, many points of failure (which is not completely resolved by decentralization, since a biped that has a mechanical failure or runs out of battery still needs to be carried by the others), difficulty of perception with many surrounding robots (which makes decentralization less appealing)
questions: questions: The comparison with centralized and specialized policies seems to mostly suggest that the learning dynamics of the decentralized policy are better than those of the centralized policy or specialist policies. But from a capacity standpoint, it seems like the centralized and specialist policies should have strictly higher potential than the decentralized one, because they have access to more information about the system and task respectively. I wonder what would happen if you pretrained the policies using the decMBC approach before continuing training with a centralized or specialized architecture. Currently the results suggest that decentralization is actually better and it would be great to attribute the cause. Can you discuss the results of section 5.2 a bit and whether I am thinking about this reasonably?

Can you add more discussion of the practical applications as mentioned above?
limitations: limitations: 2
review_summary: review_summary: The paper is well written, thorough, and novel, with high relevance to robotics. It would be enhanced by additional discussion of the results and the practical applications of its problem setting.
title: title: Official Review of Reviewer eduL
paper_summary: paper_summary: This paper designs a decentralized controller for multi-biped robot carriers, which can be applied to varying number of robots and various configurations. The controller is trained through a decentralized multi-agent reinforcement learning method, independent PPO (IPPO). The proposed method are evaluated on extensive settings in simulation, and achieve good performance with regards to drift, failure rate, power consumption for payload transport, reaching flexible and robust policy compared to baselines. The proposed framework is also validated on real robots.
main_review: main_review: # Strengths
* The paper is well-written and easy to follow.
* The method demonstrates robust policy when deployed on real robots.
* The framework has potential to work with unstructured and challenging terrains.
* The nature of decentralized control make it possible to be applied to varying number robots that are attached to a carrier in arbitrary stable configurations. The scalability is validated by sufficient experiments.
* The training curriculum design is unique and interesting for mutli-robot tasks. 
* Extensive experiments with various configurations and varying numbers of robots are provided in simulation.
* The comparisons between decentralized multi-robot policy to centralized policy is valuable.

# Weaknesses
* The paper does not provide comparisons with previous control-based methods for multi-robot payload transport. Adding discussion or baselines in the experiments would be beneficial.
* Training is performed and validated with no more than 3 robots.
* The presented framework relies on fixed and stable configurations to carry on payloads.
* The tested experiments are on the flat ground, not including more challenging terrains.
* The authors directly apply the existing RL framework (IPPO) without major revisions.
* No quantitative and baseline results are provided for real robots.
questions: questions: * It looks like the the robots are distributed with symmetry under payloads as the number of agents increases in Figure 2. Would the robustness of the policy be affected when this symmetry is disturbed?
* Another potential benefit of applying decentralized multi-agent policy to robotic scenarios is better sim-to-real transfer, as indicated in the following work. It would be beneficial to discuss this in the results or consider it in the future work.
    * Dynamic Handover: Throw and Catch with Bimanual Hands. Huang et al., CoRL 2023.
limitations: limitations: 3
review_summary: review_summary: This paper presents a solid work on multi-biped payload transport with multi-agent reinforcement learning. Extensive experiments are implemented to show its effectiveness in both simulation and on real robots.