MBC: Multi-Brain Collaborative Control for Quadruped Robots

Authors: hang  liu
yi  cheng
rankun  li
xiaowen  hu
linqi  ye
houde  liu
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-07 13:31:27

Abstract: In the field of locomotion task of quadruped robots, Blind Policy and Perceptive Policy each have their own advantages and limitations. The Blind Policy relies on preset sensor information and algorithms, suitable for known and structured environments, but it lacks adaptability in complex or unknown environments. The Perceptive Policy uses visual sensors to obtain detailed environmental information, allowing it to adapt to complex terrains, but its effectiveness is limited under occluded conditions, especially when perception fails. Unlike the Blind Policy, the Perceptive Policy is not as robust under these conditions. To address these challenges, we propose a MBC:Multi-Brain collaborative system that incorporates the concepts of Multi-Agent Reinforcement Learning and introduces collaboration between the Blind Policy and the Perceptive Policy. By applying this multi-policy collaborative model to a quadruped robot, the robot can maintain stable locomotion even when the perceptual system is impaired or observational data is incomplete. Our simulations and real-world experiments demonstrate that this system significantly improves the robot's passability and robustness against perception failures in complex environments, validating the effectiveness of multi-policy collaboration in enhancing robotic motion performance.

Main paper: 
 introduction: 
 What happens if a robot suddenly loses its perception? Can it maintain its previous stable motion? In natural environments, there are instances of sensory failure in humans, such as "dark adaptation" phenomenon, but humans and animals can rely on past experiences to immediately switch to a state of motion without sensory input, ensuring safe movement. For humans, this ability stems from two main sources. First, the human brain has a strong adaptive and memory capacity. Secondly, the human control system has a high degree of redundancy and multisensory integration. For example, when vision fails, the proprioceptive and vestibular systems enhance their role in maintaining balance movement.In the motion tasks of bipedal and quadrupedal robots, sensory systems may fail due to incomplete information or hardware malfunctions. These robots rely on various sensors to gather environmental data, such as LiDAR, cameras, and ultrasonic sensors. However, the effectiveness of these sensors can be limited in some circumstances. Therefore, researching how to maintain stable robot motion under these unfavorable conditions is a challenge in current studies.In locomotion tasks, blind policies and perceptive policies each have their advantages and limitations [1] . Blind policies rely on sensors and preset algorithms for movement, requiring no visual input [2, 3, 4, 5] . Although they are fast and consume fewer resources, their adaptability in complex or unknown environments is limited. Perceptive policies enable robots to adapt to complex terrains [6, 7, 8] . However, in less than ideal visual conditions, perceptive policies may not be as efficient as blind policies. Researching how to effectively merge these two policies to cope with complex environments is an equally challenging research issue.Addressing the challenges mentioned, this study integrates MARL [9, 10] to propose the concept of MBC: Multi-Brain Game Collaboration controller. We envision a quadruped robot system integrating multiple policies to form a collective "brain" with each policy tailored to different input policies. Specifically, we explore the interaction between a Blind Policy, and a Perceptive Policy that utilizes external information. This model excels in scenarios with incomplete observational data or impaired sensory capabilities, accurately simulating and analyzing the robot's interactions with its environment. This approach enhances decision-making and adaptability in complex environments.The primary contributions of this research are as follows:• A Novel Multi-Brain Game Collaboration System(MBC): This study introduces and successfully implements a multi-brain game collaboration system using MARL. In this system, each policy or "brain" independently and collaboratively optimizes decisions for different tasks.• Perception "Hot Swap": The research realizes "Hot Swaping" of external perception in quadruped robots control. Experiments in both simulation and real world have proven that this method can keep robust locomotion when sudden failure of external perception.• Enhanced Mobility in Complex Environments: Through the non-zero-sum game [11] between blind and perceptive policies, this policy allows the robot to make accurate and effective motion decisions. 
related work: 
 Multi-Agent Reinforcement Learning In the field of Multi-Agent Reinforcement Learning (MARL), there are generally three learning paradigms: centralized learning, independent learning, and Centralized Training with Decentralized Execution (CTDE) [12] . Among these, CTDE effectively combines the advantages of centralized learning with the flexibility of decentralized execution.MADDPG [13] is a typical representative of the CTDE paradigm, employing an actor-critic framework. However, as an off-policy algorithm, MADDPG requires extensive memory storage to save previous experiences and may not perform as stably in dynamic environments as on-policy algorithms. MATD3 [14] , a multi-agent version of TD3, enhances the stability of multi-agent coop-eration through double Q-learning and delayed policy updates, but this also increases computational complexity, especially in large-scale multi-agent environments, and is extremely sensitive to hyperparameters, which may require extensive tuning and experimentation in practical applications to achieve optimal performance. MAPPO [15] , for the first time, effectively extends the single-agent PPO algorithm to a multiagent environment, becoming an on-policy strategy that can handle complex multi-agent collaborations while maintaining the stability and efficiency of policy updates. MAPPO not only retains the advantages of PPO but also successfully addresses the collaboration problems in multi-agent environments. Its application on the SMAC platform demonstrates its high sample efficiency and consistency of policies [16] .In locomotion task of quadruped robots in complex environments, current research explores three primary policies. The first policy, termed the blind policy, relies on the robot's historical proprioception, primarily utilizing forelimb probing, to estimate terrain [3, 5, 2] . This policy faces limitations in complex environments like cliff due to perception limitation. The second policy employs external sensory inputs to gather environmental details, helping the robot plan movements and navigate complex terrains [17, 18, 19, 20, 21] . However, this often involves isolated end-to-end network architectures without testing for sensor reliability. The third, a composite policy [22, 23] integrates blind and visual policies into a synergistic mechanism, quickly adapting to sudden failures in external perception systems.In sim2real applications for vision-based motion controllers using reinforcement learning, two main approaches are prevalent: end-to-end training with depth or RGB images [21, 24] , effective in quadrupedal robots, and using elevation maps [25, 26] or height scans from a Global Reference Frame. However, the former always faces the trouble of the long training time in simulation, camera noise, camera distortion, which introduce sim2real gap.We introduce a novel framework that combining MAPPO in multi-brain system for quadruped controll. Our approach can accurately simulate and analyze the complex interactions between the robot and the environment, even under conditions of incomplete observation data or sensory loss, thereby enhancing the robot's motion performance in various environments.  
task formulation: 
 In the locomotion task of a quadruped robot, we define a process that combines a blind policy and an external perception-based policy to handle complex environments. Specifically, the quadruped robot can flexibly navigate various obstacles such as highlands, gaps, obstacles, and stairs when external perception (e.g., LiDAR elevation maps) is functioning properly. However, when external perception suddenly fails, the quadruped robot, although unable to navigate terrains such as gaps, should still retain the capability to traverse complex terrains like stairs and ramps.We have designed a two-stage training approach, as illustrated in Figure 2 . In the first stage, a training mode without external perception is used, involving only a blind policy. In the second stage, a multi-agent approach is employed, incorporating external perception and simultaneously training both the blind policy and a perceptive policy with perception capabilities. The collaboration between these two policies is guided by a terrain reconstruction error regularization term. This ensures that our robot can effectively traverse terrains both with and without perception.Theorem We describe the locomotion problem of quadruped robots using a Partially Observable Markov Decision Process (POMDP) [27, 28] .The POMDP framework effectively models decisionmaking scenarios where information is incomplete, defining key elements such as states, actions, observations, and rewards. In this model, the environment at time step t is represented by a complete state x t . Based on the agent's policy, an action a t is performed, resulting in a state transition to x t+1 with a probability P (x t+1 | x t , a t ). The agent then receives a reward r t and a partial observation o t+1 . The aim of reinforcement learning here is to identify a policy π that maximizes the expected discounted sum of future rewards:J(π) = E π ∞ t=0 γ t r t Action Space & State SpaceThe action spaces for the blind policy and perceptive policy are respectively a blind t ∈ R 12 and a percep t ∈ R 12 , representing the offset from the default position for each joint. The critic networks for both policies observe the global states critic t = [o t , v t , e t , h t , a percep t , a blind t] T , which includes proprioceptive observations o t , linear velocities v t , height map h t and latent variables e t such as body mass, center of mass position, friction coefficients, and motor strength. These global observations are crucial for the second phase of training, helping the critic network make balanced decisions during the interactions between the two policies and preventing training collapse due to excessive competition. For the actor networks, the state space for the blind policy includes proprioceptive observations o t , estimated linear velocity vt , and latent variables e t . Additionally, aligning with the multi-agent game theory approach, the state space for the blind policy also incorporates the output from the Perceptive Policy a percep t , expressed as s blind t = [o t , vt , e t , a percep t ] T . Similarly, the state space for the Perceptive Policy iss percep t = [o t , h t , a blind t ] T, where h t represents the local elevation map centered around the robot.During the training for the Blind policy, we employed the Regularized Online Adaptation (ROA) method [29] to estimate the explicit observations vt and the latent variables e t . In this phase, a percep t was set to zero. In the second phase of training, the final action a t = a percep t + a blind t .In the first stage of training, we primarily developed a proprioceptive motion system for the quadruped robot, aimed at enabling the robot to traverse various complex terrains such as uneven slopes, stairs, and discrete terrains without direct visual or elevation map input to the policy. During this phase, the output action of perceptive policy a percep t was set to zero, ensuring that the blind policy operates without interference from perceptive policy' outputs. Our blind policy, inspired by the ROA [29] , uses history proprioceptive inputs to estimate the robot's explicit privileged information and implicit environment and dynamic information. Additionally, the training utilized an asymmetric Actor-Critic structure to better evaluate the quality of the actions output by the Actor.For the robot's elevation map, we trained a Variational Autoencoder (VAE) model primarily to memorize the terrains encountered by the blind policy and to compute regularization terms for action constraints in the subsequent training phase.In the second stage of learning, we introduced a multi-agent learning approach, utilizing a Non-zerosum game thinking and MAPPO [15] to optimize the external perception controllers for quadruped robots. Unlike traditional single-policy approaches such as parkour [21] , this method allows for adaptation when one controller fails, as other controllers can detect and adjust their actions, enhancing the system's robustness. Additionally, this model supports "hot-swapping" of the perception system, enabling the robot to move based on sensory data when available and to continue proprioceptive movement without malfunction when perception is unexpectedly lost.The primary implementation policy is as follows: As shown in Figure 2 , we first load the pre-trained model of the single-agent blind policy as a blind agent, and we initialize the perceptive policy as a perceptive agent. Inputs to the perceptive policy include proprioceptive data, outputs from the blind policy, and elevation map information, primarily adjusted for terrain. The robot's final actions are a combination of perceptive and blind actions. In the second stage, challenging terrains such as gaps, pillars, and highlands were introduced, which are difficult for the quadruped to traverse without external observation. The robots were encouraged to traverse these difficult terrain without collision. Specifically, although the terrains vary, We established a general reward structure as shown in 4. The robots will not be rewarded if they fail to follow the desired heading, ensuring they traverse the terrain instead of avoiding it. Additionally, the weights of the collision penalty were set high to encourage the robots to rely on external observations in these challenging terrains rather than solely on proprioception. This framework ensures that during training, the perceptive and blind policies interact and collaborate to optimize movement. All networks use the CTDE approach with MAPPO [15] updates, where each agent's Critic network shares all environmental information, including the inputs and outputs of other agents, during training, while each operates independently during execution. The loss calculations and updates for the blind policy remain as in the first phase, while the perceptive policy's loss includes surrogate loss, value loss, entropy loss, and a Reconstruction Error Regularizer, which will be explained in detail in 3.5. In the first stage, we trained the quadruped robot to navigate rough terrains without relying on external perception. These terrains were chosen because they enable the robot to learn fundamental locomotion skills and to lift its legs and react to tripping, thereby enhancing overall mobility. We believe these terrains exemplify the types of environments a robot can navigate without perception in real-world scenarios. We employed a Variational Autoencoder (VAE) to encode and decode these features. We evaluate the VAE reconstruction error on the first stage terrain after training and obtain the maximum reconstruction error τ during testing. The VAE was trained once in first stage and frozen in second stage.In the second stage, we introduced more challenging terrains, such as highlands, gaps, and dense pillars, as shown in Figure 2 , which are difficult for the robot to navigate using only the Blind Policy trained in the first stage. In particular, the highlands require the robots to jump up and down, the gaps necessitate learning to jump over them, and the pillars demand that the robots avoid them before returning to their original direction. Therefore, it must rely on the Perceptive policy with external perception input for compensation.However, the complexity introduced by Multi-Agent Learning can lead to policies converging to local optima, with the blind policy and Perceptive policy potentially competing against each other, hindering coordinated control. To address this, we introduced a perception cooperation constraint regularization term based on elevation maps. This term helps ensure that if the current elevation map reconstruction error, as produced by the VAE, is below a threshold τ , indicating familiarity with the terrain, the regularization term increases with the Percep policy's output, limiting its action. If the reconstruction error exceeds the threshold τ , indicating unfamiliar terrain, the regularization term is set to zero, encouraging the Perceptive policy to compensate. Specifically, in the second stage, the robot's current elevation map h ij is input into the VAE, which reconstructs the elevation map ĥij . The reconstruction error is then calculated as 2 , where i represents the i-th sample in the batch, j represents the index of the dimensions of the elevation map and action, and n represents the dimension of the elevation map. Based on the reconstruction error and the threshold, we define the penalty factor:E i = 1 n n j=1 ( ĥij -h ij )I i = 0 if E i > τ 1 if E i ≤ τThis means that when the reconstruction error exceeds the threshold, the regularization term is set to 1, otherwise it is 0. The perception cooperation constraint regularization term is then introduced as:P i = 1 m m i=1 I i k j=1 a 2 ijwhere k represents the dimension of the action, and m represents the batch size. Finally, the total loss function consists of the surrogate loss, value function loss, policy entropy, and the action regularization term:L = L surrogate + λ v L value -λ e H(π) + λ a P i 4 Experimental Results 
experiment setup: 
 We used the Unitree Go2 robot as our experimental subject. The training terrain comprised six types: ramps, stairs, discrete obstacles, highlands, gaps, and pillar terrain, as shown in Figure 2 . Additional environment setup information and training specifics are detailed in the appendix. The purpose of our experiment is to prove whether our method can achieve the following behaviors:• Can the robot successfully navigate highlands, gaps, pillar with perceptual input?• Can the robot adapt and successfully navigate the first three types of terrain without any mode collapse when perceptual input suddenly fails?Terrain Passability Experiment: We first tested the survival rate of our policy across last three tough terrains with varying levels of difficulty. For each terrain and difficulty level, we conducted four trials, with each trial consisting of 100 environment samples. We calculated the success rate for each trial and averaged these four success rates to obtain the final experimental result. The success rate for the Gap and Pit terrains was defined as the robot successfully crossing or climbing over the obstacle, while for the Pillar terrain, it was defined as the proportion of environments the robot navigated without collisions. As shown in Table 1 , our policy achieved high success rates across various tough terrains. The highest difficulty level for each terrain was beyond the scope of our curriculum settings, demonstrating the robustness of our algorithm.Comparison Experiment: We evaluated the robot's ability to traverse complex terrains under perception failure conditions and compared methods with several baselines and ablations as follows: • RMA [30] : Employing an Adaptation Module to estimate all privileged observations, but directly inputting the elevation map into proprioception. • MLith [20] : Utilizing a GRU neural network as the Actor, with proprioceptive and exteroceptive inputs fed directly into the GRU. • Dual-History [31] : Utilizing a dual-history structure, where short-term and long-term history observation are processed separately. As shown in Table 2 , comparing with other methods, our method demonstrates the most robust performance under external perception failure, especially when climbing stairs. Other strategies failed to handle obstacles without perception, resulting in tripping over obstacles. In contrast, our method can easily climb steps, and the MXD indicates that our method can also follow the high desired speed (1 m/s). Figure 3 . shows the effect of our run in simulation. We believe that the difference in the success rate of going up and down stairs comes from the fact that going down stairs only requires considering the policy's ability to maintain balance, as gravity will guide the robot down, while going up stairs requires the robot to sense obstacles and determine the implicit type of obstacles in order to present a regular foot lift height.We noticed that with the regularizer, the robots performed better when going upstairs and overall in MXD. This improvement is due to the regularizer guiding the collaboration between the blind policy and the perceptive policy across different terrains during training. Table 2 : we primarily compared the success rates of different methods on stairs and discrete terrains, as well as the Mean X-Displacement (MXD) for each environment. For this experiment, all elevation map inputs were set to zero, desired x velocity was set to 1m/s, and we tested 1048 environments over 1000 steps. The stairs had a width of 0.31 and a height of 0.13, while the maximum height of the discrete terrain was 0.15. Failure conditions were defined as either the roll or pitch exceeding 1.3, or the robot's foot getting stuck and unable to move forward. The optimal value for MXD is expected to be 20 meters.Navigating Complex Terrains with Sensory Input Our policy substantially enhanced the quadruped robot's capability to navigate vertical challenges, such as wooden boxes and low walls.In our experiments, the robot was tasked with climbing a 32 cm high wooden box. It adeptly lifted its front legs preemptively and elevated its body to surmount the box, as shown in Figure 4 . This sequence of movements, successfully culminating in the robot climbing over the box, exemplifies the efficacy of our integrated elevation map and perceptual policies in enabling the robot to tackle climbing obstacles.In the obstacle avoidance trials, corresponding to the Pillar terrain used during training, the robot encountered various obstacles including trees, pillar, or human figures. Leveraging our policy, it quickly recognized a human-shaped obstacle through its elevation map, then adeptly adjusted its trajectory, sidestepping to bypass the obstacle efficiently and safely, as depicted in Figure 5 . This performance underscores our method's effectiveness, particularly noting that despite the absence of y-direction velocity training, the robot adeptly maneuvered in the y-direction, showcasing the robustness and adaptability of our approach. Long-Distance Test with Outdoor Terrain Perception Failure Initially, with effective LiDAR elevation map inputs, the robot used a comprehensive policy for movement, efficiently climbing 16 cm stairs and handling slopes. Subsequently, we deliberately covered the LiDAR, disabling the elevation map input, and conducted a long-distance test on unstructured terrains. We tested the robot over a 250m path that included grass, irregular terrain, soft and slippery grasslands, gentle slopes, and stair terrains, where the robot successfully navigated through all (see Figure 1 ). Quantitative Experiments We conducted quantitative experiments with the physical robot in both perceptive and nonperceptive scenarios. With Lidar input case, we tested the performance of climbing and descent on a highland, avoiding pillar-like terrain. For the scenario Lidar fails, The experiment includes stairs, slopes and discrete obstacle. Compared to the perceptive scenario, we defined a more stringent metric, the No-Snagged Success Rate(NS-Success Rate), in which the quadruped robot successfully traverses the terrain without noticeable tripping or stalling. Each terrain was tested ten times.During the experiments, we used a hood to cover the Lidar to simulate perception failure. As shown in Table 3 , the robot can not only traverse challenging terrain using LiDAR input, but can also maintain smooth movement in difficult terrain in the event of a sudden LiDAR failure. Please refer to supplementary for experiment videos and detailed experiments setup. 
conclusion, limitations and future directions: 
 We propose the concept of Multi-Brain Collaborative Control(MBC) based on MARL, establishing a training framework that achieves both perceptive motion and robust locomotion when perception fail. We tested our system in both simulations and real-world experiments, demonstrating the effectiveness and robustness of our algorithm. However, our robot's elevation maps are derived from LiDAR, which heavily depends on the frequency and stability of the odometry, and involves significant computational overhead. The laser radar odometer has limited accuracy in high-speed, high-frequency, and vibration-prone scenarios, which results in a lower success rate in our physical experiments than in simulation experiments in passing through challenging terrain, as shown in Table 3 . In the future, we aim to use end-to-end method to construct local elevation maps without relying on odometry [22][17] . We will also explore how to apply our algorithm to control various legged robots.  
 
 Appendix: We tested our controller across various outdoor terrains, which included actions such as climbing and dodging in complex terrains using perception, as well as navigating through grass, slopes, soft soil, and steps in cases where perception suddenly failed, as illustrated in Figure 9 and based on methodologies described by Li et al. [31] . We used the reward function as shown in Table 4 , where the Task reward guides the robot to track the desired speed and complete motions on various terrains. Our setting for the regularization reward refers to Long et al. [32] ; Kumar et al. [30] ;Agarwal et al. [20] ; Cheng et al. [4] . Through extensive training trials, we optimized our reward weight settings to ensure that the robot moves in a relatively ideal manner. i=1 (q out i , q i > q max ∨ q i < q min ) -13.0Environment detail: We used the Unitree Go2 robot as our experimental subject, which features 12 degrees of freedom in its legs. Utilizing a single NVIDIA RTX 4090 GPU, we simultaneously trained 4096 domain-randomized Go2 robot environments in Isaac Gym. During training, we employed PD position controllers for each joint, with both the Blind Policy and Perceptive Policy running at a frequency of 50 Hz. The elevation map update rate was set to 10 Hz, and the robot's control signal delay was 20 ms.Robot Domain Randomizations: During the training process, we utilized the following domain randomization parameters to enhance the robustness of our policy. The range of randomization was referenced from Long et al. [32] ; Wu et al. [33] . In actual robots, factors such as communication delays can lead to action execution delays of approximately 20ms. Therefore, domain randomization of action delays during robot training significantly improved the real-world performance of the robots.Heightmaps Domain Randomizations: We utilize the 'Fast lio' odometer [34] and the method from P. Fankhauser and M. Hutter's [25] to construct the elevation map. Due to inherent random errors typically associated with laser odometry in practical deployments, we have implemented domain randomization for both the elevation map and the z-axis height of the robot's base.• Phase One: Blind Policy Training  


Decision: Summary: This paper proposes a Multi-Brain collaborative system combining Blind and Perceptive Policies using Multi-Agent Reinforcement Learning for quadruped robots, enhancing stability and robustness in complex environments and perception failure conditions, as demonstrated by simulations and real-world experiments.

Strengths:
* Introduces a novel Multi-Brain collaborative system combining Blind and Perceptive policies using Multi-Agent Reinforcement Learning (MARL).
* Demonstrates the effectiveness of the proposed system through both simulation and real-world experiments.
* Shows significant improvements in stability and adaptability of quadruped robots in complex environments, even under perception failures.

Weakness:
* Some aspects of the methodology, particularly the training and inference processes, need better explanation.
* The real-world deployment lacks quantitative results, with some experiments only described visually without detailed findings.
* Some variables and experimental setups are not clearly defined, and the comparison experiments could be more comprehensive.
* While limitations are discussed, the paper does not explore their impact on results or propose potential solutions in detail.

---- Post rebuttal:

Post rebuttal: The paper received two accept and one reject. The reviewer who recommended reject did not further engage with the rebuttal. The AC carefully went throughs the rebuttal and revised paper and discussion and agreed with the majority of the reviewers to accept the paper.

Reviews: review: title: title: Valid paper
paper_summary: paper_summary: The paper presents a novel approach to robot locomotion learning that leverages Multi-Agent Reinforcement Learning to combine Blind and Perceptive policies. To this end, the authors develop a methodology to train a Multi-Brain collaborative system in two phases, where the first is dedicated to training the Blind policy, and the second trains both the Blind and Perceptive policies in a Multi-Agent collaborative game. The authors report experiments both in simulated and real-world experiments, demonstrating that this methodology allows to correctly switch to the correct type of policy depending on the scenario, surpassing the performance of other baseline methods.
main_review: main_review: The paper is well presented and provides technically strong and comprehensive content, that is highly relevant to robot learning. In particular, its strengths are:
- originality of the approach in combining Blind and Perceptive policies with multi-agent reinforcement learning
- experiments are done both in simulation and in the real-world
- very clear presentation, especially of the methodology and experimental results

However, there is a minor weakness worth noting. While the paper explicitly discusses limitations in the final section, it falls short in addressing the impact of these limitations and proposing potential solutions. A more detailed exploration of how these limitations affect the results, along with suggested directions for future research to mitigate these issues, would strengthen the paper further.

There are also a couple of typos the authors may want to fix:
- 70 missing period
- 167 (possibly) a missing citation
- 185 percep -> perception
- figure 3 caption: missing period
questions: questions: Aside from fixing the very few typos, the only significant improvement that could be made is a slightly more detailed discussion of the limitations (possible solutions, how impactful these limitations are for the end performance).
limitations: limitations: 2
review_summary: review_summary: I recommend acceptance, since the paper presents an original, empirically validated and relevant approach to locomotion in robot learning, and contains relevant real-world experiments.
title: title: This paper presents a solid contribution to the field, with high relevance and interesting, exhaustive results and experiments. Provided my somewhat minor comments are addressed, I believe this paper is ready to be accepted.
paper_summary: paper_summary: The paper proposes a novel multi-brain collaborative control system for quadruped robots, combining blind and perceptive policies using multi-agent reinforcement learning (MARL). This approach aims to enhance the robot's locomotion stability and adaptability in complex environments, even under conditions of perception failure. The paper presents both simulation and real-world experiments to validate the effectiveness of this system.
main_review: main_review: I really liked the idea of using MARL for multi-brain collaborative control that effectively addresses the limitations of both blind and perceptive policies individually. This framework enhances mobility in complex environments, even when sensors fail. The authors provide a detailed explanation of the task formulation, training stages, and the incorporation of VAE for terrain memory and perception cooperation constraints. Both simulation and real-world experiments are conducted, demonstrating the system's effectiveness in navigating complex terrains and handling perception failures.

There are also a few weaknesses in the paper. First, the paper could benefit from improved clarity and organization. Some sections are dense and may be difficult for readers to follow, particularly the technical details in the methodology (sections 3.3, 3.4, 3.5). Secondly, while the real-world experiments are a strong aspect, the variety of conditions could be expanded to further validate the system's robustness (such as proprioceptive sensory failure). Lastly, while the paper acknowledges the computational overhead and sensitivity to environmental noise as limitations, it does not provide a clear roadmap for addressing these issues in future work.

Furthermore, I have some small comments:
1.	Some variables are not clarified the first time they appear, such as h_t in line 136.
2.	For the obstacle avoidance experiments, the authors mention that the policy recognized a human-shaped obstacle and avoided it. However, it is not mentioned in the training process. It would be clearer if the authors could specify whether the robot was trained to do this or if it was a generalized behavior.
3.	Only the exteroceptive sensory failures are introduced in the experiment. I am curious about the proprioceptive sensory failures. Furthermore, the sensor delay is also an interesting point to investigate.
questions: questions: 1. In the comparison experiments, the RMA is trained with real elevation map input; however, in the experiment, the input of the elevation map is all zeros. Why not try to avoid using the elevation map as input for the RMA policy (blind) or add a large domain randomization (large noise) to the elevation map input for a fair comparison? And why does training with and without Perception Cooperation Constraint Regularization have almost the same performance in the experiments?

2. How do you define the threshold for the reconstruction error?

3. In the obstacle avoidance experiment,was the robot trained to do this or was it a generalized behavior?
limitations: limitations: 2
review_summary: review_summary: This paper presents a solid contribution to the field, with high relevance and interesting, exhaustive results and experiments. Provided my somewhat minor comments are addressed, I recommend this paper be accepted.
title: title: Review
paper_summary: paper_summary: This paper explores getting the best of both perceptive and blind locomotion and trying to obtain a controller that is robust. It uses a Multi-agent approach (MAPPO - multi-agent PPO) to achieve this by treating the two policies (blind/perceptive) as two different agents.
main_review: main_review: Strengths:
- The motivation of this paper is good and the overall method seems reasonable.
- novel application of multi-agent RL to robotics and control of just a single robot system.
- This work shows evidence of performance deployed on real-systems in a nice demo video - which is always very good and welcomed for a robotics conference.

Weaknesses:
- the writing and explanation of the methods can be significantly improved. There are many parts that are confusing and difficult to understand how this method works between the training stages and during inference, and what the inputs of each policy are at each stage. I think this can be clarified and explained better. (see questions for the main weaknesses)
- the real-world deployment and video is nice but there is no evidence of evaluation in the real-world, results only seem to be purely visual.
- Are there no results/quantitiative results of the “Long-Distance Test with Outdoor Terrain Perception Failure” experiment? Seems to just be a descprition with not much fidings and reuslts other than the first figure.

Typos: - Line:147-148: Rapid Online Adaptation (ROA) - should be Rapid Motor Adaptation (RMA)
questions: questions: Questions (Unclear things): 
- The description of the actor networks are confusing. During inference especially, how is it done? The blind policy need the action from the perceptive policy while the perceptive policy needs the action from the blind policy? This isn’t very clear.
- How is the threshold value for the reconstruction computed? Manually set? How does one set this value and how is it decided? 
- Line 246: what does directly inputting the elevation map into proprioception mean? This is confusing.
- Experiments: how did you control the policy (what commands were given) in the obstacle settings? How are these standardises across runs and baselines?
- Computing the success rate 4 times is very confusing. I think what is meant here is not 4 independent runs  of 100 samples are done. Runs/replications would be more suitable here.
- What do the pillar and gap environments look like? Images of this would be good.
- Why is there no blind policy baseline? Would be nice to have a blind policy baseline as well (as another sort of ablation)
limitations: limitations: 2
review_summary: review_summary: The description of the methods can be significantly improved and clarified and should be done. Greater detail on experimental methods is also required to justify results.