SLR: Learning Quadruped Locomotion without Privileged Information

Authors: shiyi  chen
zeyu  wan
shiyang  yan
chun  zhang
weiyi  zhang
qiang  li
debing  zhang
fasih  ud
din  farrukh
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-05 18:03:01

Abstract: Traditional reinforcement learning control for quadruped robots often relies on privileged information, demanding meticulous selection and precise estimation, thereby imposing constraints on the development process. This work proposes a Self-learning Latent Representation (SLR) method, which achieves high-performance control policy learning without the need for privileged information. To enhance the credibility of our proposed method's evaluation, SLR is compared with open-source code repositories of state-of-the-art algorithms, retaining the original authors' configuration parameters. Across four repositories, SLR consistently outperforms the reference results. Ultimately, the trained policy and encoder empower the quadruped robot to navigate steps, climb stairs, ascend rocks, and traverse various challenging terrains.

Main paper: 
 introduction: 
 Humans and animals inherently possess locomotion abilities, enabling them to traverse various complex terrains. In contrast, gait control for robots is highly challenging. Model-based methods have achieved some success by leveraging robots' mechanical structures and dynamic principles [1, 2, 3, 4, 5] . However, finding a balance between model accuracy and computational efficiency remains difficult, especially for real-time applications.Additionally, designing these models requires a deep understanding of a robot dynamics, posing a significant challenge for researchers. As a result, Reinforcement Learning (RL) methods are becoming increasingly popular. By simulating real-world environments and training policies with customized reward functions，these methods enable robots to perform complex locomotion tasks in real-time [6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18] .The recent mainstream RL applications in quadruped robots rely on privileged learning methods [19] . In real-world scenarios, a robot's interaction with its environment is modeled as a Partially Observable Markov Decision Process (POMDP). Solely relying on proprioceptive sensor measurements, a robot cannot fully perceive external environmental information, limiting its decision-making capabilities. Consequently, many studies leverage the "observability" advantages of simulation platforms. During training, various physical parameters (such as friction coefficients [10, 20, 21] , restitution coefficients [9, 21] , and scan-dots of the terrain [14, 22] ) are artificially added as privileged information to help the robot understand both itself and the external environment.Unlike human cognition, which navigates terrains without explicit knowledge of physical parameters, neural network-based robots may not benefit from adding such parameters. This is because the parameters are not used to solve dynamic equations but are simply input into the neural network. As a result, these privileged information pieces may not enhance the interpretability or effectiveness of neural network-based agents.Therefore, instead of relying on manually chosen physical parameters to construct privileged information, this work explores whether it is possible for robots to learn a latent representation of environmental states by themselves? It is proposed that intelligent robots should be capable of selflearning latent representations, which are inherently more suitable than those predefined by humans.To achieve this, we propose the Self-learning Latent Representation (SLR) algorithm, which generates latent representations guided by a RL Markov process without privileged information. This self-learning approach enables the robot to grasp latent environmental features and exhibit generalized locomotion capabilities across challenging terrain, as illustrated in Figure 1 .Our results show that the SLR algorithm, which operates without privileged information, outperforms traditional privileged learning methods. It consistently aligns with actual terrain conditions across various terrains. Implemented in existing open-source repositories and evaluated in the same environments as previous studies, the SLR algorithm demonstrates state-of-the-art (SOTA) performance in both simulations and real-world applications. This suggests that the self-learning approach has considerable potential for future expansion. 
related work: 
 Privileged learning in RL-based methods can be divided into explicit estimation and implicit estimation methods based on the target of supervised learning. In this paper, directly fitting specific physical parameters from privileged information is referred to as explicit estimation, while fitting the latent representation of privileged information is classified as implicit estimation.Explicit Estimation. [6] concurrently trains a policy network and a state estimator, which include real-world parameters that are difficult to obtain accurately, such as linear velocity, foot height, and contact probability. [13] sets friction coefficients and stiffness coefficients as privileged information, inferring these from observation history to assist the robot in domain randomizations. But in the real world, the quadruped robot's foot contact time is very short during fast running, making it difficult to fully perceive ground friction. Therefore, [12] proposes learning information-gathering behaviors by adding an active estimation reward, which increases the accuracy of estimates for privileged information disparities. Implicit Estimation. To train quadruped robots on complex terrain, estimating a large amount of privileged information explicitly is challenging. A common approach is to encode this highdimensional information into a latent representation. [10] utilizes a teacher encoder to compress data such as friction and terrain height into a latent representation, and then a student adaptation module learn to infer this latent representation from observation history. [11] optimized the teacher-student training process from [10] in one stage by regularizing the teacher's actual privileged latent and supervising the student's estimated privileged latent. The algorithm of [9] also includes a teacherstudent policy. During training, the teacher's encoder is used, and the student's adaptation module output aligns with the teacher's encoder. For deployment, the student's adaptation module and the teacher's trained policy are utilized together. [8] employs the Asymmetric Actor-Critic (AAC) [7] method, feeding privileged information to the critic network while using encoder-decoder estimators to assist the actor in imagining this privileged information. 
problem formulation: 
 Our goal is to construct a one-stage end-to-end system based on RL, using proprioceptive sensor data as input for measuring and controlling joint movements.Observation Space. The observations o t consist of 45 dimensions, including the robot's base angular velocity ω, commands from the joystickc t =[v cmd x ,v cmd x ,ω cmd yaw ], measurement of the gravity vector g t , joint positions θ and velocities θ, and the actions a t-1 taken by the robot at the previous time step.Action Space. The action space is a 12-dimensional vector a t corresponding to the four legs of the quadruped robot, with each leg having three motor drive units. The neural network's output is converted into actual torque τ through a PD controller.Reward Functions. The reward functions we used during the training are shown in Table A1 , which come from [10, 16, 23] . Domain Randomizations. For the training of our method, domain randomizations are used. The details are shown in Table A2 , which come from [8, 21] .The proposed training framework utilizes the Markov Decision Process (MDP) to guide self-learning of the latent representation through state transitions, distinctions, and cumulative rewards, without relying on manually defined privileged information. All networks in this framework are multi-layer perceptrons (MLPs). Table A3 in the Appendix details the MLP structures, and Table A4 lists the training hyperparameters. The training framework is illustrated in Figure 2 .Encoder: In this architecture, the encoder's input consists of the observation history o H t , which is composed of proprioceptive information o t from the previous 10 time steps. The encoder outputs a 20-dimensional latent representation z t of the observation history:EQUATIONActor-Critic: The policy (Actor) and Critic network are trained jointly via Proximal Policy Optimization (PPO) algorithm [24] . The policy takes as input the current proprioceptive observation o t and the latent representation z t , and it outputs the joint position a t . The Critic network, which shares the same input as the policy, outputs the state value v t . It is worth noting that, we turn off the gradient of z t in the policy and turn it on in the Critic network, using the backpropagation of the Critic network to update the encoder in the direction of the maximum cumulative reward:EQUATIONEQUATIONwhere sg[•] is the stop gradient operator.Transition Model: The Transition model simulates the real state transitions of the environment p (s t+1 | s t , a t ). Its input is the latent representation z t and the action a t , and it outputs the next time step's latent estimation zt+1 :EQUATION) Loss Function: We align the estimated zt+1 from the state transition model with the actual latent state z t+1 at time t + 1, while ensuring distinctiveness from other latent states z t+n at different times t + n. Drawing inspiration from [25, 26] , we formulate a latent representation loss function utilizing a triplet loss, denoted as L trip .L trip (z t+1 , zt+1 , z t+n ) = max ∥z t+1 -zt+1 ∥ 2 2 -∥z t+1 -z t+n ∥ 2 2 + m, 0 , s.t. n ̸ = 1 (5)where m is a margin that is enforced between zt+1 and z t+n pairs, set to 1.0 in the experiment. This updating strategy empowers the encoder to comprehend the dynamics of environmental state transitions and extract environmental attributes by assimilating state-action pairs derived from the MDP rollout. Finally, the triplet loss L trip is multiplied by a triplet coefficient and added to the PPO loss L ppo [24] for network updating. Details of simulation and training are in Appendix 1. 
ablation study for latent representations: 
 To evaluate the proposed algorithm against traditional privileged learning algorithms, an ablation study was conducted. In this setup, commonly used privileged information e t ∈ R 10 was chosen, including parameters such as friction, restitution, foot height, and foot contact. The methods using different types of latent representations are as follows:1) Implicit: The policy input includes the implicit privileged latent l t , which is encoded from privileged information e t by adopting the teacher policy approach in [9, 10] .2) Explicit: The policy input includes the explicit privileged information ẽt = ϕ(o H t ). 3) SLR w/ explicit: The SLR policy input is the concatenation of the self-learning latent z t and explicit privileged latent ẽt . To ensure the fairness of quantitative comparisons, the configurations used in the experiments are all based on the default settings in the code repository [27] . Training was conducted for 6000 iterations under multiple terrains by default, and the results are presented in Figure 3 . And we can draw the following conclusions:1. The SLR algorithm outperforms traditional explicit and implicit privileged learning methods, suggesting that unified privileged information struggles to effectively represent diverse terrains.2. Incorporating additional privileged latent information reduces the SLR algorithm's performance, possibly due to increased policy input disrupting the critic's guidance on updating the self-learning latent.Identifying and distinguishing terrain types is crucial for robotic systems [28] . To evaluate the self-learned latent representation, we conducted a simulation-based analysis. The test environment includes four terrains: an upward slope, descending stairs, flat ground, and ascending stairs. The robot navigates these terrains sequentially (Figure 4 ), and the latent representation output is recorded at each step. We then apply t-distributed stochastic neighbor embedding (t-SNE) to project these complex latent representations into a two-dimensional space for analysis. The latent representations from Implicit and SLR are examined separately, as shown in Figure 5 .Different Terrain Representation. As shown in Figure 5 . The self-learned latent representations reveal distinct ring-shaped regions (labeled A, B, C, D) for each terrain. Notably, regions A and D exhibit similarity, suggesting the robot perceives up slope and ascending stairs as similar processes.In contrast, representations trained with the Implicit method present scattered points, indicative of overlapping privileged information across terrains. 5 Results 
compared methods: 
 Previous comparative studies of robot algorithms often lack fairness due to the absence of a universal benchmark in robotics. Comparing algorithms within one's own environment is not objective, as reward functions and hyperparameters are typically optimized for the researcher's algorithm. Additionally, replicating previous algorithms in a new code repository can lead to incomplete reproductions, potentially affecting their performance.In this study, these issues are addressed by directly implementing the SLR algorithm in the opensource code repositories of previous SOTA works [9, 11, 13, 27, 29] . Only the algorithm framework is modified while keeping other variables consistent with the original settings. This approach ensures a fair comparison between our method and the publicly available SOTA algorithms 1 . The algorithms which are evaluated as given below:• MoB [13] : Explicit estimation, trained on flat ground with the Unitree Go1 robot.• RLvRL [9] : Implicit estimation, trained on flat ground with the Cheetah robot.• ROA [11] : Implicit estimation, trained on custom fractal noise environment using a Unitree Go1 robot equipped with a manipulator.• HIM [29] : Explicit and implicit estimation, trained in multi-terrain environments with the Unitree Aliengo robot.• Baseline [27] : No encoder, no privileged information, trained in multi-terrain environments with the Unitree A1 robot.All five code repositories implemented reinforcement learning based on the PPO algorithm, with training conducted on the IsaacGym platform [27, 30] using three different random seeds. The training curves are illustrated in Figure 6 . Additionally, to evaluate the locomotion capabilities, experiments on velocity tracking performance were conducted on flat terrain. The evaluation metrics included Linear Velocity Tracking Error (LVTE) and Angular Velocity Tracking Error (AVTE), both measured using Mean Square Error (MSE). The evaluation results are summarized in Table 1 .Figure 6 : Training curves from various repositories show that the mean reward of SLR surpasses those of Baseline, RLvRL, and ROA, is slightly higher than MoB, and is comparable to HIM. The shaded area represents the standard deviation across seeds, and the curves are smoothed using Gaussian filtering. Note: SLR achieved a higher terrain level than HIM, indicating superior performance. Based on the evaluation results of the five code repositories, the proposed SLR algorithm generally achieves higher mean rewards and superior velocity tracking capabilities compared to the original implementations. From these results, we can draw the following conclusions:1. The SLR algorithm achieves mean rewards comparable to those of the recently released topperforming HIM algorithm, but with a higher terrain level (see Figure 8 ), indicating superior performance. This demonstrates that learning from limited proprioceptive data can rival advanced privileged learning methods.2. The SLR algorithm's effectiveness in managing complex tasks with MoB (multiple commands) and ROA (manipulator-equipped) repositories suggests that self-learning latent representations offer more general benefits than manually selected privileged information.3. The SLR algorithm demonstrates enhanced tracking capabilities across various velocity ranges. One reason for this is that SLR can implicitly infer velocity from proprioceptive data. Additionally, the Critic network optimizes for maximum cumulative rewards, with velocity tracking as the most significant term in the reward function. Consequently, the backpropagation in the Critic network naturally guides the latent representation to optimize tracking performance.The trained policies are deployed on the Unitree Go2 robot in real-world as depicted in Figure 7 .Performance evaluation of the policy was conducted across various indoor and outdoor terrains, and comparative analyses were performed against code repositories [13, 27, 29] and Unitree built-in MPC control method. Each environment was tested 10 times. As presented in Table 2 , the results demonstrate the superior efficacy of our policy in real-world scenarios. 
discussion and limitations: 
 In this study, we present a quadruped locomotion algorithm that operates without privileged information. Relying solely on limited proprioceptive data, the algorithm achieves SOTA performance.Looking ahead, we anticipate that future self-learning approaches will be able to integrate privileged information and external perceptions, leading to even more impressive outcomes. While our blind policy demonstrates robust motion, achieving superior trajectory planning necessitates the use of vision sensors. Moving forward, we aim to enhance quadruped locomotion by integrating visual information for tackling even more complex challenges. Compute L trip (Eq.5) and L ppo [24] 13:Update π, µ, ϕ by optimizing L ppo + αL trip 14:Empty D 15: end for We presented the details of the Self-learning Latent Representation in Section 3.2 of the main paper. We set H to 10 and the triplet coefficient α to 1.0.In Table A1 , v is the linear velocity, σ is the tracking shaping scale equal to 0.25 here, h target is the desired base height corresponding to ground, p target z and p i z are the desired feet position and real feet position in z-axis of robot's frame and v i xy is the feet velocity in xy-plane of robot's frame. Previous evaluations in a single repository achieved noticeably better performance (see Section 4.1). To provide more conclusive evidence, we extended our experiments. 
 
 Appendix:  


Decision: I agree with the majority of the reviewers and recommend the acceptance of the paper. The reviewers are mainly positive about the paper (3 out of 4 weak accept). The main strength of the paper are
* the simplicity of the algorithm which does not rely on privileged information or student-teacher methods
* the fair evaluation of the algorithms within the original environments
* the real-world experiments of indoor and outdoor environments

I want to especially highlight the fairness of the evaluation as the author used the original environments of the previously published works and evaluated their methods under their conditions. The main weakness of the paper is the limited (potentially marginal) improvement over prior art and the debatable novelty as pointed out by reviewer b8HJ, who was initially not very constructive but improved during the rebuttal. While one could argue in either direction for novelty, I personally would consider the work novel as the exact combination has not been shown in the literature! 

I recommend accepting the paper because it proposes a simple algorithm that works on the physical system and the paper provides a fair comparison to prior art. I am very certain in my evaluation as the paper was reviewed by many reviewers that extensively published on the topic.

Reviews: review: title: title: Review
paper_summary: paper_summary: This paper proposes SLR, a representation learning technique for RL locomotion based on triplet optimization.
main_review: main_review: The paper proposes SLR, a representation learning technique for RL locomotion based on triplet optimization.

Strengths:
- SLR learns a better internal representation than some prior works without relying on privileged information. Avoiding privileged information makes the technique more general because it doesn't rely on the designer to come up with a good choice of privileged variables and compute them during training. Overall, I really like the idea of incorporating contrastive learning in this way during training.
- SLR consistently improves training in open-source codebases from diverse sources
- The visualization of latent space reveals a better clustering of latents when the robot is walking on the same or similar terrains.

Weaknesses:
- Some of the metrics do not fully evaluate the performance difference between SLR and baselines evaluated on the same environment, because they report training return in environments with performance-adaptive curriculum learning.
- Some tables should be reformatted for clarity
- More baseline comparisons against recent works would strengthen the paper
questions: questions: - Figure 6 reports the reward curves for various repositories. Is this the reward from the training rollouts or in a fixed evaluation environment? Some of these environments have a performance-dependent curriculum, e.g. on velocity or terrain, so that a higher training reward does not necessarily reflect higher performance, since the robot might be training on easier terrains or tasks. I believe Table 1 provides a fair comparison evaluating Author vs SLR on a fixed evaluation environment which would be correct, but this should be explicitly discussed. The authors should add some discussion on this to indicate what can be interpreted from each result.
- Similarly, Figure 3 reports reward curves that I expect are based on training returns; but my understanding of [27] is that there is a terrain curriculum, where some policies may advance to significantly harder terrains, and might also be responsible for a lower return. The authors should add (1) plot of the terrain level progression during training, and (2) evaluation of the policies on a fixed evaluation environment (e.g. all level 5 terrains)
- Table 2 shows that SLR can traverse terrains well, but it should be noted that the MoB and RLvRL methods are trained on flat ground. It would be more straightforward to emphasize the performance of ablations of SLR (SLR w/ explicit privileged latent, SLR w/ implicit privileged latent, SLR w/o latent) in one table and the performance vs. other controllers (MoB, RLvRL, MPC) in a separate table, and I would find the ablation table much more interesting. The naming of "Baseline" is not descriptive and is also placed far from SLR which makes it harder to interpret what I want to know from the current table.
- Comparison with DreamWaq would strengthen the paper, since it also claims to learn a better latent representation for locomotion; but I know there is no official code release so this might take more work to provide.
- https://arxiv.org/abs/2312.11460 also proposes a contrastive-based approach to legged locomotion. On one hand, the code was only released after the CoRL submission deadline so this might be considered a concurrent work. But I think comparing HIM vs SLR would strengthen the impact of the paper because I would know which contrastive technique yields better performance.
limitations: limitations: 3
review_summary: review_summary: I like the idea of contrastive learning to form a better latent representation during RL training. The authors should clarify the significance of comparing training reward curves in these environments and revise the results for clarity. Comparison to a recent/concurrent related work would strengthen the paper's impact.
title: title: Review of Self-Learning Representations for Quadruped Locomotion without Privileged Information
paper_summary: paper_summary: This paper proposes a Self-learning Latent Representation (SLR) method for robust quadrupedal locomotion over complex terrains. Unlike state-of-the-art (SOTA) learning-based methods, which rely on meticulously chosen privileged information in simulations to learn a latent representation of the physical properties of the environment, SLR self-learns these latent representations guided by state-transitions, state distinctions, and cumulative rewards under a Markov Decision Process (MDP). The authors demonstrate successful deployment of the learned policy both in simulation and on real robots across various complex terrains. Additionally, they compare the effectiveness of the SLR method against four SOTA open-source references, highlighting its robustness and adaptability in real-world applications.
main_review: main_review: This research work introduces a novel and simplified end-to-end learning-based locomotion method, which autonomously learns the latent representation without the need for laborious teacher-student privileged information learning-based methods. This approach enhances the fairness of evaluation by comparing against SOTA methods within their environment configurations. The concept of self-learning latent representations is well-motivated and holds significant importance in the field of learning-based locomotion. The paper clearly and comprehensively presents the proposed framework, demonstrating strong clarity and organization. Furthermore, the extensive real-world demonstrations substantiate the high quality and practical applicability of this research, providing robust evidence of its effectiveness and reliability. 

**Strengths**

1. **Easiness of training method**: The self-learning latent representation (SLR) method simplifies the training process by eliminating the need for selecting and experimenting with combinations of privileged information, as required in current SOTA methods [10], [12], [13] mentioned in the paper.
2. **Fair Evaluation**: Implementing SLR within SOTA open-source code repositories and comparing the reference using their optimized configurations ensures a fair evaluation. This approach demonstrates the effectiveness of SLR under the same conditions as existing methods, providing a reliable comparison of performance.
3. **Clear Ablation Study**: The t-SNE visualizations presented by the authors show that SLR latent representations have a strong correlation with the underlying terrains. The visualizations depict upslope and stair climbing shares similar understanding of the environment, effectively illustrating the method's ability to capture and differentiate the nuances of various terrains.
4. **Strong Real-World Experiments**: The submitted video shows successful and robust behaviors in indoor and outdoor environments

**Weakness**
1. **Limited Analysis on Linear Velocity Tracking**: In comparison with four reference SOTA algorithms shown in Table 1, the authors didn't provide sufficient evidence explaining why the linear velocity tracking (LVT) is underperforming compared to angular velocity tracking (AVT) in the case of SLR. This lack of analysis leaves a gap in understanding the specific challenges and limitations of SLR in LVT scenarios.
2. **Power Consumption Comparison**: The authors didn't compare the overall power drawn by their policy against other methods. Observations from the submitted video (00:35) suggest that the robot tends to raise its forelegs higher even on cobbled roads, which might lead to higher power consumption. However, this could potentially be addressed by incorporating more penalty for power consumption in the reward function.
questions: questions: **Issues**

1. In Section 3.2, please provide more details of the neural networks used in your framework, either in the manuscript or supplementary material. Specifically, what is the dimension of 𝑧𝑡 ? What is the size of the MLP for the encoder and the state transition model?
In line 112, please clarify how the state transition model and encoder share the same network structures despite having different inputs.
2. What is the size of the Replay Buffer?
3. In the loss function, what is the value of the margin 'm' used?
4. In Section 5 (Results), please consider citing the acronyms of reference methods for clarity.
5. What was the reason that RMA [10] was not considered a potential reference for comparison? While RLvRL [9] uses RMA to train its policy, [9] is more biased towards rapid locomotion tasks. It would be interesting to see a performance comparison against RMA [10] as it has also shown similar robust behaviors in indoor and outdoor environments.

[9] G. B. Margolis, G. Yang, K. Paigwar, T. Chen, and P. Agrawal. Rapid locomotion via reinforcement learning. The International Journal of Robotics Research, 43(4):572–587, 2024.

[10] A. Kumar, Z. Fu, D. Pathak, and J. Malik. Rma: Rapid motor adaptation for legged robots. 2021.
limitations: limitations: 3
review_summary: review_summary: I would like to recommend this paper for acceptance. This paper presents a significant and original contribution to the field of learning-based locomotion for quadrupedal robots. The novel approach of self-learning latent representations without relying on privileged information simplifies the training process and demonstrates robust performance across various complex terrains. While there are areas for improvement as mentioned in the weakness and rebuttal section, the strengths of this work make it a valuable addition to the research community.
title: title: Official Review of Paper228 by Reviewer Bcyx
paper_summary: paper_summary: Incorporating privileged information to learn a latent vector encoding various environmental parameters or using it to estimate hard-to-compute parameters have proven to be very effective techniques in the domain of quadrupedal locomotion. The given paper operates in the blind locomotion setting and tries to estimate these environmental parameters without explicitly using privileged information. The proposed method instead relies on using just proprioceptive information to encode the observation history into a latent embedding and learning a state transition estimator for the latent. This latent embedding allows the learned policy to operate in different terrains and transition smoothly between them, and be robust to external disturbances. The authors term this the Self-learning Latent Representation (SLR) method.
main_review: main_review: The paper presents an interesting method and is generally easy to follow. It provides sufficient coverage of related work. The authors have also clearly delineated the position of the paper.

Strengths
- The learned latents can differentiate between different terrains with the t-SNE visualizations depicting distinct ring-like structures.
- The proposed algorithm has been reimplemented in the repositories of the baseline environments without affecting the other simulation parameters to ensure fair comparison.
- The authors demonstrate impressive real-world performance, with the robot able to walk on various terrains and robust to external disturbances.

Weaknesses
- The proposed method section could benefit from additional details about the setup. This will help with clarity and reproducibility.
- The terrain transitions argument can be substantiated further, especially since this effect is being validated in sim-only. The authors can potentially test a few other terrain sequences to obtain a clearer picture of how their self-learned latent represents transitions.
questions: questions: * It would be great if the authors could provide more details about the training process for triplet optimization. For example, it is unclear how the other latents are sampled to enforce distinctiveness, or what the frequency of updates for the encoder is. It might be nice to explicitly show the overall sequence in the form of pseudocode or an algorithm. This will help with reproducibility.
* Given that some of the papers cited by the authors report more more dynamic behaviors (such as Extreme Parkour and Rapid locomotion via reinforcement learning), it might be nice to have higher target velocity ranges, especially for comparison against the MoB and RLvRL baselines which seem to have better linear velocity tracking but worse angular velocity tracking than the proposed method. 
* Another minor suggestion is to cite the respective paper with the baseline in section 5.1 as this will further help improve clarity.
* While the real-world deployment performance is impressive, it would further help to explicitly specify the success criteria and relevant metrics (such as distance covered before a fall or time before a fall etc) for further transparency.
limitations: limitations: 2
review_summary: review_summary: While the authors show improvement in performance compared to the baselines, the main differentiating factor lies in the interpretability of the latent. It might be interesting to extend it to more dynamic and agile behaviors, especially trying to transition between such motions. The paper could be improved further by providing additional details about the implementation of the proposed method and a slightly more thorough substantiation of the claims.
title: title: Review
paper_summary: paper_summary: This paper aims to propose a quadrupedal locomotion learning framework that does not need privileged information during training stage.
main_review: main_review: The paper proposes a robot locomotion learning framework that does not require privilege information during training stage and can adapt to different environment settings during deployment. It is achieved using a self-learning mechanism which learns to map different terrains to structured latent space. The results have been show in both simulation and real-world. The proposed method outperforms baseline methods in different evaluation metrics.

In the initial version of the paper, the conclusion that the method is useful is not well-supported by the numbers in the paper. The quantitative differences between the proposed method and baselines are either very limited or hard to judge. And the paper was also lack of explanation and insight to help understand the overall contribution of the paper.

In the rebuttal stage, the author provide more evidence to support the results, more result analysis is also provided. Therefore, compare to the initial version, the quality of the paper improves a lot.
questions: questions: I have no question for rebuttal.
limitations: limitations: 3
review_summary: review_summary: The paper is more convincing as new results and explanation provided during rebuttal.