Humanoid Parkour Learning

Authors: ziwen  zhuang
shenzhe  yao
hang  zhao
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-06 09:17:22

Abstract: Parkour is a grand challenge for legged locomotion, even for quadruped robots, requiring active perception and various maneuvers to overcome multiple challenging obstacles. Existing methods for humanoid locomotion either optimize a trajectory for a single parkour track or train a reinforcement learning policy only to walk with a significant amount of motion references. In this work, we propose a framework for learning an end-to-end vision-based whole-body-control parkour policy for humanoid robots that overcomes multiple parkour skills without any motion prior. Using the parkour policy, the humanoid robot can jump on a 0.42m platform, leap over hurdles, 0.8m gaps, and much more. It can also run at 1.8m/s in the wild and walk robustly on different terrains. We test our policy in indoor and outdoor environments to demonstrate that it can autonomously select parkour skills while following the rotation command of the joystick. We override the arm actions and show that this framework can easily transfer to humanoid mobile manipulation tasks. Videos can be found at https://humanoid4parkour.github.io

Main paper: 
 introduction: 
 Athletic Intelligence is an important field of research when developing an embodied intelligence system. To perform daily human tasks or replace humans in some dangerous tasks, we need to develop legged robots, quadrupeds, or humanoids, that can overcome all human-capable terrains. Parkour is an exemplary task of athletic intelligence, requiring robots to move fast and deal with various obstacles, such as jumping up to a high platform, leaping through a long gap, and jumping over a high hurdle. These tasks require a unified system to perceive and memorize its surroundings [1, 2] , autonomously make decisions to select proper skills, and execute challenging tasks with its powerful hardware [3, 4, 5] . In the context of robot learning, building a general parkour system that runs fully onboard without external support and without limitations to the non-critical properties of the terrain 2is the grand challenge to explore the limit of an athletic intelligence system.Recent progress in humanoid robots starts with Boston Dynamics Atlas robots [6] , which perform stunning parkour acts [7] in their lab environment. However, the agile ability comes from massive engineering efforts to plan all the parkour acts offline and is most likely limited to that one obstacle order. Also, the high cost of hardware construction and maintenance restricts most humanoid skills only in simulation [8, 9] . Recent research efforts on learning-based quadruped and bipedal robot locomotion have demonstrated impressive robustness in walking [10, 11, 12] , walking up and down stairs [13, 14, 15] and overcoming challenging terrains [4, 3, 16, 17, 18, 19, 20] . However, learningbased methods for humanoid robots are still limited to walking on the plane [21, 22, 23, 24, 25] where the legs cannot produce task-specific motions without any motion prior. It is great timing to explore a general way of building locomotion models with less motion prior so that humanoid robots can fully exploit the embodiment and perform tasks beyond plane locomotion.Based on recent progress in humanoid locomotion, more agile locomotion for humanoids has several challenges. First, diverse skills for a single locomotion network are challenging. Previous methods in quadruped rely on specific engineering for each locomotion task. Humanoid locomotion based on learning methods requires reward terms to encourage foot-raising and prevent drag on the floor. These implementations significantly limit the legs' motion and keep them away from generating more diverse skills. Second, mimicking animals or humans lacks egocentric perception, and a huge amount of data for different types of terrain and embodiments is needed. This significantly restricts the scalability of these methods. Third, proprioception and exteroception take time to process. The system lagging must be strictly controlled so that the entire robotic system can act on time. Lastly, parkour skills exploit the maximum actuator capacity. Using electric motors with batteries brings significant challenges to the power throughput. Proactive action smoothness and restrictions are needed to mitigate the potential damage to the humanoid robot hardware. This paper introduces a unified humanoid parkour learning system for at least 10 types of humancapable terrains, such as jumping up, jumping down, leaping over gaps, and jumping over hurdles. Our reinforcement learning system proves that fractal noise in terrain, which is frequently used in quadruped robots, trains a deployable humanoid locomotion skill without any motion reference or reward term to encourage foot raising. Our training objective is simple enough to train multiple agile humanoid locomotion skills in a unified manner while being able to deploy to the real humanoid robot with zero-shot sim-to-real transfer. Because of the straight parkour track, we train our parkour policy from a pretrained plane locomotion policy, so that the policy will respond to the turning command even if the locomotion command tells the robot to walk along the straight track. We then use DAgger [26, 27] with 4-GPU acceleration to distill a vision-based parkour policy that can be deployed on the real humanoid robot with only onboard computation, sensing, and power support. 
related work: 
 Legged Locomotion Legged locomotion research starts with model-based control [28, 29, 30, 31] . Either quadruped robots or bipedal robots require heavy mechanical design and dynamics modeling to implement the control algorithm [28, 29, 30, 31, 30] . These methods are all aimed at walking in the plane and as robust as possible. Till the release of the general physical simulator [32, 33] , reinforcement learning algorithms can be applied to legged locomotion tasks [34, 35, 36, 37, 19, 10, 38, 39] . But due to lack of exteroception, most of the learning algorithms are limited to lowspeed locomotion [19, 40] . To perceive the terrain around the robot, typical methods use vision and depth sensors to estimate the robot state [41, 42, 43, 44] and response to different terrain [15, 45, 35, 3, 4, 46, 20] . Classical methods involves state-estimation [41, 42, 43, 44] , traversability mapping [47, 48, 49, 50] , and then plan their foothold [51, 18, 52, 45] . Learning based end-to-end system reacts to different terrains by depth vision [35, 53, 54] , recovering elevation maps [13, 55, 56] or learned neural spaces [46, 57] . But only recently, locomotion on quadruped robot has demonstrated extreme agile locomotion using depth-sensing pipeline [3, 4, 58] . Humanoid Locomotion with Learning Algorithm With the rapid development of reinforcement learning research and many zero-shot sim-to-real transfer algorithms applied on legged robots, bipedal robots such as Cassie [59] have already been deployed with various reinforcement learning algorithms [14, 60, 61, 62, 20] . However, due to the low inertia of the body and the start of the humanoid industry, little research on humanoid learning algorithms has been done. Using reinforcement learning and transformer architecture, humanoid robots can walk smoothly in the wild [22] . Arms movement not affecting the balancing and locomotion of the legs has also been verified [21, 23] . However, these methods still require significant reward engineering, either reference trajectory [22, 63] or encouraging feet raising [21, 23] , to achieve possible sim-to-real robot behavior. In this work, we propose using fractal noise [11] on the terrain to encourage foot raising while leaving sufficient freedom to train downstream whole-body control tasks. In this section, we aim to describe our implementation that trains a zero-shot end-to-end policy that uses onboard depth vision and robot proprioception to autonomously do parkour tasks when encountering various obstacles. Since rendering depth image in isaacgym [33] costs too much time, recent sim-to-real methods use prebuilt terrain information to train an oracle policy and then distill a deployable policy by switching the exteroception encoder. We divide the training pipeline into 3 stages. We first train a walking policy that follows locomotion commands for walking forward, walking sideways, and turning. In order to train the walking policy with proper gait in the real world, we add fractal noise to the height field of the terrain [11] , so that the robots learn to raise their foot when moving around. Then, we train that parkour policy with all parkour skills using an auto-curriculum mechanism. We use scandots as a means of perceiving the terrain because the height field is stored in GPU memory when the terrain is being built. We implement 10 different types of terrain as shown in Figure 2 , so that the oracle policy will see as much terrain as possible and generalize to almost all human-capable terrain. We also add virtual obstacles that encourage the policy to keep away from dangerous behavior [3] . Then, we simulate the noise pattern of the Intel RealSense D435i stereo depth camera and distill a parkour policy using DAgger [26, 27] . Lastly in this section, we describe how we deploy our parkour policy on the real robot. 
training forward parkour from planar walking: 
 To build a parkour policy that also follows the command of high-level navigation, the robot must be equipped with a turning ability. Since we build the the parkour track in a straight line so that the robot can autonomously keep walking to the next track, the turning ability must be pretrained. Considering perceiving the terrain using depth sensing is quite costly due to the implementation of isaacgym [33] , we use scandots to acquire terrain information, shown in the supplementary. Different from [3] , we don't need to represent obstacles that are above the robot, so scandots are general enough.We formulate the oracle policy using a Gate Recurrent Unit (GRU) [64] and an MLP as the backbone of the actor. We use an MLP to encode a s scandot t ∈ R 11×19 scandot observation and return a s embedding ∈ R 32 terrain embedding. We use a GRU connected by an MLP as the state estimator to predict the linear velocity vt ∈ R 3 of the robot base using the onboard available proprioception s proprio t ∈ R 43 (row, pitch, base angular velocities, positions and velocities of joints) and last action a t-1 ∈ R 19 . Then, the oracle policy uses terrain embedding s embedding , estimated base velocity vt , and proprioception s proprio t to predict an action a t as all of the target joints of the robot body.We use PPO [65] to train our policy. Then, the value network uses the same architecture as the policy network, except using the ground-truth linear velocity from the simulation. For better sim-to-real deployment, we use multiple domain randomization techniques on the robotics dynamics. We describe the details in the supplementary.To train a general plane-walking policy that follows the x, y, yaw moving command, we used fractal noise [11] and multiple reward terms. The rewards and their weights are shown in the supplementary. The reward function consists of mainly three parts: 1. Task rewards, including linear velocity tracking, angular velocity tracking, and minimizing total energy consumption; 2. Regularization rewards, including torque penalty, joint acceleration penalty, joint velocity penalty, and foot contact penalty; 3. Safety rewards, including unnecessary joint movements penalty, such as arm and waist, keeping both feet away from each other, and lower action rate. As Table 1 shows, we sample the moving command uniformly and train the walking policy.To meet the needs of following navigation commands when walking in the plane and overcoming parkour obstacles, the walking policy is equipped with walking and turning ability. However, the parkour track is a straight line along the x-axis. Putting the robot facing +x direction and sending only forward command will lead to the loss of turning ability. We initialize the robot orientation uniformly in all directions and design an autonomous command mechanism to guide the robot to turn to the parkour track.Suppose the goal position is in the direction of θ goal of the robot and the robot orientation is θ, the yaw command will be v cmd yaw = α yaw * (θ goal -θ), where α yaw is a predefined ratio. If ∥θ goal -θ∥ >= π/2, the forward-moving command will be set to zero. Otherwise, the forward-moving command will be set to a sampled value in Table 1 . Then, the trained policy will keep following the straight parkour track while preserving the turn-by-command ability.To generate different terrain for massively parallel simulation, we put 10 types of parkour obstacles in a grid of 10 rows and 40 columns. Each column is a single type of obstacle, shown in Figure 2 . Each row is an obstacle with different difficulty. For the i-th row, the critical parameters of the obstacle are scored by (1 -i/10) * l easy + i/10 * l hard . In each sub-terrain, the parkour track consists of a starting plane to initialize the robot and an obstacle block where the obstacle is built. When the sampled forward command is greater than zero, the robot will move to a more difficult terrain if the robot safely finishes the task and moves at least 3/4 distance of the sub-terrain. The robot will move to an easier terrain if the robot fails at a distance smaller than 1/2 of the sub-terrain. The ranges for the critical properties of each parkour obstacle are listed in the supplementary. In order to generate a safer policy with action redundancy and more accurate action steps, we add two reward terms to train the parkour policy.For several tasks such as leaping, the robot tends to exploit the edge, which is sensitive to motor error. We add virtual obstacle [3] in some of the terrains, e.g. leap and jump. We compute the penetration penalty and apply it during reward computation. As Figure 3 shows, we bind a mesh of points around each link of the robot body. The dark blue line draws the virtual obstacle, which the robot should avoid. The green points show the estimated volume does not penetrate any virtual obstacle. The red points show the estimated volume penetrating the virtual obstacle. For this hurdle obstacle, the penetration surface is on the top. Then, the penetration depth d(p) for a given point is shown in Figure 3 . We compute penetration penalty as an additional reward term byEQUATIONwhere p is the penetration depth of all red points bound on the robot bodies. α penetrate = -5 × 10 -3 in our case.For tasks such as stairs up and stairs down, reinforcement learning does not generate accurate enough action without explicit footstep guidance [66] . We assign the recommended stepping point to the middle of each stair. We ignore the stepping distance perpendicular to the forward direction. When the robot is walking on stairs, the offset between the recommended stepping position and the actual stepping position is d = (d x , d y , d z ). We compute the footstep reward byEQUATIONwhere α step = 6 in our case.Initialize Student Policy from Teacher Policy In order to fully utilize the pretrained oracle model, we replace the original scandots encoder with a randomly initialized CNN network, which encodes a I t ∈ R 48×64 depth image into the terrain embedding ŝembedding ∈ R 32 . We initialize the weight of the GRU and MLP module of the student policy using the weight of the Oracle parkour policy. We keep using the well-trained linear velocity estimator and keep optimizing this module.Bridge the Depth Image Gap between Sim and Real While training the oracle parkour policy in Section 3.1 bridging the gap in physical and mechanical dynamics using various domain randomization techniques, the terrain information acquired by the oracle parkour policy is still lossless and unavailable in the real world. We address this gap by simulating the noise pattern of the depth image and applying built-in filters in the Intel RealSense API when acquiring depth images onboard. Shown in Figure 4 , we apply depth clipping, multi-level Gaussian noise, and random artifacts to the raw rendered depth image. We apply depth clipping, hole filling, spatial smoothing, and temporal smoothing to the raw stereo-depth image in the real world. We observe a single Nvidia 3090 GPU is inefficient in rendering depth images in the IsaacGym simulator. We use 4 Nvidia 3090 GPUs in 4 processes to accelerate the DAgger [26, 27] distillation process. We use one process as the trainer and keep loading the student trajectories and teacher action labels. We use the other 3 processes as the collectors to collect the student trajectories and label them using the Oracle parkour policy as the teacher.The trainer and collectors pass the latest student policy and the labeled trajectories using a shared filesystem, which all processes can access. Since both student policy and Oracle parkour policy outputs action in the real number domain, we distill the student policy using the L1 norm between the student action and teacher action as the objective.We directly copy the distilled parkour policy model to the Unitree H1 robot and use the same network implementation during training. We run the visual encoder and the rest of the network on two separate Python processes on a 12-core Intel i7 CPU. These two processes communicate with each other and with the motors on the humanoid robot using Cyclone DDS as well as the ROS 2 platform. For the depth vision, we use RealSense D435i which is mounted on the head of the humanoid. Other details will be presented in the supplementary file. 
experiment results: 
 In this section, we first describe the setup for our humanoid robot and the experiment setup in simulation and in the real world. Then, we use three separate experiments to demonstrate the following questions: 1. Is using fractal noise terrain feasible compared to "feet air time" reward or motion reference to make the foot raise 2. Concurrent works use "feet air time" or motion prior [67] to make the walking policy able to walk up and down stairs without vision. Is the vision for challenging terrain necessary? 3. Previous Parkour [3, 4] uses either single-GPU distillation or distill a student policy from scratch. Why does our method require both initializing student policy from the oracle policy and using multi-GPU acceleration?We use Unitree H1 and their official urdf model to train and test the policy in isaacgym [33] simulator. We evaluate different tasks in simulation by building 10 straight tracks. Each track corresponds to a single type of obstacle. Each track has 3 connected sub-tracks with 3 linearly increasing difficulties on the obstacle properties ranges in the supplementary.For the real-world experiment, we use several coffee tables with a height of 0.42m and several wooden plyo boxes with a height of 0.4m (16 inches) for jumping up and down, as well as leaping. We use curb ramps for the slopes of 0.2m height and 0.5m length (0.38 rad). We use 0.2m flat ramps to build small stairs with wooden boxes. We put the curb ramps side by side to test the tilted ramp terrain. We use foam pads to build low hurdles with heights of 0.25m.Figure 5 : Real-world quantitative results. Our parkour policy achieves the best performance in the 4 difficult tasks compared with the blind walking policy and the built-in MPC controller in Unitree H1.We use the user interface in H1's controller to raise its foot height so that it can overcome some of the obstacles. We run 10 trails for each testing configuration.We compare the success rate and average moving distance between the oracle policies trained with fractal noise terrain (ours) and with "feet airtime" [21, 23] reward terms in all 10 skills in simulation. We deploy 10 robots for each type of terrain with no fractal noise and put them in the easiest sub-track. Shown in Table 2 , the success rate is computed from the number of robots that successfully overcome the terrain with maximum difficulty. The average moving distance is computed from the average distance before the robot falls or stops in a given type of terrain. Shown in Table 2 , our method that trains on fractal noise terrain performs better than that using "feet air time" in the reward function, which further illustrates that the "feet air time" term conflicts with some of the locomotion tasks. 2 : We compare the success rate and average moving distance in simulation. Each type of terrain is a connected 3 sub-track with linearly increasing difficulty. Each sub-track has a length of 4.8m, which makes the total length of a given terrain 14.4m. The success rates are averaged across 10 runs for each case. (More experiment results for the other obstacles are shown in the appendix.)We perform real-world quantitative experiments to measure the performance of our parkour policy. Due to the safety issue, we quantitatively test the success rate of the 4 most difficult tasks in the indoor environment. We change the difficulty by modifying the key properties of each obstacle, such as the height of the platform, the gap length for leaping, and the height of each stair. Since our parkour policy follows a directional command, we filter out the operator failure. Shown in Figure 5 , we test our policy in the real-world lab environment. The robot can select the proper skill in challenging parkour tasks based on its onboard depth vision.We compare the efficiency of using multi-GPU acceleration in terms of transitions experienced by the student policy network and the success rate after 24 hours of training. The student policy distilled from scratch still struggles to keep balance. The student policy distilled using one GPU reaches only half of the performance of the fully accelerated distillation. Details are described in the supplementary. We observe that under the arm regularization term of the reward function, the whole-body-control parkour policy is still able to manage arm movement. During jumping up, the parkour policy swings up the right arm to keep the arm away from the body while keeping balance during the agile maneuver. Previous work in humanoid robots stated that overriding the arms action in a whole-body-controlled policy will significantly disrupt the locomotion stability. We aim to verify this statement. In the real world, we build a set of joint sensors that map human arm motion to Unitree H1 arm joints. Shown in Figure 7 , the robot runs the parkour policy while the arm is teleoperated by a human operator. The robot can still keep balance while the arm is not acting under the policy's command. 
conclusion, limitations and future directions: 
 We present a parkour learning system that can be applied to humanoid robots, without complex motion reference. We propose using fractal noise and a two-stage training method to simplify the reward function and let the parkour policy follow the turning command even when the parkour track is straight. We test our method on Unitree H1 to show the effectiveness of our framework. However, our method relies on manually constructed terrain. It is difficult to generalize to unseen terrains without further training. Parkour is a whole-body-control task with extreme difficulty. We show that the parkour policy is robust to arm-action override, which can be used in manipulation tasks. However, more difficult manipulation skills require further tuning to avoid disrupting the visual system. We will investigate how to utilize additional training to cooperate with manipulation skills while preserving the extreme locomotion ability.  
 
 Appendix: In Table 6 , we list out the critical attribute of each type of parkour obstacle. The depth images in both the simulation and the real robot are acquired from a 480 × 640 sensor resolution. Then, the depth image is processed and resized to 48 × 64 resolution. The proprioception is sent by the motor system through Cyclone DDS at 500Hz. We run the visual encoder process at 10Hz and the rest of the parkour network at 50Hz. The vision encoder process acquires a depth image, computes the image latent, and sends the latent vector to the parkour network using ROS message. The parkour network sends joint position targets to the motors with different kp and kd factors on each joint, shown in Table 8 . The motors generate torque through their built-in PD controller. To ensure safety, we clipped the joints target position based on the torque limit τ max : clip (q target , (kd * q -τ max )/kp + q, (kd * q + τ max )/kp + q)We use the Unitree H1 humanoid robot for our real-world test, which is equipped with an Intel RealSense D435i and an Intel Core i7 12-core NUC onboard. The robot has 19 joints including the arm and waist. We assign different kp, kd parameters for each joint as shown in Table 8 . We use ROS2 and Cyclone DDS for the communication between the policy and motors. The depth encoder sends its embedding to the recurrent network at 10Hz. The recurrent network receives and sends the target positions of all joints to the motors at 50 Hz. The motors compute the torque based on their built-in PD controller at 1000Hz.Non-critical properties example: the texture of a jumping platform, the depth of a gap for leaping, and the order of a series of obstacles 


Decision: This paper introduced a sim-to-real RL framework for learning vision-based humanoid locomotion behaviors using egocentric depth.  It utilizes a three-stage RL training process to achieve agile parkour behaviors and successfully deploys the learned controller to real robot hardware (Unitree H1). This work received mixed initial reviews, with one strong accept, one weak reject, and one weak accept. The reviewers pointed out the work's strengths as 1) the real robot results demonstrated convincing evidence of the effectiveness of this method, 2) the overall presentation of the work is good, and the detailed description of the training process is clear, and 3) they demonstrated the robustness of the controller while teleoperating the humanoid upper body. In the meantime, some major weaknesses were also mentioned by the reviewers, including 1) the work lacks substantial technical innovations, especially considering prior work on quadruped locomotion, 2) additional ablation studies and analyses (e.g., separation of the three training phases, the impact of training on diverse terrains, performance limits of the proposed method) are needed to improve the quality of the draft, and 3) the demonstrated robot behaviors, while impressive, may not fully encompass the ambitious goal of "humanoid Parkour," so a more accurate title that avoids overclaiming might be considered.

**Post-rebuttal update:**

The AC appreciated the detailed responses and additional experiment results in the authors' rebuttal, which definitely improved the quality of the submission. Toward the end of the discussion period, Reviewer CnFz upgraded their rating to Weak Accept, making the three votes to be unanimously positive. The AC believes that, with the positive changes during the rebuttal phrase, this paper has passed the threshold of publication for CoRL and would recommend acceptance of this work.

Reviews: review: title: title: Strong real world results on an important problem
paper_summary: paper_summary: The paper proposes a sim to real RL based method for parkour on a humanoid robot (H1) in the real world using egocentric depth. The method is gives very strong real world results.
main_review: main_review: - The paper is very well written, clear and easy to follow. The method figures and the tables are also quite clear
- The paper employs a 3 stage simulation based RL training to first train a velocity tracking policy, followed by a teacher parkour policy that uses privileged information and then finally distills the teacher policy into the student policy. The high level pipeline is a common approach to learned locomotion and the paper extends that to achieve sim to real transfer of complex behaviors for bipeds which poses difficulty in behavior learning, as well as sim to real transfer since the bar is much higher for both. 
- The paper will have an impact is demonstrating the feasibility of learning complex maneuvers in sim and then transferring them zero-shot to real.   
- The paper over looks strong, modulo a few ablation experiments which are detailed below.
- The section on "upper limb control" is a very impressive side application of the proposed method. 
- A more detailed limitations discussion would help the paper. For example, the best humanoid parkour till date is still the ones shown by boston dynamics. However, the limitation section is limited to talking about manipulation as the next missing piece without addressing the limited behaviors achieved in parkour.
questions: questions: - The paper proposes 3 learning phases. Is there an ablation where the learning in the first two phases is combined together to understand the extent to which it affects performance?
- An analysis of sim to real gap on high speed or agile maneuvers is missing. I would expect the performance of sim to real to be worse for fast and agile motions than slow motions. It would help to have an understanding of that in any simple way possible. 
- It is not clear if the paper maxes out the limits of the hardware in terms of jumping, etc. It would be good to understand to what limit is the hardware used of its full capacity. 
- The limitations section should additionally discuss whats missing to achieve the same level of agility as achieved by Boston Dynamics via the use of learning based method.
limitations: limitations: 2
review_summary: review_summary: The paper pushes the frontier on sim-to-real RL for parkour on humanoid robots. Modulo a few clarification questions, the paper looks quite strong.
title: title: Review for Humanoid Parkour Learning
paper_summary: paper_summary: This paper showcases how to leverage the recent succuss of quadruped parkour learning and apply it to a much harder problem - humanoids. The real world demos are impressive and convicing, but the paper writing and experimental results need more justifications.
main_review: main_review: **Strengths**
- Very impressive humanoid parkour demos in the real world.
- The paper is easy to follow and the presentation is clear.
- Training these parkour policies is not easy, and it is really nice to these sim2real tranfer works for humanoid parkour.

**Weaknesses**
- Methodology wise, I don't see a big difference of this paper and previous quadruped parkour works. 
- Some claims in paragraph 3 of introduction are not well-justified:
    - why diverse skills for a single locomotion network are challenging? Many works have already proven diverse skills for a single locomotion network. 
    - why mimicking animals or humans lacks egocentric perception? Adding perception is definitely doable to train vision-based mimicking policy. It is an option to add on, and mimicking-style work could scale to perception. I don't see why "This significantly restricts the scalability of these methods."
- Fractal noise needs more detailed introduction since the paper claims several times the advantage of fractal noise over legged reward tuning.
- Why use straight parkour lanes for training instead of omni-directional terrains? This could saves the difficulty to handle "Auto Command to Walk along the Track"
- Also, for the oracle parkour policy training, each column is a single type of obstacle. Does this mean during training, the humanoid is trained to transverse over a lane with same obstables? I wonder how the humanoid performs when there is a lane with various terrain types ahead. This experiment is needed 
- The following claim seems not right "Concurrent works use “feet air time” [21, 23] or motion reference to make the walking policy able to walk up and down stairs without vision". However, these two metioned works do not show any stairs walking demo to the best of my knowledge.
- The caption and content of Table 2 is too close.
- The paper reports a lot of success rates but the number of runs are missing, both in real world and simulation.
- This paper needs more experimental results to justify the design choices. 
    - For example, how does GRU compare with other more recently used architecures like Transformer or simple MLPs with short-term histories.
    - Curves in Figure 5 does not show the limit of the proposed learning method. When will the proposed method fail? Is it a hardware limit or training design limit?
    - Some abaltions are needed, for example, why types of 10 terrains? Is 10 enough of not enough? What is the performance of parkour policy with only 2/3/5/7 terrains types? How much is the proposed method gonna benefit from more terrains?
questions: questions: The questions are summarized in the weakness section. I am happy to increase my scores if my concerns could be answered.
limitations: limitations: 3
review_summary: review_summary: The real-world demo is impressive, but the paper needs more detailed writing and more experiments to justify the design choices. I vote for weak reject for the current paper presentation.  But I would be happy to raise my scores if my concerns could be addresed.
title: title: Review
paper_summary: paper_summary: This paper proposes a sim-to-real pipeline that enables humanoid robots to successfully parkour in the real world.
main_review: main_review: **Strengths:**

1. The paper introduces a novel framework for training humanoid robots to perform parkour skills using fractal noise in terrain, which is a creative solution to the challenge of locomotion without motion priors.

2. The authors provide a detailed description of their training method, including the use of scandots for terrain perception and DAgger for policy distillation.

3. The successful demonstration of sim-to-real transfer is significant, showing that the policies trained in simulation can be effectively deployed in real-world scenarios.

4. The framework is very robust in the real world, and can even balance when the upper body is teleoperated.

**Weaknesses:**

1. Though I appreciate the efforts in engineering, the overall technical novelty is limited. The framework to do robot parkour has been well proposed in previous works such as Robot Parkour Learning and Extreme Parkour.

2. The pure RL pipeline might make the parkour motions of humanoid robots not so natural, compared to motion-based methods.

3. I would suggest a title change to make the paper more accurate and not overclaimed. I fully appreciate the success of making humanoid robots walk in challenging terrain in real world, while I do think the current demo shown by authors is not enough to cover "Humanoid Parkour", which is an ambitious goal.
questions: questions: See weaknesses.
limitations: limitations: 3
review_summary: review_summary: This paper presents high quality real world experiments, and thus I recommend an accept. However, the current title seems to be overclaiming. I would recommend a title change before acceptance to be rigorous.