Learning Semantics-Aware Locomotion Skills from Human Demonstration
Authors: Yuxiang Yang
Xiangyun Meng
Wenhao Yu
Tingnan Zhang
Jie Tan
Byron Boots
Venue: CoRL 2022 Poster (2022)
Keywords: Legged Locomotion
Semantic Perception
Imitation Learning
Hierarchical Control

Abstract: The semantics of the environment, such as the terrain type and property, reveals important information for legged robots to adjust their behaviors. In this work, we present a framework that learns semantics-aware locomotion skills from perception for quadrupedal robots, such that the robot can traverse through complex offroad terrains with appropriate speeds and gaits using perception information. Due to the lack of high-fidelity outdoor simulation, our framework needs to be trained directly in the real world, which brings unique challenges in data efficiency and safety. To ensure sample efficiency, we pre-train the perception model with an off-road driving dataset. To avoid the risks of real-world policy exploration, we leverage human demonstration to train a speed policy that selects a desired forward speed from camera image. For maximum traversability, we pair the speed policy with a gait selector, which selects a robust locomotion gait for each forward speed. Using only 40 minutes of human demonstration data, our framework learns to adjust the speed and gait of the robot based on perceived terrain semantics, and enables the robot to walk over 6km without failure at close-to-optimal speed

Learning Semantics-Aware Locomotion Skills from Human Demonstration
Yuxiang Yang 12, Xiangyun Meng', Wenhao Yu², Tingnan Zhang", Jie Tan", Byron Boots!
University of Washington, 2 Roobotics at Google
lyuxiangy xi xiingn,, bboots /Ccs washington. edu Imagicmelon www.cmmmmonnnnniin tingnan, jietan @
The semantics of the environment, such as the terrain types and prop- erties, reveal important information for legged robots adjust their behaviors. In this work, we present a framework that uses semantic information from RGB im- ages to adjust the speeds and gaits for quadrupedal robots, such that the robot can traverse through complex offroad terrains. Due the lack of high-fidelity offroad simulation, our framework needs to be trained directly in the real world, which brings unique challenges in sample efficiency and safety. To ensure sample effi- ciency, pre-train the perception model the risks of real-world policy exploration, we leverage human demonstration to train speed policy that selects desired forward speed from camera images. For maximum stability, we pair the speed policy with a gait selector, which selects robust locomotion gait for each forward speed. Using only 40 minutes of hu- man demonstration data, our framework learns to adjust the speed and gait of the robot based on perceived terrain semantics and enables the robot to walk over 6km safely and efficiently.
Abstract:
off-road driving dataset. To avoid
Keywords: Legged Locomotion, Semantic Perception, Imitation Learning, Hier- archical Control
Introduction
To operate in complex offroad environments, is crucial for quadrupedal robots to adapt their mo- tion based on the perception of the terrain ahead. When encountering new terrains, the robot needs to identify changes in key terrain properties, such as friction and deformability, and respond with the appropriate locomotion strategy to maintain reasonable forward speed without incurring failures. In many cases, information about such terrain properties is more easily inferred from a terrain's 's semantic class (e.g. grass, mud, asphalt, etc.) instead of its geometric shape (e.g. slope angle, smoothness) [1, 2]. However, recent works in perceptive locomotion [3, 4, 5, 6, 7, 8, 9] mostly focus on the geometric aspect of the terrain, and do not make use of such semantic information.
In this work, we present framework for quadrupedal robots to adapt locomotion behaviors based on perceived terrain semantics. The central challenge in learning such semantic-aware locomotion controller is the high cost of data collection. On the one hand, while simulation has become an effee- tive data source for many robot learning tasks, modeling the complex contact dynamics accurately and rendering photorealistic offroad terrains is not yet possible in simulation. On the other hand, data collection in the real world is time-cone consuming and requires significant human labor. Moreover, the robot needs to remain safe during the data collection process, as any robot failure can cause significant damage to the hardware and surrounding environment. Therefore, it is difficult to use standard reinforcement learning methods for this task.
Our framework addresses all concerns above, and learns semantics-awan locomotion skills directly in the real world. To reduce the amount of data required, we pre-train a semantic segmentation network an off-road driving dataset and extract a semantic embedding from the model for further fine-tuning. To avoid policy exploration in real-world environments, we collect speed choices from human demonstrations and train the policy using imitation learning [10]. Additionally, inspired by previous results on the relationship between speed and gait in animals [11] and legged robots
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

[12, 13], we pair the speed policy with gait selector to further improve the robot's stability. With the pre-trained image embedding, the imitation learning setup, and the gait selector, our framework learns semantics-aware locomotion skills directly in the real world safely and efficiently.
We deploy framework on an A1 quadrupedal robot from Unitree [14]. Using only 40 minutes of human demonstration data, our framework learns semantics-aware locomotion skills that can be directly deployed for offroad missions. The learned skill policy inspects the environment and selects fast and robust locomotion skill for each terrain, from slow and cautious stepping on heavy pebbles to fast and active running on flat asphalts. The learned framework generalizes well and operates without failure on a number of trails not seen during training (over 6km in total). Moreover, our framework outperforms the manufacturer's default controller in terms of speed and safety. We further conduct ablation studies to justify the important design choices.
The technical contributions of this paper include the following:
1. We develop hierarchical framework that adapts locomotion skills from terrain semantics.
2. We propose safe and data-efficient method to train our framework directly in the real world, which only requires 40 minutes of human demonstration data.
3. We evaluate the trained framework on multiple trails spanning 6km with different terrain types, where the robot reached high speed and walked without failures.
2 Related Works
Perception for Legged Robots Creating perceptive locomotion controller is a critical step en- able legged robots to walk in offroad, unstructured environments. Most importantly, it allows robots to detect and react to terrain changes proactively before contact. Many prior works have focused on understanding terrain geometry from perceptive sensors [15, 16, 3, 7, 6]. However, such information can be insufficient as it does not reveal important terrain properties such as deformability or contact friction [8, 9, 17, 18]. To ameliorate this, recent works proposed to update this geometric under- standing of terrain with proprioceptive information [8, 9, 17]. However, these methods sacrifice proactivity, as the update cannot happen until after the robot has stepped on the terrain.
Another approach is to infer the terrain properties from its semantics [19, 20, 21, 22] that the robot can detect changes in terrain property before contact, and select its locomotion strategies proactively. Recently, Suryamurthy et al. [23] trained models to predict terrain class and roughness and used the prediction to modulate the height and navigational path of a wheel-leg hybrid robot in an indoor environment. Our framework uses similar semantics- based approach in the perception module and extends the result to off-road environments with a wide variety of terrains by adapting both the speed and gait of the robot.
Terrain Traversability Estimation The goal of our perception module is to assess the traversabil- ity of the terrain ahead of the robot. Researchers have proposed number of approaches to estimate traversability from perception data, including manually designed [24], learned from self-exploration [16, 25], or learned from human demonstration [26]. While learning- more flexibility, they usually require large amounts of data, which is difficult to collect in the real world. As a result, most approaches rely on simulation [27] as a source for training data. However, simulation is not feasible for task, as it is currently difficult to accurately model the complex contact dynamics and create photorealistic renderings off-road environments. Unlike previous approaches, our framework can be trained directly in the real world and requires only 40 minutes of human demonstration data.
approaches provide
Motion Controller Design for Perceptive Locomotion Another important question in perceptive locomotion is the design of a motion controller that effectively makes use of the perceptive infor- mation. A common strategy is to create a low-level motion controller that plans precise foothold placements based on the perceived terrain [3, 4, 5, 6, 7]. While these methods have shown good results in highly uneven terrains, the high computational cost required for terrain understanding and rapid planning makes it infeasible for complex offroad environments. In this work, we devise novel way to interface berween perception and low-level motion controllers for legged robots, where the high-level perception model outputs the desired locomotion skills, including forward speed and

---

Tuming
Speed
Policy
Forward Speed
Speed Policy
Forward Speed
Convex MPC
Gait Selector
Parameters
Proprioceptive
Metor Command
Camera Image
High Level Skill Palicy (3Hz)
Low Level Motar Controllar (400Hz)
Figure 1: Our framework consists a high-level skill policy a low-level motor controller. The skill policy selects locomotion skills (gait and speed) based on camera images. The low-level controller computes motor commands for robot control.
Speed Map
FCAReLU
Crop Average
Forward
Speed
HxWx1
Hidden
Semantic Embedding
Semantic Segmentation
Layers
HX
x 48
FCHarDNet
Pre- Trained on RUGD Dataset
Trained on
Robot Dataset
Figure 2: Architecture of our perception model. We extract semantic embedding from pre-trained semantic segmentation network and use it learn and predict forward speeds.
robot gait, low-level motor controller. With our framework, the robot can select safe and fast walking strategy for different terrains, which is crucial for offrrod traversal.
3 Overview
Our hierarchical framework (Fig. 1) consists of a high-level skill policy and low-level motor con- troller. At the high level, the skill policy receives the RGB image stream from the onboard camera and determines the corresponding locomotion skill. Each skill consists of desired forward speed and a corresponding locomotion gait, which are computed by the speed policy and gait selector, respectively. We train the speed policy using imitation learning from human demonstrations and manually design the gait selector to find the appropriate gait for each forward speed. At the low level, convex MPC controller [28] receives the skill command from the skill policy and computes motor commands for robot control. In addition, the convex MPC controller can optionally receive steering command from external teleoperator, which specifies the desired turning rate.
4 Learning Speed Policies
In unstructured offroad terrains, is crucial for robot adjust its speed response terrain changes so that it can traverse through different terrains efficiently and without failure. To achieve that, we design speed policy, which computes the desired forward speed the robot based on camera images. We train the speed policy using two-staged procedure: First, we pre-train a mantic embedding from an offline dataset. After that, we collect human demonstrations and train the speed policy using imitation learning.
3

---

4.1 Pre-trained Semantic Embedding
To reduce the amount of real-world data required train the speed policy, we pre-train semantic segmentation model and extract a semantic embedding for subsequent finetuning. We implement the model based on FCHarDNet-70 [29], which is a compact fully-convolutional encoder-decoder architecture with good real-time performance. We pre-train the model on the RUGD dataset [30], an off-road driving dataset with pixel-wise semantic labels (grass, dirt, rock, etc.). We choose RUGD because of its similarity to the images collected by the robot camera.
The next step extract an embedding from the pre-trained FCHarDNet [29] model for finetuning on robot data. Although the pre-trained model performs well on the RUGD dataset, its predicted segmentation becomes less accurate on robot images due to distribution shift. Meanwhile, the output of the hidden layers still provides continuous semantic description for each pixel. Therefore, we extract a semantic embedding from the output of the last hidden layer in FCHarDNet. which assigns a 18-dimensional embedding vector to each pixel in the input image (Fig. 2). We then compute a speed map by feeding the embedding of each pixel through fully-connected layer and compute the forward speed by averaging over a fixed region at the bottom of the speed map, which roughly corresponds to a rectangular area 1m long, 0.3m wide in front of the robot. The speed map provides straightforward and intuitive way to understand the model's predictions and can be used in navigational tasks such as path planning.
4.2 Learning Speed Commands from Human Demonstration
Even with the pre-trained semantic embedding, finding the appropriate speeds for offroad terrains using reinforcement learning is still challenging due to omnipresent noise and safety concerns in the real world. As an alternative, human operator can readily assess the robot's stability and adjusts the speed command accordingly based on the operator's previous experience with the robot platform. Therefore, we collect speed commands from human demonstrations and train the speed policy using imitation learning.
We collect human demonstrations by tele-operating the robot on variety of terrains, including asphailt pebble, grass and dirt. During data collection, the operator gives speed commands using joystick, while other components of the pipeline, such as the gait selector and motor controller, function accordingly (Fig. 1). Each time the camera captures a new image, we store the image and the corresponding speed command. We then train the speed policy using behavior cloning [10), where the objective to minimize the difference between predicted speed and human command.
Speed-based Gait Selector
In addition to speed, the gait of legged robot, such as its foot swing height, can greatly affect its traversability, especially on uneven terrains. While the perception policy can output speed and gait parameters jointly, training such a policy using imitation learning can be challenging, as it difficult for the human operator to demonstrate speed and gait choices at the same time. Meanwhile, previous studies in animal [11] and robot [12, 13] locomotion have revealed close connection between speed and gait choices. Inspired by this discovery, we simplify the demonstration and learning process by designing heuristic-based gait selector, which computes the appropriate gait parameters based on desired forward speed.
Gait Parameterization In our design, each gait is parameterized by three parameters, stepping frequency (SF), swing foot height (SH), and base height (BH). The stepping frequency (SF) de- termines the number of locomotion cycles each second. Similar to [12), we adopt a phase-based parameterization for gait eycles, where each leg alternates between swing and stance. In addition, we assume a trotting pattern for leg coordination, where diagonal legs move together and are 180 out-of-phase with the other diagonal. The trotting pattern is known for its stability, thereby being the default gait choice in most quadrupedal robots [28, 14]. The swing foot height (SH) determines the leg's 's maximum ground clearance in each swing phase. While higher swing height improves stabil- ity on uneven terrains by preventing unexpected contacts, a lower swing height is usually necessary for high-speed running. The base height (BH) specifies the height of the robot's center-of- -fass... While a low base height gives better stability at high speeds, a higher base height can be beneficial when traversing through unknown obstacles.

---

Stegping (SFIHZ
0.20
0.30
Stepping Frequency (SF)
3.75
00188
Swing Foot HeighL (SH)
0.29
Swing Foot Height (SH)
Height
3.50
0.28
0..6
3.25
0.27
Ba5e Height
0.14
3.00
0.26
0.25
2.75
0.24
0.4
0.6
0.8
1.6
Speed/ims))
Figure 3: The gait selector selects gait parameters (SF, SH and BH) based on desired forward speed. For example, when the desired speed is 0.5m/s, the speed selector would choose a stepping frequeney 2.8Hz, a swing foot height of 0.16m, and a base height of 0.29m.
Speed-Based Based Gait Selection We use empirical evidence to design the speed-based gait selector, which finds a gait with high traversability for each speed. More specifically, for the boundary speeds (0.5m/s and 1.5m/s), the lowest SF that would still ensure base stability (2.8Hz and 3.5Hz). After that, we sweep over different values of SH and BH find the highest value of both that would allow the robot to walk robustly without falling. Lastly, we linearly interpolate the parameter values between the boundary speeds to find the gait for intermediate speeds. See Fig. for details.
first try different SFs with nominal SH (0. 12m) and BH (0.26m), and find
6 Low-level Convex MPC Controller
The low-level convex MPC controller computes and applies torques to each actuated degree of freedom, given the locomotion skills from the skill policy. Our low-level convex MPC controller is based on Di Carlo et al. [28] with two important modifications. Firstly, due to the robot's s small form factor, it needs to constantly re-orient its body on uneven terrains, such as bumps and potholes. Therefore, implemented state estimator to estimate the ground orientation and adjust the robot pose to fit the ground, similar to Gehring et al. [31]. Secondly, to reduce foot slipping, we implement impedance controller for stance legs [32]. In addition the motor torque command computed by MPC, the impedance controller adds a small feedback torque to track the leg in its desired position. We found both techniques to improve locomotion quality significantly. Please refer Appendix A for details.
7 Experiment and Result
To see whether our framework can learn to adapt locomotion skills based on terrain semantics, we deploy it to quadruped robot and test it in a number of outdoor environments in the real world. We aim to answer the following questions our experiments:
1. Can our framework operate without failure in complex offroad terrains for extended periods time, and how does it compare with existing baselines?
2. How does our framework generalize to terrain instances not seen during training?
3. Can our framework walk high speed while ensuring safety?
4. What are the important design choices in training the perception module, and how does it affect performance?
7.1 Experiment Setup
We implement framework on Al quadrupedal robot from Unitree [14]. We equip the robot with an Intel Realsense D435i camera to capture RGB images and a GPS receiver to track its real- time location. We implement the entire control stack in the Robot Operating System (ROS) frame- work [33] and deploy it on a Mae Mini with MI chip, which mounted on the robot. The convex 3Hz.
MPC controller runs at 400Hz, and the speed policy and gait selector
To train the speed policy, we collected 7239 frames of data on a variety of terrains, which corre- sponds 40 minutes of robot operation. The entire process, including robot setup, data collection,
5

---

Shalow Grocs
Gracel
Grawi
1.50
Deep
Light Grawi
Shiilin
0.75
Forward Speed (mls)
avail
Light Gravel
Asphaik
Figure 4: The 450m-long test trail consists of multiple terrain types such deep grass, shallow grass, gravel, and asphalt. The learned skill policy adjusts the speed and gait hased on terrain semantics and walks faster on casier terrains.
Policy Type
Speed (m/s)
Gait Params (SF, SH, BH)
Traversal Time
Number of Failures
(min)
Fixed-Slow
0.5
[2.8, 0.16, 0.29]
15
0
Fixed-Mean
0.8
[3.0, 0.15, 0.28]
00
3
Mixed-Medium
1
[3.1, 0.14, 0,28]
00
4
Fixed-Fast
1.5
[3.5, 0.12, 0.27]
00
10+
Speed-Only
Adaptive 0.8
[3.1. 0.14, 0.28]
9
Gait-Only
Adaptive
00
Unitree-Normal
Tele-operated
N/A
11+00.4
0
Snotree-Sport
Tele-operated
N/A
00
2
Fully-Adaptive (ours)
Adaptive
Adaptive
9.6.E..2
0
Table I: Performance different policies on the test trail (450m). Compared to other policies, our framework
completes the entire trail without failure in the shortest time. We repeat the Unitree-Normal and Fully-Adaptive
policies 3 times and report the mean and standard deviation of the traversal time. We do not repeat the other policies due excessive robot damage.
and battery swaps, took less than an hour. The speed policy is trained on standard desktop com- puter with an Nvidia 2080Ti GPU, which took approximately 20 minutes to complete.
7.2 Fast and Failure-free Walking on Multiple Terrains
To evaluate the adaptivity of our framework, test our framework tiple terrain types, including deep grass, shallow grass, gravel, and asphalt (Fig. 4). Our controller switches between wide range of skills traverses through the trail, from slow and careful step- ping to fast and active walking, and completes the 450m-long trail in 9.6 minutes, comparable to the performance of human demonstrations (10 minutes).
outdoor trail with mul-
We compared our learned framework with the following baselines on the same test trail (Fig. 4), including Unitree's 's built-in controllers and variants of our controller with no or limited adaptation. The result is summarized in Table. 1. Please refer to Appendix. B.2 for further details.
Unitree's built-in controllers We tested two modes of the built-in controller, normal mode (Unitree-Normal) that walks up to 1m/s, and a sports (Unitree-Sport mode that walks up to 1.5m/s. Both controllers do not include perception and assume a fixed gait all times. Although normal mode completed the entire trail without failure, it walked slower than our learned framework, espe- cially on asphalts, due to limitations on the maximum speed. On the other hand, the sports mode controller failed to complete the course and got stuck in deep grass twice due to insufficient swing foot clearance.
Fixed Skill with No Adaptation For these baselines, we disabled the perception module and operated the robot with a fixed locomotion skill. We tested four skills, namely slow, mean, medium, and fast, operating at 0.5m/s, 0.8m/s, Im/s, and 1.5m/s, respectively, with the corresponding gait selected according to Fig. 3. The slow, medium and fast skills cover the range of possible speeds achievable by our lowelevel controller, and the mean skill walks at speed similar to the average speed

---

Slow
0.5m/s
0.75m/s
1mis
1.255/ls
1.5mls
Fast
Figure 5: Desired speed computed by the skill policy. The policy prefers faster skills for rigid and flat terrains and prefer
Rock
Pebble
Asphalt
(a) We test robot different terrain types.
Rock
Pebble
Grass
Asphalt
Success fate5%
100%
100%
100%
100%
50%
50%
1.0
spaad/(must
speadk(nta)
spead)
(b) Success rate of fixed locomotion skills (bluc) and the learned policy (red).
Figure 6: Our framework learns fast and safe locomotion skills. Top:
We deployed our skill policy to 4 different terrains. Bottom: Our policy finds a high speed in the safe region of cach terrain.
achieved by our adaptive policy (0.78m/s). The mean, medium, and fast skills failed to complete the trail and incurred failures. While the slow skill completed the trail without failure, its traversal time 50% longer than our learned framework.
Adapt Speed or Gait Only In our framework, we design a robot skill to be a combination of gait and forward speed. To justify this design, design two policies, where the robot adapts the gait or the forward speed only. For the speed-only policy, we fix the gait parameters as if the forward speed 1.0m/s Fig. 3 and adapt the speed using our framework. For the gait-only policy, we fix the base speed to be 0.8m/s, similar to the average speed attained by our learned policy, and adapt the gait using our framework. Both policies failed to complete the trail. For the speed-only policy, we found the fixed gait to only work well when the base speed was close to Im/s and frequently failed either higher or lower speeds. For the gait-only policy, the robot managed to walk through most of the trail but slipped twice on rocky terrains.
7.3 Generalization to Unseen Terrain Instances
To further test the generalizability of our framework, we deploy the robot on number of outdoor trails not seen during training. The trails contain diverse terrain types, such as dirt, gravel, mud, grass, and asphalt. The robot traverses through these test trails without failure and adjusts its loco- motion skills based on terrain semantics. Please refer to Appendix. B.1 for details. To demonstrate the skill choices of our framework, we select few key frames from the camera images and plot the corresponding speed in Fig. 5. Generally, the skill policy selects faster skill on rigid and flat terrains and slower speed on deformable or uneven terrains. At the time of writing, the robot has traversed through over 6km of outdoor trails without failure.
7.4 Analysis on Speed and Safety
To test the performance of the learned skill policy in terms of speed and stability, we deploy the leamedd skill policy on four different terrains, including rock, pebble, grass, and pebble (Fig. 6a). We compare our semantics-aware skill policy with 5 fixed skills, where the speed linearly interpolates between 0.5m/s and I 1.5m/s. For each speed, the corresponding gait is selected according to Fig. 3. For each terrain and skill combination, we repeat the experiment times and report the success rate, where trial is considered successful if the robot does not fall over during the traversal (Fig. 6b).
7

---

By comparing the success rate at different speeds, we obtained an approximation of the safe speed range for each terrain. We then test the performance of our framework by comparing the average speed obtained by our learned skill policy on each terrain against these safe speed ranges.
The maximum safe speed varies significantly different terrains. For example, while the robot can walk up to 1.25m/s without failure on asphalt, it can only walk up to 0.5m//s on rock, due to unexpected bumps and foot slips on the surface. Although not directly optimized for speed robot safety, our learned policy finds a close-to-maximum speed in the safe region of each terrain after learning from human demonstrations. We also noted that on pebble and grass, there is a slightly larger gap between the maximum safe speed and the speed selected by the skill policy. One reason for this is that the speed demonstrated by the human operator can be more conservative than the maximum safe speed.
7.5 Ablation Study on Perception Module
We compare our way of training the perception-enabled speed policy with few baselines, which either train the policy from scratch without pre-training or extract the pre-trained embedding directly from the predicted semantic classes. We find that our policy, which is fine-tuned from the output of the hidden layer, achieves the smallest error on the validation set and predicts the speed map with high precision. Please see Appendix B.3 for further details.
Limitations and Future Work
In this work, we present a hierarchical framework to learn semantic-aware locomotion gaits from human demonstrations. Our framework learns to adapt locomotion skills for variety of terrains using 40 minutes of human demonstration and enables robot to traverse over 6km of outdoor terrains without failure. One limitation of our framework is that, while our robot walks robustly on variety of off-road terrains, its performance is limited by the low dimensionality of human demonstrations. For more difficult terrains such as steps or gaps, the robot will need more agile behaviors such as jumping, which requires deeper integration between the perception system and low-level motor controller and learning more skills than speed or gait demonstrations. Another limitation is that the perception system assumes that there the robot and therefore does not adjust the heading of the robot. In future work, we plan to increase the agility of controller and integrate path planning into our framework that the robot can operate fully autonomously in challenging off-road environments.
non-traversable obstacles ahead of
8

---

References
[1] J. Gibson. The ecological approach to visual perception: classic edition. Psychology press, 2014.
[2] Y.-W. Chao, Z. Wang, R. Mihalcea, and J. Deng, Mining semantic affordances of visual object categories. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4259-42565 2015.
[3] W. Yu, D. Jain, A. Escontrela, A. Iscen, P. Xu, E. Coumans, S. Ha, J. Tan, and T. Zhang. Visual- locomotion: Learning to walk on complex terrains with vision. In 5th Annual Conference on Robot Learning, 2021.
[4] S. Gangapurwala, M. Geisert, R. Orsolino, M. Fallon, and I. Havoutis. Rloc: Terrain- aware legged locomotion using reinforcement learning and optimal control. arXiv preprint urXiv:2012.03094 2020.
[5] P. Fankhauser, M. Bjelonic, C. D. Bellicoso, T. Miki, and M. Hutter. Robust rough-terrain locomotion with a quadrupedal robot. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 5761-5768 IEEE, 2018.
[6] O. Villarreal, V. Barasuol, P. M. Wensing, D. G. Caldwell, and Semini. Mpc-hased controller with terrain insight for dynamic legged locomotion. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 2436-2442. IEEE, 2020.
[7] Jenelten, T. Miki, A. E. Vijayan, M. Bjelonic, and M. Hutter. Perceptive locomotion in rough terrain-online foothold optimization. IEEE Robotics and Automation Letters, 5(4):5370-5537, 2020.
[8] T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning robust per- ceptive locomotion for quadrupedal robots in the wild. Science Robotics, x(62)eab 2022.
[9] Z. Fu, A. Kumar, A. Agarwal, H. Qi, Malik, and D. Pathak. Coupling vision and pro- prioception for navigation of legged robots. CoRR, abs/21 12.02094, 2021. URL https: // laaaiivvrgg///aialllaaaa.aa..lle abaaaba//1111 /atp//21...000 02094.
[10] D. A. Pomerleau. Alvinn: An autonomous land vehicle in a neural network. Advances in neural information processing systems, 1, 1988.
[11] D. F. Hoyt and C. R. Taylor. Gait and the energetics of locomotion in horses. Nature, 292 (5820):239-244 1981.
[12] Y. Yang. T. Zhang. E. Coumans, J. Tan, and B. Boots. Fast and efficient locomotion via learned gait transitions. In Conference on Robot Learning, pages 773-783. PMLR, 2022.
[13] Z. Fu, A. Kumar, J. Malik, and D. Pathak. Minimizing energy consumption leads to the emer- gence of gaits in legged robots. arXiv preprint arX1v:2111.00167,, 2021.
[14] Al Website. URL https: : //www. uni mairree.er http://wwwnn.eeeeeeorrr/rroaaaaa cowpproduccss/////
[15] Fankhauser, M. Bloesch, and M. Hutter. Probabilistic terrain mapping for mobile robots with uncertain localization. IEEE Robotics and Automation Letters, 3(4):3019 30266, 2018.
[16] J. Frey, D. Hoeller, S. Khattak, and M. Hutter. Locomotion policy guided traversability learning using volumetric representations of complex environments. arXiv preprint arXiv:2203. 15854, 2022.
[17) R. Yang, M. Zhang, N. Hansen, H. Xu, and X. Wang. Learning vision-guided quadrupedal lo- comotion end-to-end with cross-modal transformers. International Conference on Learning Representations, 2022. URL
id=nhn.J3oo6AF www.:/eettfoorioeeoo nett//orrumiiimmm..Booo....
[18] L. Wellhausen, A. Dosovitskiy, R. Ranftl, K. Walas, C. Cadena, and M. Hutter. Where should i walk? predicting terrain properties from images via self-supervised learning. IFFE Robotics and Automation Letters, 152::100--1166 2019.

---

[19] J.-F. Lalonde, N. Vandapel. D. F. Huber, and M. Hebert. Natural terrain classification using diree-dimensional ladar data for ground robot mobility. Journal of field robotics, 23(10):839 861, 2006.
[20] Y. N. Khan, P. Komma, and A. Zell. High resolution visual terrain classification for outdoor robots. In 2011 IEEE International Conference on Computer Vision Workshops (ICCV Work- shops), pages 1014-1021, IEEE, 2011.
[21] F. Schilling, X. Chen, J. Folkesson, and P. Jensfelt. Geometric and visual terrain classification for autonomous mobile navigation. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2678-2684. IEEE, 2017.
[22] A. Shaban, X. Meng, Lee, B. Boots, and D. Fox. Semantic terrain classification for off-road autonomous driving. In Conference on Robot Learning. pages 619-62) PMLR, 2022.
[23] V. Suryamurthy, V. S. Raghavan, A. Laurenzi, N. G. Tsagarakis, and D. Kanoulas. Terrain seg- mentation and roughness estimation using rgb data: Path planning application on the centauro robot. In 2019 TEEE-RAS 19th International Conference Humanoid Robots (Humanoids), pages 1-8. IEEE, 2019.
[24] D. D. Fan, K. Otsu, Y. Kubo, A. Dixit, J. Burdick, and A.-A. Agha-Mohammadi Step: Stochastic traversability evaluation and planning for risk-aware off-road navigation. In Science and Systems, pages 1-21. RSS Foundation, 2021.
Robotics:
[25] K. Walas, D. Kanoulas, and P. Kryczka. Terrain classification and locomotion parameters adap- tation for humanoid robots using force/torque sensing. In 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), pages 133-140. IEEE, 2016.
[26] X. Cai, M. Everett, J. Fink, and J. P. How. Risk-aware off-road navigation via learned speed distribution map. arXiv preprint arXiv:2203. 13429, 2022.
[27] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun. Carla: An open urban driving simulator. In Conference on robot learning, pages 1-16. PMLR, 2017.
[28] J. Di Carlo, P. M. Wensing, B. Katz, G. Bledt, and S. Kim. Dynamic locomotion in the mit cheetah 3 through convex model-predictive control. In 2018 IEEE/RSJ international confer- ence on intelligent robots and systems (IROS), pages 1-9, IEEE, 2018
[29] Chao, C.-Y. Kao, Y.S.. Ruan, C.-H. Huang, and Y.-L. Lin. Hardnet: low memory traffic network. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3552-3561, 2019.
[30] M. Wigness, S. Fum, Rogers, D. Han, and H. Kwon. rugd dataset for autonomous navigation and visual perception in unstructured outdoor environments. In International Con- ference on Intelligent Robots and Systems (IROS), 2019.
[31] C. Gehring, C. D. Bellicoso, S. Coros, M. Bloesch, P. Fankhauser, M. Hutter, and R. Sieg- wart. Dynamic trotting slopes for quadrupedal robots. In 2015 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5129-5135. IEEE, 2015.
[32] F. Jenelten, J. Hwangbo, F. Tresoldi, C. D. Bellicoso, and M. Hutter. Dynamic locomotion on slippery ground. IEEE Robotics and Automation Letters, 4(4):41 77--476,. 2019.
[33] M. Quigley, K. Conley, B. Gerkey, J. Faust, Foote, Leibs, R. Wheeler, A. Y. Ng, al. Ros: an open-source robot operating system. In ICRA workshop on open source software, volume 3, page Kobe, Japan, 2009.
10


Reviews:
[{"review_id": "KOz1rC1aNLSM", "reviewer": "Reviewer_FzJu", "summary": "This paper presents a framework to learn a set of perceptive locomotion skills. The key idea of this work is to leverage existing offroad driving dataset for quadrupedal offroad navigation tasks. ", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- A very clean and neat framework to enable perceptive locomotion skills\n- The use of existing datasets from other domains for the target downstream tasks\n- Both quantitative and qualitative evaluation of the approach, especially the robustness of the models.\n\n\nWeakness:\n- Line 96: Does this mean that authors assume there are no obstacles in terrain that would stop robots from traversing at all? The limitation section mentions future work of navigation, but that would require the robot to reorient its heading directions. This should also be mentioned in the limitations.\n- Line 122: Authors should provide more intuition on why using FC on each pixel instead of using convolutional layers over pixels and directly output the forward speed. What is the intuition behind having this intermediate speed map?\n- For real robot videos, it would be even better if there is a list of average speed over different domains. When watching the multi-terrain type video, it’s hard to tell if robots are executing at the same speed as individual evaluation. \n- Also, since the locomotion skills are not high-speed behaviors, it’s still difficult to tell the speed differences by playing each video individually. Maybe putting all video clips on one single canvas would give a better idea of the gaits’ differences.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I strongly recommend to accept this publication with the following two reasons: 1) it shows how to leverage existing datasets for robotics research. It is notoriously known that it’s hard to collect large-scale datasets for robot control as contrast to existing vision or nlp datasets. This work shows that we can leverage large datasets for robot control. And specifically, it’s a novel for the perceptive locomotion community. 2) It shows a working locomotion system that is very impressive. The no-cup video clip of full trail locomotion strongly shows the robustness of the approach. I think it’s quite rare to see in the robotics community nowadays, and it sets up a good example of what is a solid robot learning approach is. ", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "Aside from the weakness mentioned, several comments on the details of writing:\n- Line 25: Here, although it seems intuitive, lacks citations from either perception literature or previous works to support this claim. For example, maybe JJ Gibson's Affordance chapter could support this claim.\n- Section 6: Since the MPC controller is from previous works, I suggest putting it into a single paragraph instead of a single section. \n- Line 172: \"design\" or \"implement\"? Maybe authors say \"design\" because they are different from how previous works implement impedance control for locomotion. I am confused if this is a novel design or an adaptation from a previous approach.  \n- Line 182: I think unseen terrains refer to unseen \"instances\" of terrain types. It's ambiguous about \"unseen terrains.\"\n- Table 1: I think it's important to say what's the length of a single trail, as the authors only report a total number of lengths (6km) while the traverse time says around 10 min, which was quite surprising at first. Stating the length of trails themselves would help readers understand the result much clearer.\n"}}, {"review_id": "BOpj-hK59AA", "reviewer": "Reviewer_ofNu", "summary": "The paper presents a quadruped robot locomotion method, where MPC-based locomotion is enhanced with visual-based terrain type identification that drives correctly speed and gait. The whole method learns from human demonstrations on the real hardware.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- This is a clever work. It is a basic concept, i.e., when moving on different terrains, the speed and gait might need to adapt accordingly.\n- The technical part is easy to follow and sound -- having said that the development part is not extremely challenging.\n- The experiments are overall nice, with real-world locomotion on various terrains.\n\nWeaknesses:\n- Abstract: \"close-to-optimal speed\" is a bit of an exaggeration. It has not been shown anywhere in the results what is the optimal, and thus this sentence is not supported. Moreover, I have the feeling that it is also not true.\n- Intro/RW: the intro and later the related work, lacks some work on terrain semantic classification for locomotion, e.g., \"Terrain Classification and Locomotion Parameters Adaptation for Humanoid Robots Using Force/Torque Sensing\", by K. Walas et al. (IJHR'16) for proprioception and \"Terrain Segmentation and Roughness Estimation using RGB Data: Path Planning Application on the CENTAURO Robot\", by V. Suryamurthy et al. (IJHR'19) for exteroceptive segmentation.\n- Intro: \"human expert demonstration\" there is a big discussion now how those humans became experts and what does this mean?\n- Intro: \"Fast and Failure-free: this is again very vague and generic. In best case, terrain types need to be mentioned at least. Otherwise, this does not have any particular meaning.\n- RW: In the end of the RW section, the selection to learn from real demonstration is highly proposed. Although, we know that learning in simulation (e.g., in isaac) achieves very good results without the need to adapt anything on the physical robot (e.g., the work on ANYmal in the latest Science publication). It needs to be justified better why in this case simulations could not do the job.\n- Sec 4: \"instead, we extract a semantic embedding\" this is very vague and needs to be explained further. Why the semantic segmentation is not good, but the embedding is? Why it is feeded via a fully connected layer? More justification is needed.\n- Sec. 4: \"1m long, 0.3m wide in front of the robot.\" what if the camera changes orientation or position. How much of the method can work as is?  How much retraining is needed?\n- Sec. 4: \"using imitation learning\": it is still confusing how the humans decided the \"optimal\" speed for each type of terrain.\n- Sec. 5: the speed-gait selection seems to be very tedious and not extendable easily to other robots. Any alternative? Any less heuristic approach?\n- Sec. 6: slope handling is claimed but none of the experiments include slopes. Not sure if it is good to be mentioned if it was not tried at all.\n- Sec. 7: you compare with Unitree's controller, but in reality [21] should be your baseline. How much better than [21] do you perform?\n- Sec. 7: \" it can only walk up to 0.5m/s on rock\", how was this evaluated? What if different gaits were applied? \n- Sec. 7.3: \"traverses through all of them without failure\", this is very generic and probably not true. Unless the authors claim we have found the optimal MPC-based control to deal with all terrains. In other words, a specification of gait and terrain type is needed to justify this.\n- General: What happens when you traverse from one terrain type to another? How the robot adapts to the new gait and speed? What happens when terrain segmentation is noisy, and what happens when the robot walks on a misclassified terrain?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I believe the work requires some improvement on the weaknesses mentioned above, but in general it is nice. The main problem is overselling the approach. But if we ignore this part, the paper appears to be solid, sound, easy to follow, and nice.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "All weaknesses above need to be handled. Most are on the writing part and I think one is on the experimental part (slopes).", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {}}, {"review_id": "9Mx6uKDyNyW", "reviewer": "Reviewer_Rr19", "summary": "The paper proposes a system which learns to adapt walking speed based on semantics of terrain. The proposed approach learns, from human demonstration, to regulate forward speed as a function of RGB input. The model is pretrained on a semantic segmentation task and optimized via behavior cloning for the speed control problem. A gait selector is manually designed as a function of desired walking speed. The result is a system which is capable of traversing rough terrain more efficiently than the built-in Unitree controllers. ", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths - \n\nThe paper studies an important problem - clearly some amount of semantic information is important for locomotion in offroad environments. Existing works typically fail to adequately address this in that they only model the geometry of the surface rather than material properties like friction, or by being entirely reactive. Thus the paper suggests an interesting innovation. \n\nWeakness - \n\nThe learning problem proposed is perhaps too simple - controlling speed alone is likely insufficient for more challenging terrain, as illustrated in recent papers which additionally learn to regulate gait (e.g. the Hutter Science paper cited). It seems as though some combination of these ideas is required for robust locomotion, controlling both the desired forward speed and the parameters of the gait together as a function of anticipated terrain properties and proprioceptive data. That the method never fails on the test domains suggests that perhaps the tests are not sufficiently challenging. Moreover, comparison to a method which is purely geometry-based would be instructive in evaluating whether the semantic module indeed constitutes a substantial improvement in performance in the domain studied (e.g. even a simple variant without the semantic segmentation pretraining). \n\nAdditionally, the reliance on human demonstrations may present a challenge in scaling the method. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I'm recommending weak rejection on the basis of the simplicity of the learning problem posed and the relatively limited experimental evaluation. ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "Abstract: consider changing \"terrain type and property, reveals\" to \"...properties, reveal\"", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {}}, {"review_id": "1hMQ889MRR", "reviewer": "Reviewer_oYTQ", "summary": "This paper presents an architecture for adapting the locomotion of a legged robot in response to semantic terrain information. A human teleoperates the robot with a joystick to provide scalar velocity labels associated with different terrains. A mapping from RGB images to velocities is learned using the features of a pretrained vision module to promote generalization. Then, velocity commands are heuristically mapped to gait parameters, and the resulting locomotion skill is actuated using an MPC controller from prior work. The complete system enables the robot to automatically run faster on easier terrains and walk carefully on harder terrains. The system can complete novel trails in this manner without failure.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: The perception of semantic terrain information is a promising research area for locomotion, and this paper is the earliest known functional proposal. The proposed architecture is novel and demonstrates nice integration of several control components.\n\nWeaknesses: A main conclusion of the paper is drawn from a single trial of a high-variance metric.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The perception of semantic terrain information is a promising research area for locomotion, and this paper is the earliest known functional proposal. The proposed architecture is novel and demonstrates nice integration of several control components. However, a main conclusion of the paper is drawn from a single trial of a high-variance metric. The authors should either (a) increase the number of trials and provide details about each failure, or (b) choose a less brittle evaluation metric. Also, the architectural choices in the paper may have some limitations, which the authors should express clearly to invite follow-up work in this area.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "- A main result of the paper, the failure rate in Table 1, is only reported for one trial per policy. This metric seems to have high variance and is also exposed to influence by the human operator. For example, the \"unexpected bumps\" you refer to in Section 7.5, which I read as small but dangerous features like potholes, would cause high variance in the failure rate depending on whether the robot happens to step on them or not in that trial. Do you have some video showing those failures? Can you report more trials or propose a less brittle evaluation metric?\n- Also in Table 1, the Fixed-Medium controller has a much higher average speed than the Fully-Adaptive controller. It seems fair to add an evaluation of a fixed controller with equal avg speed (looks like 0.78m/s?)\n- Some potential limitations of this framework should be discussed in the paper. The low-level policy is limited in how it can adapt to terrain properties. For example, there is a fixed friction value in the model used for MPC controller. Consider that if this were tuned for each terrain, the locomotion on slippery terrains could be stable at higher speeds. But it's impractical for the demonstrator to provide a demo that captures this, or in general to demonstrate adaptation in anything more than one or two highly interpretable dimensions of gait.\n- During a discrete transition between terrains, something interesting happens. If transiting from challenging to easy terrain, the robot needs to speed up *after* it leaves the challenging terrain. If transiting the opposite way, from easy to challenging terrain, the robot must slow down *before* leaving the easy terrain. It seems like this behavior cannot be captured by your approach which is based on averaging in a fixed region. Please elaborate on this. "}}]

Author rebuttal:

Decision: Accept (poster)