TOP-Nav: Legged Navigation Integrating Terrain, Obstacle and Proprioception Estimation
Authors: Junli Ren
Yikai Liu
Yingru Dai
Junfeng Long
Guijin Wang
Venue: CoRL 2024 (2024)
Keywords: Navigation
Task Planning
Reinforcement Learning
TL;DR: We present a legged navigation system that achieves open-world navigation surpassing limitations posed by  visual conditions or prior knowledge by integrating terrain, obstacle and proprioception.

Abstract: Legged navigation is typically examined within open-world, off-road, and challenging environments. In these scenarios, estimating external disturbances requires a complex synthesis of multi-modal information. This underlines a major limitation in existing works that primarily focus on avoiding obstacles. In this work, we propose TOP-Nav, a novel legged navigation framework that integrates a comprehensive path planner with Terrain awareness, Obstacle avoidance and close-loop Proprioception. TOP-Nav underscores the synergies between vision and proprioception in both path and motion planning. Within the path planner, we present a terrain estimator that enables the robot to select waypoints on terrains with higher traversability while effectively avoiding obstacles. In the motion planning level, we construct a proprioception advisor from the learning-based locomotion controller to provide motion evaluations for the path planner. Based on the close-loop motion feedback, we offer online corrections for the vision-based terrain and obstacle estimations. Consequently, TOP-Nav achieves open-world navigation that the robot can handle terrains or disturbances beyond the distribution of prior knowledge and overcomes constraints imposed by visual conditions. Building upon extensive experiments conducted in both simulation and real-world environments, TOP-Nav demonstrates superior performance in open-world navigation compared to existing methods.

TOP-Nav: Legged Navigation Integrating Terrain, Obstacle and Proprioception Estimation
Junli Ren Yikai Liu",, Yingru D Junfeng Long', Guijin Wang
Tsinghua University. ?ShanghaiTech University
Abstract: Legged navigation has been widely applied in open-world, off-road, and challenging environments. In these scenarios, estimating external distur- bances requires complex synthesis of multi-modal information. This underlines major limitation in existing works that primarily focus avoiding obstacles. In this work, we propose TOP-Nav, novel legged navigation framework that inte- grates comprehensive path planner with Terrain awareness, Obstacle avoidance and close-loop Proprioception. TOP-Nav underscores the synergies between vi- sion and proprioception in both path planning and locomotion control. Within the path planner, we present terrain estimator that enables the robot to select waypoints on terrains with higher traversability while effectively avoiding obsta- cles. The locomotion controller tracks the planned waypoints and provides mo- tion evaluations the proprioception advisor. Based on the closed- looed-llop motion feedback, we offer online corrections for the vision-based terrain and obstacle es- timations. Consequently, TOP-Nav achieves open-world navigation that the robot can handle terrains or disturbances beyond the distribution of prior knowledge and overcomes constraints imposed by visual conditions. Building upon extensive ex- periments conducted in both simulation and real-world environments, TOP-Nav demonstrates superior performance in open-world navigation compared exist- ing methods. Project page top-nav nap-nae-eeged, gith.bb.....ithuubio.
Keywords: Navigation, Task Planning, Reinforcement Learning
1
Introduction
Defiult direction
Optimized direction
Law serrain
Highh-est lerain
Neod high-re leronn
Obsiacle
Figure 1: TOP-Nav achieves open-world navigation in both simulation and the real world. The robot plans obstacle-free path terrains with better traversability. The robot rapidly estimates its traversability for novel terrains based on proprioception experience.
Despite recent advancements in legged locomotion allowing the robots navigate various terrains based on simulation-learnee strong controller [1, 2, 3, 4, 5, 6, 7, 8, 9], the complexity of currently hard-to-sing simulate real-world factors makes it impossible for the robot to traverse all potential terrains encountered in reality. As a result, simply connecting the locomotion controller with a vision-based path planner often restricts legged navigation to limited scenarios [10, 11, 12, 13, 14, 15].
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

An effective solution to overcome these limitations is equipping the robot with terrain awareness: a traversable path can be planned based on the robot's preferences on terrains [16, 17] with an ap- propriate distribution of contact heights and forces [18]. Unlike obstacle estimation, the distinct terrain features are typically encoded as semantic information, whereas traditional methods collect sufficient data and train segmentation or classification models [19] to learn these features. Never- theless, compiling an exhaustive catalog of all conceivable terrains and their corresponding walking preferences is impractical [20]. Compounding the issue, the dynamic real-world conditions, such as lighting, humidity, and temperature, may introduce inaccuracies in the correspondence between images and walking preferences, especially when relying exclusively on vision in this context [21].
To address the mentioned challenges of relying solely on vision and ignoring motion states in path planning, we complement the vision-only terrain estimator with online corrections derived from mo- tion evaluations. We construct proprioception advisor convey information about the traversabil- ity cost of novel terrains and alert the robot to unexpected disturbances, such as invisible obstacles.
By integrating the Terrain estimator, Obstacle estimator, and Proprioception advisor, we formu- late TOP-Nav: a hierarchical path planning and motion control framework navigating a quadruped robot through diverse and challenging terrains proficiently. Within TOP-Nav, we develop terrain estimator trained from previously collected data to inform the robot of terrain traversability. For novel terrains, compensate proprioceptive history to offer online corrections for the vision-based estimation of unexpected and unknown environmental disturbances. We evaluate TOP-Nav both in simulation and on physical robot, with comparative analysis against existing legged navigation systems.
2 RELATED WORK
2.1 Vision and Legged Proprioception Integration
Vision-aided legged navigation has been extensively explored in existing literature [12, 11, 22, 14, 15], with performance heavily dependent on robust perception module [23]. This reliance makes transferring specific system to different hardware platforms difficult and costly. Recent research has introduced proprioception to improve task planning, provide comprehensive task observation, and reduce dependence on vision systems. A majority of these works learn proprioception represen- tations along with visual features in simulation and then implement the cross-modal features through end-to-end [24], hybrid [25] or decoupled frameworks [26]. Despite the effectiveness demonstrated these works, the high-dimensional representation space presents challenges for adaptation to novel scenarios and sim-to-real transfer. Alternatively, Fu al. [27] introduced a hierarchical navigation framework that derives evaluation scores from motion states, yet overlooks visual observation inte- gration. To mitigate these limitations, we propose novel approach within TOP-Nav by maintaining a series of lightweight cost maps derived from multi-modal observations. This integration achieves a dynamic balance between vision and proprioception. Furthermore, we leverage the learning- bbaed locomotion controller to derive motion evaluations from the value function, offering an efficient solution without additional training.
2.2 Terrain Traversability Estimation
Terrain traversability is determined by factors such terrain geometry, texture, and physical proper- ties [28]. These features could be estimated by identifying the semantic class with a predefined static traversability score [29, 19, 30, 16]. These solutions notably depend on large-scale datasets [31] or are limited to structured environments like urban scenarios [32, 33]. In oof-road navigation, the motion states involved in the dynamic interactions between the robot and the environment provide valuable metrics for assessing terrain traversability [34]. These insights have inspired methods that eliminate the need for manual annotation by autonomously deriving terrain traversability from pro- prioception through self-supervised learning [21, 35, 36, 37, 38, 39]. Nevertheless, the performance of these studies is contingent upon the quality of the collected datasets [28]. Researchers have

---

8; denotes the robot state, R(s;) represents the rewards accrued at timestep i, which commonly in- cludes guiding the robot to track a given velocity command with stable gaits and attitude. Substantial efforts in reward engineering formulate R(s;) have enabled the value function q evaluate comprehensive set of interactions between the robot and its environment. This provides an essential proposed various approaches to handling novel observations to emphasize the challenges in un- constrained navigation. For instance, Frey et al. [20] updated the traversability estimation network online with anomalies into consideration. Karnan et al. [40] performs nearest-neighbor search in the proprioception space to align visually novel terrains with existing traversability. Drawing inspi- ration from those works estimating traversability for novel terrains, we propose terrain estimator that employs the proprioception advisor online corrections. Our method diverges from previous approaches primarily in two key aspects: 1) We employ an estimated value function from reinforce- ment learning to assess terrain traversability, providing a comprehensive evaluation of robot-terrain interactions. 2) We make online corrections on the vision-based estimation without additional train- ing, providing a data-efficient solution for identifying novel terrains.
Background
Learning-based legged locomotion controllers have been well developed through reinforcement learning, which is generally achieved by updating the policy within the asymmetric actor-critic training:
ai Tactor (of o oi ot)) C₁ Teinn oj, oi o.
T1
(1)
the locomotion policy receives privileged observation of, proprioception observation of, scanned dots external observation of and historical observation of respectively. Tactor is modeled as Gaus- sian policy and infers the optimized actions a, to compute the joint positions Ades. Cr stands for the estimated value function from Tertic which is updated through:
T
Leiitic = (c: targ and tagg
R((s),
(2)
foundation for the proposed proprioception advisor to leverage the estimated e; for motion evalua- tions within the path planner.
The complete training paradigm will involve second stage, training a depth encoder and student network reproduce of and of from real-world accessible observations Ia (depth image), of and of The motion controller will track the target direction Ayaw and velocity command Vin, the control signal is represented by the desired joint position Odes.
4 Method
4.1
System Overview
TOP-Nav connects path planner and a motion controller tackle the task of legged navigation (Fig. 2). The path planner generates waypoints from integrated cost map. Mc, which is built around the robot and updated through online proprioception and visual observations. Mc offers a comprehensive estimation of external environment that encompasses terrain traversability costs Mr, obstacle occupancy Mo, proprioception advice Mp and goal approaching Mg. The planned waypoint is tracked by the RL-learned locomotion controller. In the following section, we will highlight the terrain estimator and proprioception advisor to demonstrate how the proposed system informs the robot of unexpected obstacles and unknown terrains based on walking experience. De- tails on the integration of the costmaps within the path planner provided in the appendix (Section 7.2).
3

---

Path Planner
Motion Controller
Way point
goal
P Puorte Russtd
Depth Encoder
oF
P Gool
M
Obstacle Estimator
la
of
M
Aigh
Terrain Estimator
Mr
you
Oluervaionn....
Ades
Motion Evaluations
Propriocepcion
Advisar
Critic
Actor
Costutap
Hligh
C₁
Cost
Figure 2: TOP-Nav framework: the path planner synthesizes Mr. Mo, Mp, Mg into combined cost map, from which it computes waypoints based on the overall cost considerations. The controller tracks the desired velocity and provide motion evaluations for the proprioception advisor.
4.2 Proprioception Advisor
We design the proprioception advisor to identify motion abnormalities that may arise from un- expected external disturbances. To offer a stable and comprehensive evaluation of the robot- environment interactions, we train the estimated value function (Section 3) to synthesize multiple metrics and incorporate historical observations. As demonstrated in Fig. 3(b), the motion evaluation will decline sharply when robot encounters a locomotion failure caused by transitioning onto challenging terrain. The reward design and normalization implementation details are provided in Section 7.4.
Within the path planner, we first directly utilize the motion evaluations to enhance the robot's 's aware- ness of unforeseen disturbances. This involves estimating invisible collisions (Mp) and integrating them into the combined cost map (Mc). We propose a continuously varied proprioception cost along the lateral direction (which is the y-axis Mp).
1- 1-Nom((c,)
Mp v) eh ([ll)))s==>>))))>>
(3)
here kp is hyperparameter and c is the estimated motion evaluation. Since a lower motion eval- uation indicates potentially challenging terrain an invisible obstacle in the current direction of the robot, we allocate to the centroid column in Mp and decrease it towards the edges, this will prevent the robot from continuing to move forward when its motion is disturbed by an unexpected obstacle or challenging terrain. We further discuss the relationship between the proposed motion evaluations, terrain difficulty, and ground truth motion states Section 8.3.
4.3 Vision-based Terrain Estimation
The proposed terrain estimator first leverages prior knowledge map the visual observations the reference terrain traversability. We apply perspective transformation Ligh map the pixels to hird-eye-view MBEv. which has corresponding coordinates MT. We then discretize MBEV into patches (Fig. 3(a)) and assign the same difficulty within each patch. The terrain within the prior knowledge Dierain is identified through terrain classification network Tiemain
Considering that such identification falls short when faced with unfamiliar terrains, we calculate the predictive entropy to approximate the identification uncertainty [41] and obtain the confidence f of current visual estimation accordingly. With P. denoting the predicted probability of terrain i, the predicted explicit Terrain names and confidence of the prediction for the current patch can

---

aa
Online Corrections
Visual Estimator
Panch
Tnp
may
Porspective
Transform
Prop Advisor
Experience Buffer
Terrain Classifier
Proprioception Evaluations
T( Vol.
(Prop Advice)
Tplitn
[S(o,r)
Tarmin
Confislene
Similarity
S(o,7)
(Voon-hased)
Reference
Corrected) My(K.:X
My (inn
Traversabülity
Lew-ooet
Bash
Brack
Snow
Gravel
Slah
Comeni
Paved
Craas
Figure 3: (a) The proposed terrain estimator incorporates a visual estimator and online corrections.
(b) The motion evaluations respond rapidly when the robot encounters difficult terrains.
be calculated as:
Conf =1 Σ P3 log and Terrain argmax(PP) (P:).
(4)
then a vision based traversability cost map Mr. could be directly obtained from pre-defined (Terrain, Cost) mapping. The mapping process leverages experience from previous works [20, 37), where we collect motion evaluations as the robot traverses various terrains in Diemain (Fig.
3(a)). Terrains with higher motion evaluations are assigned higher traversability scores and lower terrain costs. Detailed discussions are provided in section 7.5.
4.4 Online Terrain traversability Corrections
Through the seamless integration of vision and proprioception, we provide online corrections for the vision-based traversability estimation Mr, without additional training (Fig. 3(a)). This enables the robot terrain awareness beyond the limitations the collected data Dvemain
We address the mechanism by which the robot recalls the traversability of terrain once it has been seen and traversed within the navigation process. With the robot location (Xoase- Ylase). we record duration of 1s proprioception advice to indicate the terrain traversability cost at the robot location:
Tp Tr(Dbuse Vhuse = Norm
(5)
Besides Tp (T Mbase,, we can access latent feature L( Vouse) extracted by the classifier Tierrin when the same patch was observed few steps ago, we record both Tp and L at location (Dbases Drose) into an experienced list P.. Now given a new observation at location (Io, Bo), We can find a traversed and seen patch (xy, yr) that looks closest to (Io, yo) based on cosine similarity:
S(or)) argmax cos (L(ær, Vr), L(xo: Vol):
reP.
Note that also have the historical traversability cost Tp(e,, wr) infered by the proprioception advisor at (Iy, yr), can compute proprioception adapted traversability cost with the similarity
5

---

S(orr) to access the historical proprioception correction MI, (Zo: vol:
Tp Ur)
Mrp (Zor Vo)
+ e Rrn SSo)
due the delay in estimation from the proprioception history, we intend for My, to be utilized when encountering novel terrains where MT. becomes unreliable. Therefore, we normalize the visual uncertainty (z Vo) into confidence fonf(xon yo) to adjust the contribution of Mr, and Mr.:
MIo MIP
Mr( Uo)
1 kn Co) + MTa:(Ho
here Ky,, So, kT, and Co are hyperparameters.
By integrating the adapted Mr the terrain costmap into the path planner, our system demonstrates rapid udaptability to different terrains and visual conditions.
5
Evaluations
5.1 Experimental Setup
We assess TOP-Nav in challenging navigation environments across both simulations and real-world scenarios, emphasizing the following evaluations: 1) The improvements in navigation performance and locomotion stability achieved by introducing terrain awareness and motion evaluations into the navigation system. 2) The effectiveness of the proposed Proprioception advisor. 3) The effectiveness of the proposed Terrain estimator faced with novel terrains.
Evaluation Settings Our simulation experiments are conducted within Nvidia Isaac Gym (Fig. 4). We create 8 x independent navigation cells grid. Each cell is 5m x 5m in size, featuring a robot assigned to a point goal navigation task. The robot and point goal is randomly generated, with a minimum initial distance of 5m within the cell. The simulation experiments are conducted 25 times in each of the 64 navigation cells. For real-world evaluations, we conduct outdoor navigation tasks across different scenarios involving challenge obstacles and various terrains, with each scenario replicated 5 times for each method under investigation.
Metrics: We evaluate TOP-Nav with the following metrics: SR (Success Rate); The percentage of successful experiments. We define a success experiment as approaching the point goal into 0.5m within 20 seconds; TD (Terrain Difficulty): The percentage of average traversed terrain costs, TD provides the ground truth traversed terrain costs within each episode; UT (Unstable Time): The percentage of unstable motion states ([roll) >> 0.15 or |pitch| > 0.15) in each episode; VFT (Ve- locity Tracking Failure): The average percentage of velocity tracking failures (l|vin Vaell > 0.2) in each episode; AEC (Average Energy Consumption): The average energy consumption (7 [42] in each successful episode. We calculate the variance across different navigation scenarios to assess the robustness of the proposed method.
Beyond ablation study, compare TOP-Nav against state-of-the-arr legged naviga- tion frameworks and segmentation based terrain estimators. VP-Nav [27] integrates vision and proprioception to develop a collision detector and a fall predictor within the navigation pipeline. GA-Nav [19] achieves terrain segmentation relying solely on vision. Sterling [37] learns terrain traversability in a self-supervised manner by assigning traversability to terrains with similar propri- oception representations. However, this approach requires prior training for encountered terrains.
Baselines:
5.2 Simulation Evaluations
Improvements with Terrain Awareness: TOP-Nav empowers the robot to select terrains with higher traversability, leading significant improvement in the success rate of navigation: TOP- Nav surpasses the VP-Nav baseline by approximately 8% in success rate (SR) and achieve a TD of nearly 20% (Table 1), which is half of the 77) achieved in methods without terrain awareness.

---

Simulation Terrinss
Elaesic
Skppe
Dovnaar Hide
Proprocption
Sas
barguar
Figure 4: Each navigation cell consists of randomly generated challenging terrains with distinct traverse difficulty, which marked by the irregularity and complexity of the terrain. The proposed terrain awareness navigation framework plans an optimal path to navigate challenging terrains. The robot demonstrates the capability to recover from unexpected obstacles or irregular terrains with the proprioception advisor.
Table 1: Simulation Results with comparison experiments and ablation study
Method
SR (%)
TD(%)
UT(%) 4
VF7(%)
AEC
VP-Nav
65.62 = 2.4
47.18 = 0.8
36.18 = 0.5
26.09 + 0.4
101.80 = 49.6
wo/Terrain
68.00 + 2.6
45.83 + 0.7
34.92 0.6
25.06 + 0.5
101.70 + 46.6
wo/Proprioception
62.75 + 1.6
20.47 + 0.2
32.87 0.5
26.66
88.17 + 15.3
Obstacle-Only
52.81 2.3
47.79 0.8
0.8
39.13
96.82 = 48.0
TOP-Nav
73.62 + 1.2
22.65 0.2
27.21 0.2
18.14
92.08 = 24.4
Improvements with Proprioception Advisor: The proposed proprioception advisor could recover the robot from locomotion failures such getting stuck irregular terrain surfaces colliding with unseen obstacles during directional changes (Fig. 4). In contrast methods without proprioception, our approach exhibits approximate 11% enhancement in SR. Meanwhile, even without terrain awareness, the proposed system outperforms VP-Nav by 2.5%, signifying an improvement in our integration the proprioception advisor compared to existing methods.
Ad Advancements in Locomotion: TOP-Nav achieves the lowest UT and VFT, indicating that select- ing simpler terrains contributes locomotion stability. We evaluate energy consumption with the assumption that when traversing simpler terrain, the robot should exhibit more natural gaits. As a result, TOP-Nav exhibit 10% reduction in energy consumption compared to methods without terrain awareness.
5.3 Real World Evaluations
To evaluate the efficacy of the online corrections for providing the robot with traversability on novel terrains, we conduct experiments using degraded terrain classifier trained without gravel (Fig 5(A)) and in terrains without any prior information (Fig 5(B). We demonstrate that the robot can recall previously encountered challenging terrain during the phase of online corrections and plan way- point to avoid it. This showcases the fast adaptation ability of TOP-Nav to plan an optimal path when faced with novel terrains.
The quantitative results are provided in Table In the Unkown Gravel experiments, despite GA-Nav possessing complete prior knowledge and correctly identifying the paved road, our online correc- tions only lead to 7% increase in UT, mainly due the forward movement in phase 1. VP-Nav allows the robot to exit the gravel terrain with the safety advisor, but it can not retain this infor-
7

---

Table 2: Real World Evaluation Results on Novel Terrains
Scenerios
Unkown Gravel
Slippy Tarpaulin
TOP-Nav
VP-Nav
GA-Nav
Obstacle-Only
TOP-Nav
VP-Nav
Sterling
SR 1
5/5
4/5
5/5
5/5
5/5
3/5
2/5
UT(%)
14.30
26.43
7.98
42.33
1.19
2.64
3.85
AEC
57.46
70.30
62.57
72.60
83.40
102.0
113.8
Time(%))
62.72
77.90
67.42
70.64
67.62
63.05
76.52
'The Time metric measures the proportion of time the robot takes complete the navigation task.
mation. In the Slippy Tarpaulin experiments, TOP-Nav demonstrates effectiveness in maintaining stability and improving the success rate by avoiding challenging terrain. Slippy Tarpaulin ex- periments, TOP-Nav demonstarting effectiness in maintaining stability and improving the success rate by avoiding challenging terrain. Among the comparisons, Obstacle-Only keeps moving forward on the slippy surface and exhibit the worst motion performance. We include Sterling in this com- parison to demonstrate the importance of online corrections when incorporating proprioception into terrain estimation. Without online corrections, Sterling behaves similarly to Obstacle- -nlly,-On,, unable to adapt to novel terrains and continuing forward. The results against novel terrains highlight the system's 's ability to perform open-world navigation.
Faverity
Likazza Challige TOTEs
Liccan Sali Terai
Orline Crnsston
Unkanwe
Anidd
Tapeeks
Figure 5: (A) The terrain classifier does not include high-cost gravel in prior. (B) The robot en- counters terrains with no prior knowledge, including slippery detergent surface. We evaluate the effectiveness of the proposed terrain estimator within TOP-Nav in the experiments.
In addition, we conducted a consecutive long-distance experiment (Fig. 5-C) in an off-road scenario with grass and gravel on both sides the path. The robot is initially given a target direction straight ahead, relying on the terrain and obstacle estimator keep navigating on the paved road. In such scenarios, the challenging terrains help guide the robot in changing its direction, ensuring that it stays the paved road automatically. More hardware evaluations provided in Section 8.2 and the supplementary video.
6 Conclusion
We present TOP-Nav, a legged navigation system that achieves closed-loop integration of visual and proprioception at both task and motion planning levels. Through the extensive quantitative ex- periments conducted in both simulation and real world, we underscore the success of our system in achieving open-wor world navigation, surpassing limitations posed by visual conditions or prior knowl- edge.
The camera-based based vision system in TOP-Nav fails in providing robust visual odom- etry in varying light conditions outdoor environments. This limitation could be mitigated by integrating LiDAR- -odmmetryy into the perception module. Another significant limitation related to the costmap-bast based local planner, where the planned waypoint can continuously shift when sim- ilar minimum costs arise different locations on the map. We plan address this in future work by introducing diffusion based task planner, which can integrate different cost modalities while providing smoother sequence of future waypoints.
Limitations:

---

Acknowledgments
We would like to acknowledge the Visual Computing Lab at Tsinghua University and Unitree Robotics for providing the resources for this paper. We would also like to thank the reviewers for their helpful and insightful comments.
References
[1] Jenelten, He, F. Farshidian, and M. Hutter. Die: Deep tracking control. Science Robotics, (80))eedd55401, 2024.
[2] J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, and M. Hutter. Learn- ing agile and dynamic motor skills for legged robots. Science Robotics, 4(26))eauu8822, 2019.
[3] X. Cheng, K. Shi. A. Agarwal, and D. Pathak. Extreme parkour with legged robots. arXiv preprint arXiv:2309. 14341, 2023.
[4] Z. Zhuang, Z. Fu, J. Wang, C. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao. Robot parkour learning. arXiv preprint 0ww::339900560.. 2023.
[5] C. Zhang, N. Rudin, D. Hoeller, and M. Hutter. Learning agile locomotion on risky terrains. arXiv preprint arXiv:2311 2023.
[6] Lee, Hwangbo, L. Wellhausen, Koltun, and Hutter. Learning quadrupedal locomo- tion over challenging terrain. Science Robotics, x(47)xeabc5986, 2020.
[7] A. Kumar, Z. Fu, D. Pathak, and J. Malik. Rma: Rapid motor adaptation for legged robots. arXiv preprint
2021.
[8] A. Agarwal, A. Kumar, J. Malik, and D. Pathak. Legged locomotion in challenging terrains using egocentric vision. In Conference Robot Learning, pages 403-415. PMLR, 2023.
[9] T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning robust per- ceptive locomotion for quadrupedal robots in the wild. Science Robotics, r(62)seb 2022.
[10] D. Hoeller, L. Wellhausen, F. Farshidian, and M. Hutter. Learning a state representation and navigation in cluttered and dynamic environments. IEEE Robotics and Automation Letters, (3):5081-5088, 2021.
[11] K. Caluwaerts, A. Iscen, J. C. Kew, W. Yu, T. Zhang, D. Freeman, K.-.. Lee, L. Lee, S. Sal- iceti, V. Zhuang, al. Barkour: Benchmarking animal-level agility with quadruped robots. arXiv preprint a005v:2305. 14654, 2023.
[12] Kareer, N. Yokoyama, D. Batra, Ha, and J. Truong, Vinl: Visual navigation and loco- motion over obstacles. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 2018 2024.. IEEE, 2023.
[13] J. Truong, D. Yarats, T. Li, F. Meier, S. Chernova, D. Batra, and A. Rai. Learning naviga- tion skills for legged robots with learned robot embeddings. In 2021 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 491. IEEE, 2021.
[14] M. Wermelinger, P. Fankhauser, R. Diethelm, P. Krüsi, R. Siegwart, and M. Hutter. Navigation planning for legged robots in challenging terrain. In 2016 IEEE/RSI International Conference on Intelligent Robots and Systems (IROS), pages 1184-1189 189. IEEE, 2016.
[15] M. Mattamala, N. Chebrolu, and M. Fallon. An efficient locally reactive controller for safe navigation in visual teach and repeat missions. IEEE Robotics and Automation Letters, 7(2): 2353-2360, 2022.

---

[16] P. Ewen, A. Li, Y. Chen, S. Hong, and R. Vasudevan. These maps are made for walking: Real- time terrain property estimation for mobile robots. IEEE Robotics and Automation Letters, (3)77083-7090, 2022.
[17] L. Gan, Y. Kim, W. Grizzle, J. Walls, Kim, M. Eustice, and M. Ghaffari. Multitask learning for scalable and dense multilayer bayesian map inference. IEEE Transactions on Robotics, 39(1):699-717, 2022.
[18] G. Jrni, Frey, T. Miki, M. Mattamala, and M. Hutter. Mem: Multi-modal elevation mapping for robotics and learning. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1 11011-1101 IEEE, 2023.
[19] Guan, Kothandaraman, R. Chandra, A. Sathyamoorthy, Weerakoon, and D. Manocha. Ga-nav: Efficient terrain segmentation for robot navigation in unstructured out- door environments. IEEE Robotics and Automation Letters, 7(3):1188 01441,1,,211 2022.
[20] J. Frey, M. Mattamala, N. Chebrolu, C. Cadena, M. Fallon, and M. Hutter. Fast traversabilityy estimation for wild visual navigation. arXiv preprint 2rXiv:2305.08510, 2023.
[21] X. Yao, J. Zhang, and J. Oh. Rea: Ride comfort-aware visual navigation via self-super learning. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7847-7852. IEEE, 2022.
[22] N. Rudin, D. Hoeller, M. Bjelonic, and M. Hutter. Advanced skills by learning locomotion and local navigation end-to-end. In 2022 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2497-2503, IEEE, 2022.
[23] D. Hoeller, N. Rudin, D. Sako, and M. Hutter. Anymal parkour: Learning agile navigation for quadrupedal robots. arXiv preprint arXiv:2306. 14874, 2023.
[24] R. Yang, M. Zhang, N. Hansen, H. Xu, and X. Wang. Learning vision-guided quadrupedal lo- comotion end-to-end with cross-modal transformers. arXiv preprint arXiv:2107.03996, 2021.
[25] T. He, C. Zhang, W. Xiao, G. He, C. Liu, and G. Shi. Agile but safe: Learning collision-free high-speed legged locomotion. arXiv preprint arXiv:2401.17588 2024.
[26] C. Zhang, J. Jin, J. Frey, N. Rudin, M. Mattamala Aravena, C. Cadena, and M. Hutter. Resilient legged local navigation: Learning to traverse with compromised perception end-to- end. In 41st IEEE Conference on Robotics and Automation (ICRA 2024), 2024.
[27] Z. Fu, A. Kumar, A. Agarwal, H. Qi. J. Malik, and D. Pathak. Coupling vision and propriocep- tion for navigation of legged robots. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17273 77273-17283, 2022.
[28] J. Frey, S. Khattak, M. Patel, D. Atha, J. Nubert, C. Padgett, M. Hutter, and P. Spieler. Roadrunner-leaminn traversability estimation for autonomous oof-road driving. arXiv preprint arXiv:2402.193.4 2024.
[29] P. Roth, J. Nubert, F. Yang, M. Mittal, and M. Hutter. Viplanner: Visual semantic imperative learning for local navigation. arXiv preprini arXiv:2310.00982 2023.
[30] K. Viswanath, K. Singh, P. Jiang, P. Sujit, and S. Saripalli. Offseg: A semantic segmentation framework for off-road driving. In 2021 IEEE 17th international conference on automation science and engineering (CASE), pages 354-359. IEEE, 2021.
[31] X. Meng, N. Hatch, Lambert, A. Li, N. Wagener, M. Schmiule, J. Lee, W. Yuan, Z. Chen, S. Deng, et al. Terrainnet: Visual modeling of complex terrain for high-speed, off off-road navi- gation. arXiv preprint drXhv:2303.1577, 2023.
10

---

[32] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3213-3223, 2016.
[33] H. Abu Alhaija, S. K. Mustikovela, L. Mescheder, A. Geiger, and C. Rother. Augmented reality meets computer vision: Efficient data generation for urban driving scenes. International Journal of Computer Vision, 126:961-999 -972, 2018.
[34] D. D. Fan, K. Otsu, Y. Kubo, A. Dixit, J. Burdick, and A.-A. Agha-Mohammadi Step: Stochastic traversability evaluation and planning for risk-aware off-road navigation. arXiv preprint arXiv:2/03.02828, 2021.
[35] X. Cai, Ancha, L. Sharma, R. Osteen, B. Bucher, S. Phillips, Wang, M. Everett, N. Roy, and How. Evora: Deep evidential traversability learning for risk-aware off-road autonomy. arXiv preprint arXiv:2311.0623: 2023.
[36] Jung, J. Lee, X. Meng, B. Boots, and A. Lambert. V-strong: Visual self-supervised traversability learning for off-road navigation. arXiv preprint drXiv:2312.16016. 2023.
[37] H. Karnan, E. Yang. D. Farkash, G. Warnell, J. Biswas, and P. Stone. Sterling: Self-supervised terrain representation learning from unconstrained robot experience. In Conference on Robot Learning, pages 2393-2413, PMLR, 2023.
[38] M. Elnoor, A. J. Sathyamoorthy, K. Weerakoon, and D. Manocha. Pronav: Proprioceptive traversability estimation for autonomous legged robot navigation in outdoor environments. arXiv preprint arXiv:2307.09754. 2023.
[39] G. B. Margolis, X. Fu, Y. Ji, and P. Agrawal. Learning see physical properties with active sensing motor policies. arXiv preprint arXiv:231.001405, 2023.
[40] H. Karnan, Yang, G. Warnell, J. Biswas, and Stone. Wait, that feels familiar: Learn- ing to extrapolate human preferences for preference aligned path planning. arXiv preprint arXiv:2309.09911 2023.
[41] G. Wang. W. Li, M. Aertsen, J. Deprest, S. Ourselin, and T. Vercauteren. Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks. Neurocomputiing. 338:34 2019.
[42] Z. Fu, A. Kumar, J. Malik, and D. Pathak. Minimizing energy consumption leads to the emer- gence of gaits in legged rohots. arXiv preprint arXiv:2111.01674, 2021.
[43] J. Long, Z. Wang, Q. Li, J. Gao, L. Cao, and J. Pang. Hybrid internal model: A simple and efficient learner for agile legged locomotion. arXiv preprint arXiv:23/2.114660 2023.
[44] Schulman, Wolski, Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 170.006777 2017.
[45] M. Wigness, S. Eum, J. G. Rogers, D. Han, and H. Kwon. rugd dataset for autonomous navigation and visual perception in unstructured outdoor environments. 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5000-5007. IEFE, 2019.

---

Appendix
7 Implementation Details
7.1 Task Configuration
To elucidate the task configuration, the robot is given a point goal with its location Pyoal The location of the robot is denoted as (Poase: Tbase). For external observation, we utilize bio-channel perception module for both the path planner and the controller. This module includes depth (Ia) and RGB (Zrgh) channels. The path planner computes the desired velocity command (Vin Ayaw) for the robot, which is tracked by the controller along with the appropraite joint position signal.
7.2 Path Planner Details
TOP-Nav integrates the combined cost map Mc. offering comprehensive estimation that encom- passes terrain traversability cost Mr, obstacle cost Mo. proprioception advice Mp and the goal approaching cost Mc. This integration addresses both visible and unexpected external disturbances against the robot.
To keep balance between navigation efficiency and safety, we formulate the combination of Mc with dynamic scaling factors follows:
Mc = Mr + Mo + aT + ag Ma.
(9)
KT
kG:
OT (d-dd)))) 0G 1
1 1- ko,
KGo
(10)
here M, are 2-d matrix with spatial resolution of 0.15m, do, lo, kT, ke are hyperparameters, t denotes current time consuming and d denotes the distance from the point goal at the current step. We accord the highest priority to Mp and Mo since they represent non-traversable locations where the robot cannot pass through. The terrain traversability scale ar decreases as the robot approaches the target. This design is made considering that, as the robot nears the target (d decreases), taking detour to avoid challenging yet traversable terrain would be inefficient behaviour. Conversely, the goal approaching scale og increases with the duration of the task.
With the integrated map Mc, we select the optimal waypoint Pway based on the lowest combined cost.
(e.y)=p
Pway argmin E
Σ
Mc(x,y).
(11)
|Pase
pll
here E denotes the set points on the edge of Mc. For each point p in E, we calculate the average combined cost along the path from the robot location Pease to the edge point p. The optimal waypoint Pway is then chosen as the point with the lowest path cost.
After determining the optimized local target Pway. the target direction is calculated based on the relative position Ayaw arctan(Pway - Phose) Ybase. here Phase denotes the yaw direction of the robot base. The linear velocity is constrained to mitigate the impact of high angular variation with Uhin Upe ka thereby preventing abrupt and substantial turning at high speeds.
In simulation, Mc is configured with dimensions (1.65m, 1.5m) centered around the robot's loca- tion at Phase (0.45m, 0.75m). This design prioritizes a light-weight, real-time updated path plan- ner and maintains consistency with the height map used in locomotion. The environments include randomly generated obstacles and terrains. Each cell is equipped with at least one wall obstacle, two column obstacles and the remaining space is divided into 1m x 1m sections. For terrain traversabil- ity assignments, we uniformly partition the obstacle-free space into difficulty levels [0, 0.25, 0.5, 0.75, I] and generate terrain with various heights of steps and the intensity of irregular terrain cor- responding to the difficulty levels. This difficulty serves ground-truth walking preferences in simulation.
map
12

---

To obtain a more comprehensive observation in the real world, we configure the cost map Mc with dimensions of (3m, 3m). The location of robot is at Piwe = (0m, 1.5m) in real world Mc. The criteria for success include reaching the goal within the specified time constraints, consistent with simulation setting.
map
Obstacle Estimation and Localization: In simulation, the robot has access the ground truth location. For each point p = (x, v). Ma y) liP pll- In real world, we set target direction Tgoal to compute the goal map Mc (z,(zz, y) = || gg arctan(p Prase)ll Il| We construct the obstacle estimation based on the depth channel Ig- The perceived obstacles are converted into point clouds, and for each point in Mo. we compute the signed distance MSDE to the closest obstacles within distance d'axx the cost of obstacles therefore can be computed as:
Mo v) max(0, dmax MSDF (2, y))
(12)
dmax
For reference, we provide the default values of the hyperparameters used in the path planner in Table 3. Parameters do, in adjust the weights of Mr and Mg as the robot approaches the target and as time progresses. fic keeps minimum weight for the robot approaching the target. The listed values remain consistent in simulation; however, considering the various conditions the real world, we make slight changes to these values in different real-world experiments for better validation of the contribution of this work. For instance, in outdoor navigation experiments, assign a higher value to kT, expand the scale of the terrain estimator, which is effective for evaluating the accuracy of the proposed terrain estimator.
The default commanded velcoity vo is set to 0.5m/s in both simulation and the real world. The maximum distance dmax we considered for computing the obstacle map Mo is 0.3m.
Table Hyperparameters in the path planner
Parameter
do
fo
kr,
krz
kGo
kG;
kGa
Default Value
0.5
0.5
1.0
0.1
0.4
-10
7.3 Motion Controller Details
The motion controller is implemented following [3, 43]. Both the actor and critic networks within the locomotion policy ı have hidden layer sizes of [512, 256, 128). The proprioception observation of includes angular velocity (3), Orientation (2), velocity commands (3), joint positions (12), joint velocities (12), and the last action (12). We store the last 10 steps of of into the history observation of. The depth encoder learns the exteroception latent features from onboard observation Ia using GRU structure. is updated using PPO [44] for 20K iterations, the batch size is 160000 divided into 4 at each step are the sum of the reward functions listed in Table 4.
The depth encoder is optimized for 5k iterations. The rewards obtained
7.4 Proprioception Advisor Details
The default Teriki requires privileged observations, we train another value function estimation net- work using observable motion states for the proprioception advisor to be deployed on hardware. The estimated motion evaluation C; is normalized using a sigmoid function. The proprioception advisor trained with the locomotion policy æ, with exact network architechture Weritie- The input only includes proprioception observation of and historical observation of updated through the MSE loss with fay The estimated value is normalized through sigmoid function:
Norm(c)=
(13)
+ (2.2-a))))
13

---

Table 4: Reward Functions
Target Velocity Tracking'
min( Pact, Vin))
Target Direction Tracking
e |Ayerl
Orientation Penalty
-(roll + pitch?)
Hip Joint Position Penalty
z-direction velocity Penalty
Collision Penalty"
Σ (call F2 2 0.1N)
Torques Variation Penalty
|| 7--1l
Unct is the projection component Vact in the target direction.
2 F is the contact force of calf and thigh indices.
In practice, and we set minimum threshold Caa. for the normalized critic value be considered in the path planner, in simulation Czb. 0.8, while in the real world сaΛ = 0.5, Norm(ce) larger than Ozb. will be set to 1.
Mp calculated with kp 0.3, we demonstrate an example of Mp after the robot collides with wall (with vision and obstacle detection disabled in this trial) in Fig 6. In this scenario, the normalized o returns 0.02, Mp has width of 0.75m along the moving direction of the robot with costs greater than 0.5.
Mp 0.22
P Phase
0.54
P base
0.98
0.54
0.22
1. 65m
Figure 6: We demonstrate an example of Mp when the robot collides with an obstacle.
Generally, the proprioception advisor engages when locomotion states compromised. To enable recovery from being obstructed by the unexpected terrains, we have designed the following recovery strategy in simulation: when Norm((;) falls below specific threshold (0.5), we initially assign backward velocity command of 0.5 m/s for 0.5 seconds. Next, to address potential terrain-relatee constraints that may prevent the robot from reaching planned waypoints at its default velocity, we reduce the frequency of the path planner to 0.5 Hz until the robot reaches the waypoint or makes directional change of 0.3 radians.
7.5 Terrain Estimator Details
For prior data collection of the terrain estimator, we teleoperate the robot to walk for 5 minutes on each of the concerned terrains. We capture the motion evaluations as well as the first-view observations during these demonstrations. We convert the RGB observations into BEV maps and dividing them into patches. The collected data is labelled based on the corresponding terrain from which was acquired. The classification network is implemented using the MobileNet backbone and trained a Nvidia GTX 2080 Ti for 6 hours. The proposed pipeline provides light-weight inference model of MB to be deployed for onboard computation. During deployment, to approx- imate the uncertainty U with predictive entropy, We perform K 8 stochastic forward inferences
14

---

with different data augmentations as Monte Carlo simulation, We estimate the expectation as the average the predicted probability P Σ$ Ph //K
Based on the collected motion evaluations, we compute the average e observed during walks on each terrain. These walking experiences provide reference values for assigning the terrain costs defined by the operator. Table 5 presents the validation accuracy of the terrain classifier, along with the reference terrain cost and the average collected motion evaluations N Norm((c) for each terrain. Since the terrains encountered in the real world did not exhibit significant differences in motion evaluations, we did not strictly set the reference cost based on linear correlation.
Table 5: Terrain Cost and Classify Accuracy
Name
Bush
Brick
Snow
Gravel
Slab
Cement
Paved
Grass
Accuracy (%)
89.58
92.28
83.20
83.47
90.67
82.87
93.81
88.36
Norm(cr)
0.481
0.523
0.445
0.507
0.581
0.580
0.580
0.572
Cost
0.7
0.6
0.9
0.7
0.0
0.0
0.0
0.2
The hyperparameters used for online corrections are fine-tuned based on the distribution of the learned latent features L from the pre-collected data. Figure 7-(b) depicts t-SNE visualization of the features inferred by the terrain classifier, involving data both within and outside of D Pierrann We observe that Teemmin can learn distinctive features for specific terrains, showcasing unique clustering in the latent space, even for previously unseen terrain types. The average cosine similarity ((or) within each terrain class is 0.95, set the middle point So be 0.85 and kT, to be 15. This allows the terrain cost derived from historical experience to decrease rapidly when the similarity falls below 0.8.
(a) Distribution of samples across differeat confidence level
(b) T-SNE Plat of Terrain Fentures
150
epoch
bush
Requency
content
grads
20-
gravel
paived
disb
show
leveestooeel
0.0
0.2
0.4
0.6
0.8
100
0-
novel
epoch 200
Trequency
-20
60-
20
Figure 7: (a) We illustrate the distribution of samples across different confidence levels, with the training of the terrain classifier, the confidence values for known terrains predominantly clustered beyond 0.9, while samples of novel terrains remained dispersed across the confidence axis. (b)T- SNE visualization of the features inferred by the terrain classifier.
To demonstrate the effectiveness of computing confidence in deciding whether the observed terrains outside Diemann we illustrate the distribution of samples across different confidence levels in Fig
7-(a). After 200 steps of training, the confidence values for known terrains predominantly clustered beyond 0.9, while samples of novel terrains remained dispersed across the confidence axis. The confidence is used to adjust the weight of online correct ions in terrain estimation. We set ky, 20 and Co = 0.9 in the experiments.
15

---

7.6 Hardware Details
We implement TOP-Nav on the Unitree-Gol and Unitree-Go2 quadruped robot, with weight of approximately 12kg and dimensions of 645mm x 280mm. The robot is equipped with 12 brushless motors, each capable of producing torque of m.. The perception module is equipped with a RealSense D435i camera, offering simultaneous depth and RGB channels. All computations are processed onboard with an NVIDIA Jetson NX asynchronously, i.e. the locomotion controller oper- ates at a fixed frequeney of 20H2 and receives the latest depth latent & inference from the perception module. The path planner operates frequency of 3H2.
8 Additional Experiments
In this section, we provide additional experiments to justify the effectiveness of the key components in TOP-Nav.
8.1 Evaluations on Different Evaluation Metries
This section gives a detailed description of the evaluation choices throughout the simulation and hardware experiments.
Uhrecholss
Comparison thresholes
ToP
1.
E 055
02
0.1
Tacking Fulkur Threhsak
Uniaate Thrastkk
Figure 8: Evaluations on different threholds for VFT and UT metrics.
We set The Timeout for Success Rate 20 seconds, which corresponds to expected path length of 10 meters given the commanded velocity of 0.5m/a /s. Note that the target goals will be randomly generated at least 5m away from the robot initial location within the 5m x 5m nav-cell. Therefore, 10 meters represents the longest path the robot would plan to complete the navigation episode without backtracking. We set the relatively loose time limit to emphasize the robot's ' ability to complete the task, as avoiding challenging terrains and obstacles may require taking detours. The experimental results demonstrate that most failure cases occur when the robot gets stuck due to obstacles or challenging terrains. As result, extending the timeout limit does not lead significant change in the success rate. For example, when we raise the timeout limit to 30 seconds, the success rate for TOP-Nav increases from 73.62% to 75.93%, the success rate for Obstacle-Only raises from 52.81% to 59.39%.
We set the metrics of Unstable Time and Velocity Tracking Failure to evaluate the advantage in motion stability when the robot follows the planned path using the proposed system. Generally, the robot achieves higher speeds with less roll and pitch when walking the optimal path. As demonstrated in Fig. 8, the robot performs better on these metries across different threshold settings.
8.2 Evaluations on Vision-Based Terrain Estimator
We conduct additional quantitative hardware experiments terrains included in the prior datasets (Fig 9), demonstrating superior performance compared to existing legged navigation systems.
16

---

1-
1-A
1-5
10
1.0
1-6
2-8
Scmrr
Figure 9: conduct the evaluations in scene 1-3, the green line represents the trajectory TOP- Nav, while the red line represents Obstalce-Only.
The evaluation results are presented in Table 6. TOP-Nav successfully completes all 15 trials with superior walking stability and minimal energy consumption. These results affirm the suc- cessful deployment of the proposed system on real hardware, allowing the robot to select way- points on terrains with better traversability and thus execute more natural gaits. In contrast, GA- Nav, trained on RUGD [45], demonstrates notable limitations when confronted with open-world navigation. We observe that the improvements of TOP-Nav over GA-Nav mainly stem from: 1) In scene 1,2, GA-Nav fails identify the slabbed road as low-cost terrain from the bush and gravel. 2) In scene GA-Nav incorrectly identifies the lower part of the obstacle cement floor, which could be addressed by our integration of obstacles and terrain estimation. Nevertheless, GA-Nav surpasses VP-Nav and Obstacle-Only, which navigates without terrain awareness. With Obstacle-Only, the robot gets stuck by the stones or Grassroots, leading to higher time consumption.
Table 6: Evaluation Results on Vision-Based Terrain Estimator
Method
SR f
UT(%) L
AEC
Time(%)
GA-Nav
13/15
19.80
74.42
68.76
VP-Nav
11/15
23.02
71.45
80.12
Obstacle-Only
12/15
22.75
71.09
79.72
While VP-Nav effectively alerts the robot to locomotion failures that could be brought by such
TOP-Nav
15/15
7.79
59.07
65.03
terrains, the absence of terrain awareness prevents the robot from successfully navigating out of these challenging terrains, resulting in inferior locomotion performance.
8.3 Evaluations Proprioception Advisor
In this section, we discuss the relationship between the proposed proprioception advisor and terrain difficulties, along with ground truth reward terms such as velocity tracking and attitude stability.
We conduct a series of straight navigation tasks in simulation, as demonstrated in Fig 10. Each robot's waypoint is
straight line, covering terrains of various difficulties Dj and types:
Irregular Surface: The height of each point in the terrain is randomly changed within the range of (0.01 + 0.04 Dr, 0.07 0.04 * Dt)m.
17

---

Slope & Holes
Discrete Steps
Irregular Surface
1.0
0.75
0.50
0,25
1.0
0,75
0.50
0.25
1.0
0.75
0.50)
0.25
Figure 10: We conduct series of straight navigation tasks in simulation. Each robot's 's waypoint is set in a straight line, covering terrains of various difficulties [0.25, 0.5, 0.75, 1] with types covering slopes, discrete steps and irregular surfaces.
Discrete Steps: This terrain type consists of 12 Dr mall steps within Im x Im area, where each small step has dimensions of (0.1, 0.1, 0.08)m. Slope & Holes: Randomly switches between upper slopes and holes, where the maximum height/depth each slope/hole set as 1.2m Dj.
Table 7: Evaluation of the proprioception advisor different types of terrains.
Terrain Difficulty
Flat
Method
Slope& Holes
Discrete Steps
Irregular Surface
Ground
0.25
0.7560
0.7964
0.7001
0.50
0.3316
0,3633
0,2589
Prop Advisor
0.8600
0.75
0.1893
0.1614
0.1058
1.0
0.0573
0.0837
0.03/17
0.25
0.8649
0.9836
0.9811
0.50
0.3076
0.7403
0.81/11
Decoupled Training
0.9900
0.75
0.2312
0.2323
0.4831
1.0
0.0770
0.3230
0.2563
We conduct 30k steps of navigation for each robot and compute the average motion evaluations estimated by the proprioception advisor. The results are shown in Table 7. We observe that as terrain ruggedness, irregularity, and the presence of obstacles increase, there is a corresponding decrease the motion eval- uations. Although this correlation is not strictly linear, the results provide supportive for us estimate terrain traversability based on the proprioception advisor.
10-
scores
For comparison, we train an independent motion eval- uation function Taeg- which is decoupled from the training process of the motion controller. Taaq is up- dated using the MSE loss between the ground truth motion states and the network prediction Ger- The ground truth walking states include velocity tracking. orientation penalty, and energy consumption:
0.0
Figure 11: The estimated & and the actual motion states of the robot has strong lin- car correlation.
Cre
parget
min (uac,, Vinn) (roll? + pitch?) 0.001 Tÿ.
(14)
train idog for 10k iterations, Jurget is normalized using sigmoid function.
18

---

As shown in Table 7, Qee demonstrates noticeable decline as terrain difficulty increases, yet it does not exhibit a significant advantage compared to the proposed proprioception advisor.
On the other hand, the same terrain may exhibit different traversability for robots with varying locomotion capabilities. Therefore, the evaluation of walking states represents another important metric for assessing terrain traversability. We demonstrate the correlation between the predicted motion evaluations 6 and the ground truth motion states Get in Fig 11.
The perason correlation coefficient 0.914, indicating a strong linear correlation between the esti- mated Ct and the actual motion states the robot.
Building upon the discussion above, we validate that constructing the proprioception advisor with critic output enables efficient motion evaluation, including information both the terrain difficulty and walking states of the robot. Moreover, the proposed advisor be implemented without the need for additional training or sensors, making it more appropriate approach to provide motion evaluations for robot path planning compared to existing methods.
8.4 Different Velocities
We conducted quantitative experiments in simulation with different velocities (0.25, 0.5, 0.75, 1.0) (m/s), and the results demonstrate that proper velocity design does affects the performance of the navigation system. As demonstrated in Table 8, a slower velocity command (0.25) prevents the robot from reaching the goal on time, resulting in lower success rate. Moreover, it can be concluded that lower the speed would help decrease the energy consumption.
Table 8: Simulation Results with velocity comparisons
Velocity (m/s)
SR (%) f
TD(%)
UT(%)
VFT(%)
AEC
0.25
49.69 + 3.43
26.22 = 0.33
26.56 = 0.41
25.95 + 0.45
65.55 + 352.19
0.5
73.62 1.21
22.65
27.21 0.24
18.14 0.16
92.08 24.39
0.75
73.91
2.46
22.22 + 0.26
22.53 0.37
16.79 + 0.28
80.32 46.82
1.0
72.03
2.80
21.65 + 0.23
41.92 0.49
21.30 1 0.44
123.53 1 39.61
Metrics
Va-Nav
Obstacle-Oni Only
TOP-Nav
SR t
5/5
0/5
5/5
UT(%)
1.45
/
0.79
AEC 4
42.97
/
34.63
S7(s)
3.30
+00
1.98
The robot avoids invisble ob- stacles with the proprioception advisor.
Figure 12:
Table Evaluation results on the Invisible Obstacles.
On the other hand, setting large velocity command will introduce locomotion instability, as indicated by the results with velocity of 1m/s. However, conducting quantitative analysis of the influence of speed is challenging in complex terrains. For instance, in slopes, robots with higher locities may navigate more effectively, whereas terrains with discrete steps could lead to locomotion failures at faster speeds. Consequently, experiments with velocity commands of 1m/s still achieve a high success rate.
The performances vary between 0.5m/s and 0.75m/ is significant; therefore, consider this range to be appropriate commanded velocity range for deployment. In rerl-world experiments, demonstrate the robot's velocity set at 0. 0.75m/s /s in the supplementary video.
19

---

8.5 Invisible Obstacles
The proposed proprioception advisor provides close-loop feedback to alert the robot to invisible obstacles. We assess this capability in Fig. 12, where a glass wall is located along the planned path. In phase I, the obstacle estimator is oblivious to the presence of the glass wall, and the robot continues moving forward. In phase 2, the robot collides with the glass wall, causing a rapid increase in Mr along the direction of movement. As result, the robot adjusts its waypoint to steer clear of the high-cost area, successfully avoiding the glass wall. The quantitative results are provided in Table 9.
20


Reviews:
[{"review_id": "o5KYifCB1G", "reviewer": "Reviewer_CGbG", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper represents a method for legged navigation that combines different sources of information, such as RGB vision for semantics, depth estimates for collision avoidance, and proprioceptive sensing for dealing with unobservable obstacles. The approach is based on a RL approach with a costmap representation.  The experimental results are done in simulation and real platforms, showing advantages to other similar methods.", "summary_of_recommendation": "I believe the paper needs a general revision: the method is presented in a convoluted manner, important pieces are not explained, and it is not clear if the experiments are really assessing the important aspects and contributions of the work. Apart from presentation issues that were also raised in terms of the wording and typos.", "technical_quality": 3, "clarity_of_presentation": 2, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 1, "questions_for_rebuttal": "- Please explain why the motion planner needs to be learned if most of the planning happens in the cost map in the end. Why is a traditional local planner insufficient for this purpose?\n- Similarly, how does the motion planner leverage depth information twice? (as an input, and as part of the costmap).\n- How do you train the method in simulation if the vision input cannot be simulated? Wouldn’t this affect the policies learned by the motion planner?\n- The idea of combining different cost maps recalls the work by Wermelinger (https://ieeexplore.ieee.org/document/7759199). Other recent approaches have also been proposed to combine “fields” for navigation (e.g. https://arxiv.org/abs/2201.03938 ). None of them are discussed.\n- It is not really clear how the costmap is defined. Is it aligned with the robot’s pose or is it dependent on some fixed frame (like ROS’ grid_map)? How about the magnitude of the cost itself? Is it [0,...,1]?\n- How are the semantic classes (from the visual terrain estimator) mapped to costs? Are the values reported in Table 9 hand-crafted?\n- The overall presentation of the experiments requires further clarifications. Experiments are presented in an arbitrary way, not really explaining the purpose of them, the conditions, and what’s being assessed. \n- Further, without discussing and providing further insights on the failure cases and limitations it’s really hard to assess and understand why the method could overcome challenges of previous approaches. It is not clear to me from the experiments that all the components together ensure the success of the method, since the experiments demonstrate examples that can be solved by one “modality” at a time - the glass door is solved by proprioception and depth sensing (similar to Fu (2022)), the path tracking experiment is similar to Wellhausen (2019, not cited either: https://ieeexplore.ieee.org/document/8627373) or Frey (2023), who solved it with vision only. It would be necessary to provide a further analysis and demonstration on how these different modalities and the overall system design comes together and effectively justify the success of the approach.\n\nMinor text fixes:\n- _“Legged navigation is typically examined within open-world, off-road, and challenging environments“_ -> I’m sure that examined is not the right word to use here, but I’m not sure what was the intention of the sentence to suggest an alternative\n- _“insimulable”_ -> does not exist, I suggest to write “hard to simulate” instead\n- _“open-loop local planner”_ -> a local planner is closed-loop by construction, as it uses online sensing to compute local paths\n- _”A comprehensive legged navigation framework Integrating multi-modal”_ -> “integrating” should not be capitalized\n- _“As demonstarted in Fig. 3(b)”_ -> should be “demonstrated. There are way too many other typos in the figures and the text, please revise.\n- Please do not use red and green together for the figures. They are hard to distinguish for colorblind people.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YQ1MjKbEkw", "reviewer": "Reviewer_BuXi", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper presents a method for integrating perception in the form of terrain classification, obstacle identification, and proprioception for the purposes of improving legged locomotion over challenging and potentially unseen terrain.  The proposed framework is validated in simulation and in real-world experiments and is compared against existing approaches. The contributions are further validated via an ablation study examining the contribution of each of the proposed elements.", "summary_of_recommendation": "The proposed framework in well validated, however grammatical errors, poorly communicated methodology, and evaluation metric thresholds need to be addressed.", "technical_quality": 2, "clarity_of_presentation": 1, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 1, "questions_for_rebuttal": "1. More clearly explain the methodology in Section 4 and fix grammatical errors throughout the paper.\n\n2. Evaluate the effect of the evaluation metric thresholds on the proposed method and comparisons.\n\n3. Include a more thorough discussion on the limitations of the proposed methodology.\n\n4. Address minor comments.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tcrCxj1iwB", "reviewer": "Reviewer_VJBQ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper introduces a framework for estimating traverabilitiy for a legged robot. The method can be updated online with historical experiences.", "summary_of_recommendation": "I lean towards accept but there are some issues with the paper which need to be addressed.", "technical_quality": 3, "clarity_of_presentation": 2, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 1, "questions_for_rebuttal": "1. Add a limitations section.\n2. Please address the question of scalability for the terrain experience list and explain how this method works over larger distances.\n3. Please clarify the following details:\n - What exactly are the proprioceptive signals used? Joint velocities? IMU? Please explicity describe all sensing including dimensionality.\n - How is the proprioception advisor signal mapped to the Mp like in Figure 10? Why does a collision in the front show low cost behind the wall?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: In summary, the initial reviews highlight the following strengths and weaknesses of this submission.

Strengths:
- The paper proposes an interesting framework for online adaptation for legged robot navigation.
- The paper provides experimental results with a real platform showing impressive and sensible behavior of the robot.

Weaknesses:
- The paper lacks sufficient details and comprehensibility to fully understand the method.
- The motivation of several design choices is not well explained.
- The experiments are not fully clear and convincing. For instance, it is not clear how the joint consideration of different modalities improves performance, or the thresholds for evaluation metrics seem arbitrarily chosen.
- The paper lacks discussion of limitations, failure cases, and assumptions.

After the author response and discussion phase, the reviewers have been mostly satisfied how the authors adressed their comments, raised their scores and recommend accepting the paper. The authors should additionally consider improving the structure of the paper and providing a definition of traversability as suggested by the reviewers.

The revised paper must not violate the page limit for submission.
Decision: Accept