TOP-Nav: Legged Navigation Integrating Terrain, Obstacle and Proprioception Estimation

Authors: junli  ren
yikai  liu
yingru  dai
junfeng  long
guijin  wang
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-02 10:44:31

Abstract: Legged navigation is typically examined within open-world, off-road, and challenging environments. In these scenarios, estimating external disturbances requires a complex synthesis of multi-modal information. This underlines a major limitation in existing works that primarily focus on avoiding obstacles. In this work, we propose TOP-Nav, a novel legged navigation framework that integrates a comprehensive path planner with Terrain awareness, Obstacle avoidance and close-loop Proprioception. TOP-Nav underscores the synergies between vision and proprioception in both path and motion planning. Within the path planner, we present a terrain estimator that enables the robot to select waypoints on terrains with higher traversability while effectively avoiding obstacles. In the motion planning level, we construct a proprioception advisor from the learning-based locomotion controller to provide motion evaluations for the path planner. Based on the close-loop motion feedback, we offer online corrections for the vision-based terrain and obstacle estimations. Consequently, TOP-Nav achieves open-world navigation that the robot can handle terrains or disturbances beyond the distribution of prior knowledge and overcomes constraints imposed by visual conditions. Building upon extensive experiments conducted in both simulation and real-world environments, TOP-Nav demonstrates superior performance in open-world navigation compared to existing methods.

Main paper: 
 introduction: 
 Despite recent advancements in legged locomotion allowing the robots to navigate various terrains based on a simulation-learned strong controller [1, 2, 3, 4, 5, 6, 7, 8, 9] , the complexity of currently hard-to-simulate real-world factors makes it impossible for the robot to traverse all potential terrains encountered in reality. As a result, simply connecting the locomotion controller with a vision-based path planner often restricts legged navigation to limited scenarios [10, 11, 12, 13, 14, 15] .An effective solution to overcome these limitations is equipping the robot with terrain awareness: a traversable path can be planned based on the robot's preferences on terrains [16, 17] with an appropriate distribution of contact heights and forces [18] . Unlike obstacle estimation, the distinct terrain features are typically encoded as semantic information, whereas traditional methods collect sufficient data and train segmentation or classification models [19] to learn these features. Nevertheless, compiling an exhaustive catalog of all conceivable terrains and their corresponding walking preferences is impractical [20] . Compounding the issue, the dynamic real-world conditions, such as lighting, humidity, and temperature, may introduce inaccuracies in the correspondence between images and walking preferences, especially when relying exclusively on vision in this context [21] .To address the mentioned challenges of relying solely on vision and ignoring motion states in path planning, we complement the vision-only terrain estimator with online corrections derived from motion evaluations. We construct a proprioception advisor to convey information about the traversability cost of novel terrains and alert the robot to unexpected disturbances, such as invisible obstacles.By integrating the Terrain estimator, Obstacle estimator, and Proprioception advisor, we formulate TOP-Nav: a hierarchical path planning and motion control framework navigating a quadruped robot through diverse and challenging terrains proficiently. Within TOP-Nav, we develop a terrain estimator trained from previously collected data to inform the robot of terrain traversability. For novel terrains, we compensate proprioceptive history to offer online corrections for the vision-based estimation of unexpected and unknown environmental disturbances. We evaluate TOP-Nav both in simulation and on a physical robot, with a comparative analysis against existing legged navigation systems. 
related work 2.1 vision and legged proprioception integration: 
 Vision-aided legged navigation has been extensively explored in existing literature [12, 11, 22, 14, 15] , with performance heavily dependent on a robust perception module [23] . This reliance makes transferring a specific system to different hardware platforms difficult and costly. Recent research has introduced proprioception to improve task planning, provide comprehensive task observation, and reduce dependence on vision systems. A majority of these works learn proprioception representations along with visual features in simulation and then implement the cross-modal features through end-to-end [24] , hybrid [25] or decoupled frameworks [26] . Despite the effectiveness demonstrated in these works, the high-dimensional representation space presents challenges for adaptation to novel scenarios and sim-to-real transfer. Alternatively, Fu et al. [27] introduced a hierarchical navigation framework that derives evaluation scores from motion states, yet overlooks visual observation integration. To mitigate these limitations, we propose a novel approach within TOP-Nav by maintaining a series of lightweight cost maps derived from multi-modal observations. This integration achieves a dynamic balance between vision and proprioception. Furthermore, we leverage the learning-based locomotion controller to derive motion evaluations from the value function, offering an efficient solution without additional training.Terrain traversability is determined by factors such as terrain geometry, texture, and physical properties [28] . These features could be estimated by identifying the semantic class with a predefined static traversability score [29, 19, 30, 16] . These solutions notably depend on large-scale datasets [31] or are limited to structured environments like urban scenarios [32, 33] . In off-road navigation, the motion states involved in the dynamic interactions between the robot and the environment provide valuable metrics for assessing terrain traversability [34] . These insights have inspired methods that eliminate the need for manual annotation by autonomously deriving terrain traversability from proprioception through self-supervised learning [21, 35, 36, 37, 38, 39] . Nevertheless, the performance of these studies is contingent upon the quality of the collected datasets [28] . Researchers have proposed various approaches to handling novel observations to emphasize the challenges in unconstrained navigation. For instance, Frey et al. [20] updated the traversability estimation network online with anomalies into consideration. Karnan et al. [40] performs nearest-neighbor search in the proprioception space to align visually novel terrains with existing traversability. Drawing inspiration from those works estimating traversability for novel terrains, we propose a terrain estimator that employs the proprioception advisor as online corrections. Our method diverges from previous approaches primarily in two key aspects: 1) We employ an estimated value function from reinforcement learning to assess terrain traversability, providing a comprehensive evaluation of robot-terrain interactions. 2) We make online corrections on the vision-based estimation without additional training, providing a data-efficient solution for identifying novel terrains. 
background: 
 Learning-based legged locomotion controllers have been well developed through reinforcement learning, which is generally achieved by updating the policy π 1 within the asymmetric actor-critic training:EQUATIONthe locomotion policy receives privileged observation o p t , proprioception observation o i t , scanned dots external observation o e t and historical observation o h t respectively. π actor is modeled as a Gaussian policy and infers the optimized actions a t to compute the joint positions q des . c t stands for the estimated value function from π critic , which is updated through:EQUATIONs i denotes the robot state, R(s i ) represents the rewards accrued at timestep i, which commonly includes guiding the robot to track a given velocity command with stable gaits and attitude. Substantial efforts in reward engineering to formulate R(s i ) have enabled the value function c targ t to evaluate a comprehensive set of interactions between the robot and its environment. This provides an essential foundation for the proposed proprioception advisor to leverage the estimated c t for motion evaluations within the path planner.  
method: 
 4.1 System Overview TOP-Nav connects a path planner and a motion controller to tackle the task of legged navigation (Fig. 2 ). The path planner generates waypoints from an integrated cost map, M C , which is built around the robot and updated through online proprioception and visual observations. M C offers a comprehensive estimation of external environment that encompasses terrain traversability costs M T , obstacle occupancy M O , proprioception advice M P and goal approaching M G . The planned waypoint is tracked by the RL-learned locomotion controller. In the following section, we will highlight the terrain estimator and proprioception advisor to demonstrate how the proposed system informs the robot of unexpected obstacles and unknown terrains based on walking experience. Details on the integration of the costmaps within the path planner are provided in the appendix (Section 7.2). We design the proprioception advisor to identify motion abnormalities that may arise from unexpected external disturbances. To offer a stable and comprehensive evaluation of the robotenvironment interactions, we train the estimated value function (Section 3) to synthesize multiple metrics and incorporate historical observations. As demonstrated in Fig. 3 (b), the motion evaluation will decline sharply when the robot encounters a locomotion failure caused by transitioning onto challenging terrain. The reward design and normalization implementation details are provided in Section 7.4.Within the path planner, we first directly utilize the motion evaluations to enhance the robot's awareness of unforeseen disturbances. This involves estimating invisible collisions (M P ) and integrating them into the combined cost map (M C ). We propose a continuously varied proprioception cost along the lateral direction (which is the y-axis in M P ).EQUATIONhere k P is hyperparameter and c t is the estimated motion evaluation. Since a lower motion evaluation indicates a potentially challenging terrain or an invisible obstacle in the current direction of the robot, we allocate c t to the centroid column in M P and decrease it towards the edges, this will prevent the robot from continuing to move forward when its motion is disturbed by an unexpected obstacle or challenging terrain. We further discuss the relationship between the proposed motion evaluations, terrain difficulty, and ground truth motion states in Section 8.3.The proposed terrain estimator first leverages prior knowledge to map the visual observations to the reference terrain traversability. We apply a perspective transformation on I rgb to map the pixels to bird-eye-views M BEV , which has corresponding coordinates as M T . We then discretize M BEV into patches (Fig. 3(a) ) and assign the same difficulty within each patch. The terrain within the prior knowledge D terrain is identified through a terrain classification network π terrain .Considering that such identification falls short when faced with unfamiliar terrains, we calculate the predictive entropy to approximate the identification uncertainty [41] and obtain the confidence Conf of current visual estimation accordingly. With P i denoting the predicted probability of terrain i, the predicted explicit T errain names and confidence of the prediction for the current patch can be calculated as:EQUATIONthen a vision based traversability cost map M T O could be directly obtained from a pre-defined (T errain, Cost) mapping. The mapping process leverages experience from previous works [20, 37] , where we collect motion evaluations as the robot traverses various terrains in D terrain (Fig. 3 (a)). Terrains with higher motion evaluations are assigned higher traversability scores and lower terrain costs. Detailed discussions are provided in section 7.5.Through the seamless integration of vision and proprioception, we provide online corrections for the vision-based traversability estimation M T O without additional training (Fig. 3(a) ). This enables the robot to terrain awareness beyond the limitations of the collected data D terrain .We address the mechanism by which the robot recalls the traversability of terrain once it has been seen and traversed within the navigation process. With the robot location (x base , y base ), we record a duration of 1s proprioception advice to indicate the terrain traversability cost at the robot location:EQUATIONBesides T P (x base , y base ), we can access a latent feature L(x base , y base ) extracted by the classifier π terrain when the same patch was observed a few steps ago, we record both T P and L at location (x base , y base ) into an experienced list P e . Now given a new observation at location (x o , y o ), We can find a traversed and seen patch (x r , y r ) that looks closest to (x o , y o ) based on cosine similarity:EQUATIONNote that we also have the historical traversability cost T P (x r , y r ) infered by the proprioception advisor at (x r , y r ), we can compute a proprioception adapted traversability cost with the similarity S (o,r) to access the historical proprioception correction M T P (x o , y o ):EQUATIONdue to the delay in estimation from the proprioception history, we intend for M T P to be utilized when encountering novel terrains where M T O becomes unreliable. Therefore, we normalize the visual uncertainty U (x o , y o ) into confidence Conf (x o , y o ) to adjust the contribution of M T O and M T P :EQUATIONhere k T3 , S 0 , k T4 and C 0 are hyperparameters.By integrating the adapted M T as the terrain costmap into the path planner, our system demonstrates rapid adaptability to different terrains and visual conditions. 
experimental setup: 
 We assess TOP-Nav in challenging navigation environments across both simulations and real-world scenarios, emphasizing the following evaluations: 1) The improvements in navigation performance and locomotion stability achieved by introducing terrain awareness and motion evaluations into the navigation system.2) The effectiveness of the proposed Proprioception advisor.3) The effectiveness of the proposed Terrain estimator faced with novel terrains.Evaluation Settings Our simulation experiments are conducted within Nvidia Isaac Gym (Fig. 4 ). We create a 8 × 8 independent navigation cells grid. Each cell is 5m × 5m in size, featuring a robot assigned to a point goal navigation task. The robot and point goal is randomly generated, with a minimum initial distance of 5m within the cell. The simulation experiments are conducted 25 times in each of the 64 navigation cells. For real-world evaluations, we conduct outdoor navigation tasks across different scenarios involving challenge obstacles and various terrains, with each scenario replicated 5 times for each method under investigation.We evaluate TOP-Nav with the following metrics: SR (Success Rate): The percentage of successful experiments. We define a success experiment as approaching the point goal into 0.5m within 20 seconds; TD (Terrain Difficulty): The percentage of average traversed terrain costs, TD provides the ground truth traversed terrain costs within each episode; UT (Unstable Time): The percentage of unstable motion states (|roll| > 0.15 or |pitch| > 0.15) in each episode; VFT (Velocity Tracking Failure): The average percentage of velocity tracking failures (∥v lin -v act ∥ > 0.2) in each episode; AEC (Average Energy Consumption): The average energy consumption (τ q) [42] in each successful episode. We calculate the variance across different navigation scenarios to assess the robustness of the proposed method.Baselines: Beyond ablation study, we compare TOP-Nav against state-of-the-art legged navigation frameworks and segmentation based terrain estimators. VP-Nav [27] integrates vision and proprioception to develop a collision detector and a fall predictor within the navigation pipeline. GA-Nav [19] achieves terrain segmentation relying solely on vision. Sterling [37] learns terrain traversability in a self-supervised manner by assigning traversability to terrains with similar proprioception representations. However, this approach requires prior training for encountered terrains.Improvements with Terrain Awareness: TOP-Nav empowers the robot to select terrains with higher traversability, leading to a significant improvement in the success rate of navigation: TOP-Nav surpasses the VP-Nav baseline by approximately 8% in success rate (SR) and achieve a TD of nearly 20% (Table 1 ), which is half of the TD achieved in methods without terrain awareness. Improvements with Proprioception Advisor: The proposed proprioception advisor could recover the robot from locomotion failures such as getting stuck on irregular terrain surfaces or colliding with unseen obstacles during directional changes (Fig. 4 ). In contrast to methods without proprioception, our approach exhibits an approximate 11% enhancement in SR. Meanwhile, even without terrain awareness, the proposed system outperforms VP-Nav by 2.5%, signifying an improvement in our integration of the proprioception advisor compared to existing methods.Advancements in Locomotion: TOP-Nav achieves the lowest UT and VFT, indicating that selecting simpler terrains contributes to locomotion stability. We evaluate energy consumption with the assumption that when traversing simpler terrain, the robot should exhibit more natural gaits. As a result, TOP-Nav exhibit a 10% reduction in energy consumption compared to methods without terrain awareness.To evaluate the efficacy of the online corrections for providing the robot with traversability on novel terrains, we conduct experiments using a degraded terrain classifier trained without gravel (Fig 5(A) ) and in terrains without any prior information (Fig 5 (B) ). We demonstrate that the robot can recall previously encountered challenging terrain during the phase of online corrections and plan a waypoint to avoid it. This showcases the fast adaptation ability of TOP-Nav to plan an optimal path when faced with novel terrains.The quantitative results are provided in Table 2 . In the Unkown Gravel experiments, despite GA-Nav possessing complete prior knowledge and correctly identifying the paved road, our online corrections only lead to a 7% increase in UT, mainly due to the forward movement in phase 1. VP-Nav allows the robot to exit the gravel terrain with the safety advisor, but it can not retain this infor- In addition, we conducted a consecutive long-distance experiment (Fig. 5-C ) in an off-road scenario with grass and gravel on both sides of the path. The robot is initially given a target direction straight ahead, relying on the terrain and obstacle estimator to keep navigating on the paved road. In such scenarios, the challenging terrains help guide the robot in changing its direction, ensuring that it stays on the paved road automatically. More hardware evaluations are provided in Section 8.2 and the supplementary video. 
conclusion: 
 We present TOP-Nav, a legged navigation system that achieves closed-loop integration of visual and proprioception at both task and motion planning levels. Through the extensive quantitative experiments conducted in both simulation and real world, we underscore the success of our system in achieving open-world navigation, surpassing limitations posed by visual conditions or prior knowledge. 
motion controller details: 
 The motion controller is implemented following [3, 43] . Both the actor and critic networks within the locomotion policy π have hidden layer sizes of [512, 256, 128] . The proprioception observation o p t includes angular velocity (3), Orientation (2), velocity commands (3), joint positions (12) , joint velocities (12) , and the last action (12) . We store the last 10 steps of o p t into the history observation o h t . The depth encoder learns the exteroception latent features from onboard observation I d using a conv-GRU structure. π is updated using PPO [44] for 20K iterations, the batch size is 160000 divided into 4 mini-batches. The depth encoder is optimized for 5k iterations. The rewards obtained at each step are the sum of the reward functions listed in Table 4 .The default π critic requires privileged observations, we train another value function estimation network using observable motion states for the proprioception advisor to be deployed on hardware. The estimated motion evaluation c t is normalized using a sigmoid function. The proprioception advisor is trained with the locomotion policy π, with an exact same network architechture as π critic . The input only includes proprioception observation o p t and historical observation o h t , updated through the MSE loss with c targ t . The estimated value c t is normalized through sigmoid function: In practice, and we set a minimum threshold c th for the normalized critic value be considered in the path planner, in simulation c th = 0.8, while in the real world c th = 0.5, N orm(c t ) larger than c th will be set to 1.N orm(c t ) = 1 1 + e 2 * (2.2-ct) (13)M P is calculated with k P = 0.3, we demonstrate an example of M P after the robot collides with a wall (with vision and obstacle detection disabled in this trial) in Fig 6 . In this scenario, the normalized c t returns 0.02, M P has a width of 0.75m along the moving direction of the robot with costs greater than 0.5. Generally, the proprioception advisor engages when locomotion states are compromised. To enable recovery from being obstructed by the unexpected terrains, we have designed the following recovery strategy in simulation: when N orm(c t ) falls below a specific threshold (0.5), we initially assign a backward velocity command of 0.5 m/s for 0.5 seconds. Next, to address potential terrain-related constraints that may prevent the robot from reaching planned waypoints at its default velocity, we reduce the frequency of the path planner to 0.5 Hz until the robot reaches the waypoint or makes a directional change of 0.3 radians.For prior data collection of the terrain estimator, we teleoperate the robot to walk for 5 minutes on each of the concerned terrains. We capture the motion evaluations c t as well as the first-view observations during these demonstrations. We convert the RGB observations into BEV maps and dividing them into patches. The collected data is labelled based on the corresponding terrain from which it was acquired. The classification network is implemented using the MobileNet backbone and trained on a Nvidia GTX 2080 Ti for 6 hours. The proposed pipeline provides a light-weight inference model of 8 MB to be deployed for onboard computation. During deployment, to approximate the uncertainty U with predictive entropy, We perform K = 8 stochastic forward inferences with different data augmentations as Monte Carlo simulation, We estimate the expectation as the average of the predicted probability P i = k P k i /K. Based on the collected motion evaluations, we compute the average c t observed during walks on each terrain. These walking experiences provide reference values for assigning the terrain costs defined by the operator. Table 5 presents the validation accuracy of the terrain classifier, along with the reference terrain cost and the average collected motion evaluations N orm(c t ) for each terrain. Since the terrains encountered in the real world did not exhibit significant differences in motion evaluations, we did not strictly set the reference cost based on linear correlation. The hyperparameters used for online corrections are fine-tuned based on the distribution of the learned latent features L from the pre-collected data. Figure 7-(b ) depicts a t-SNE visualization of the features inferred by the terrain classifier, involving data both within and outside of D terrain . We observe that π terrain can learn distinctive features for specific terrains, showcasing unique clustering in the latent space, even for previously unseen terrain types. The average cosine similarity S (o,r) within each terrain class is 0.95, we set the middle point S 0 to be 0.85 and k T3 to be 15. This allows the terrain cost derived from historical experience to decrease rapidly when the similarity falls below 0.8. We implement TOP-Nav on the Unitree-Go1 and Unitree-Go2 quadruped robot, with a weight of approximately 12kg and dimensions of 645mm × 280mm. The robot is equipped with 12 brushless motors, each capable of producing a torque of 35.5N m. The perception module is equipped with a RealSense D435i camera, offering simultaneous depth and RGB channels. All computations are processed onboard with an NVIDIA Jetson NX asynchronously, i.e. the locomotion controller operates at a fixed frequency of 50Hz and receives the latest depth latent ℓ inference from the perception module. The path planner operates at a frequency of 3Hz. 
additional experiments: 
 In this section, we provide additional experiments to justify the effectiveness of the key components in TOP-Nav.This section gives a detailed description of the evaluation choices throughout the simulation and hardware experiments. We set The Timeout for Success Rate at 20 seconds, which corresponds to an expected path length of 10 meters given the commanded velocity of 0.5m/s. Note that the target goals will be randomly generated at least 5m away from the robot initial location within the 5m x 5m nav-cell. Therefore, 10 meters represents the longest path the robot would plan to complete the navigation episode without backtracking. We set the relatively loose time limit to emphasize the robot's ability to complete the task, as avoiding challenging terrains and obstacles may require taking detours. The experimental results demonstrate that most failure cases occur when the robot gets stuck due to obstacles or challenging terrains. As a result, extending the timeout limit does not lead to a significant change in the success rate. For example, when we raise the timeout limit to 30 seconds, the success rate for TOP-Nav increases from 73.62% to 75.93%, the success rate for Obstacle-Only raises from 52.81% to 59.39%.We set the metrics of Unstable Time and Velocity Tracking Failure to evaluate the advantage in motion stability when the robot follows the planned path using the proposed system. Generally, the robot achieves higher speeds with less roll and pitch when walking on the optimal path. As demonstrated in Fig. 8 , the robot performs better on these metrics across different threshold settings.We conduct additional quantitative hardware experiments on terrains included in the prior datasets (Fig 9 ), demonstrating superior performance compared to existing legged navigation systems. The evaluation results are presented in Table 6 . TOP-Nav successfully completes all 15 trials with superior walking stability and minimal energy consumption. These results affirm the successful deployment of the proposed system on real hardware, allowing the robot to select waypoints on terrains with better traversability and thus execute more natural gaits. In contrast, GA-Nav, trained on RUGD [45] , demonstrates notable limitations when confronted with open-world navigation. We observe that the improvements of TOP-Nav over GA-Nav mainly stem from: 1) In scene 1,2, GA-Nav fails to identify the slabbed road as a low-cost terrain from the bush and gravel. 2) In scene 3, GA-Nav incorrectly identifies the lower part of the obstacle as a cement floor, which could be addressed by our integration of obstacles and terrain estimation. While VP-Nav effectively alerts the robot to locomotion failures that could be brought by such terrains, the absence of terrain awareness prevents the robot from successfully navigating out of these challenging terrains, resulting in inferior locomotion performance.In this section, we discuss the relationship between the proposed proprioception advisor and terrain difficulties, along with ground truth reward terms such as velocity tracking and attitude stability.We conduct a series of straight navigation tasks in simulation, as demonstrated in We conduct 30k steps of navigation for each robot and compute the average motion evaluations estimated by the proprioception advisor. The results are shown in Table 7 . We observe that as terrain ruggedness, irregularity, and the presence of obstacles increase, there is a corresponding decrease in the motion evaluations. Although this correlation is not strictly linear, the results provide supportive for us to estimate terrain traversability based on the proprioception advisor. As shown in Table 7 , c ref demonstrates a noticeable decline as terrain difficulty increases, yet it does not exhibit a significant advantage compared to the proposed proprioception advisor.On the other hand, the same terrain may exhibit different traversability for robots with varying locomotion capabilities. Therefore, the evaluation of walking states represents another important metric for assessing terrain traversability. We demonstrate the correlation between the predicted motion evaluations c t and the ground truth motion states c target ref in Fig 11 .The perason correlation coefficient is 0.914, indicating a strong linear correlation between the estimated c t and the actual motion states of the robot.Building upon the discussion above, we validate that constructing the proprioception advisor with critic output enables efficient motion evaluation, including information on both the terrain difficulty and walking states of the robot. Moreover, the proposed advisor can be implemented without the need for additional training or sensors, making it a more appropriate approach to provide motion evaluations for robot path planning compared to existing methods.We conducted quantitative experiments in simulation with different velocities (0.25, 0.5, 0.75, 1.0) (m/s), and the results demonstrate that proper velocity design does affects the performance of the navigation system. As demonstrated in Table 8 , a slower velocity command (0.25) prevents the robot from reaching the goal on time, resulting in a lower success rate. Moreover, it can be concluded that lower the speed would help decrease the energy consumption. Table 9 : Evaluation results on the Invisible Obstacles.On the other hand, setting a too large velocity command will introduce locomotion instability, as indicated by the results with a velocity of 1m/s. However, conducting a quantitative analysis of the influence of speed is challenging in complex terrains. For instance, in slopes, robots with higher velocities may navigate more effectively, whereas terrains with discrete steps could lead to locomotion failures at faster speeds. Consequently, experiments with velocity commands of 1m/s still achieve a high success rate.The performances vary between 0.5m/s and 0.75m/s is not significant; therefore, we consider this range to be an appropriate commanded velocity range for deployment. In real-world experiments, we demonstrate the robot's velocity set at 0.75m/s in the supplementary video. 
 
 Appendix: The camera-based vision system in TOP-Nav fails in providing robust visual odometry in varying light conditions in outdoor environments. This limitation could be mitigated by integrating LiDAR-odometry into the perception module. Another significant limitation is related to the costmap-based local planner, where the planned waypoint can continuously shift when similar minimum costs arise at different locations on the map. We plan to address this in future work by introducing a diffusion-based task planner, which can integrate different cost modalities while providing a smoother sequence of future waypoints.To obtain a more comprehensive observation in the real world, we configure the cost map M C with dimensions of (3m, 3m). The location of robot is at p map base = (0m, 1.5m) in real world M C . The criteria for success include reaching the goal within the specified time constraints, consistent with the simulation setting.Obstacle Estimation and Localization: In simulation, the robot has access to the ground truth location. For each point p = (x, y), M G (x, y) = ∥p goal -p∥. In real world, we set a target direction r goal to compute the goal map M G (x, y) = ∥r goal -arctan(p -p base )∥. We construct the obstacle estimation based on the depth channel I d . The perceived obstacles are converted into point clouds, and for each point in M O , we compute the signed distance M SDF to the closest obstacles within distance d max , the cost of obstacles therefore can be computed as:EQUATION)For reference, we provide the default values of the hyperparameters used in the path planner in Table 3 . Parameters d 0 , t 0 adjust the weights of M T and M G as the robot approaches the target and as time progresses. k G0 keeps a minimum weight for the robot approaching the target. The listed values remain consistent in simulation; however, considering the various conditions in the real world, we make slight changes to these values in different real-world experiments for better validation of the contribution of this work. For instance, in outdoor navigation experiments, we assign a higher value to k T1 to expand the scale of the terrain estimator, which is effective for evaluating the accuracy of the proposed terrain estimator.The default commanded velcoity v 0 is set to 0.5m/s in both simulation and the real world. The maximum distance d max we considered for computing the obstacle map M O is 0.3m. 8th Conference on Robot Learning (CoRL 2024), Munich, Germany. 


Decision: In summary, the initial reviews highlight the following strengths and weaknesses of this submission.

Strengths:
- The paper proposes an interesting framework for online adaptation for legged robot navigation.
- The paper provides experimental results with a real platform showing impressive and sensible behavior of the robot.

Weaknesses:
- The paper lacks sufficient details and comprehensibility to fully understand the method.
- The motivation of several design choices is not well explained.
- The experiments are not fully clear and convincing. For instance, it is not clear how the joint consideration of different modalities improves performance, or the thresholds for evaluation metrics seem arbitrarily chosen.
- The paper lacks discussion of limitations, failure cases, and assumptions.

After the author response and discussion phase, the reviewers have been mostly satisfied how the authors adressed their comments, raised their scores and recommend accepting the paper. The authors should additionally consider improving the structure of the paper and providing a definition of traversability as suggested by the reviewers.

The revised paper must not violate the page limit for submission.

Reviews: review: title: title: An interesting “unifying” method for navigation that requires further clarifications
paper_summary: paper_summary: The paper represents a method for legged navigation that combines different sources of information, such as RGB vision for semantics, depth estimates for collision avoidance, and proprioceptive sensing for dealing with unobservable obstacles. The approach is based on a RL approach with a costmap representation.  The experimental results are done in simulation and real platforms, showing advantages to other similar methods.
main_review: main_review: **Summary**
I believe the paper is interesting but it’s hard to understand the full method. I get that this is in part because the method builds upon 3 different components, but it’s challenging to grasp the details, as many details are missing in the main text and, if not explained in the appendix, are not explained at all. Lastly, the experiments shown are not particularly challenging for a legged platform to highlight the different modalities supported by the method.

**Strengths**
The overall system proposed is sound. I appreciate that there are experiments performed in closed-loop with the real platform, displaying sensible behavior for a legged platform.

**Weaknesses**
In my opinion, the main weaknesses of the paper are in the design decisions to combine all the sources of information: It is not clear if the goal must be defined within the costmap to be found, it is not clear why the depth information is used twice (in the costmap and the motion planner), it is not clear what terrain estimator is being used, or why defining a virtual barrier in front of the robot for proprioceptive sensing is adequate too. I will provide more details about these questions in the next part.
questions: questions: - Please explain why the motion planner needs to be learned if most of the planning happens in the cost map in the end. Why is a traditional local planner insufficient for this purpose?
- Similarly, how does the motion planner leverage depth information twice? (as an input, and as part of the costmap).
- How do you train the method in simulation if the vision input cannot be simulated? Wouldn’t this affect the policies learned by the motion planner?
- The idea of combining different cost maps recalls the work by Wermelinger (https://ieeexplore.ieee.org/document/7759199). Other recent approaches have also been proposed to combine “fields” for navigation (e.g. https://arxiv.org/abs/2201.03938 ). None of them are discussed.
- It is not really clear how the costmap is defined. Is it aligned with the robot’s pose or is it dependent on some fixed frame (like ROS’ grid_map)? How about the magnitude of the cost itself? Is it [0,...,1]?
- How are the semantic classes (from the visual terrain estimator) mapped to costs? Are the values reported in Table 9 hand-crafted?
- The overall presentation of the experiments requires further clarifications. Experiments are presented in an arbitrary way, not really explaining the purpose of them, the conditions, and what’s being assessed. 
- Further, without discussing and providing further insights on the failure cases and limitations it’s really hard to assess and understand why the method could overcome challenges of previous approaches. It is not clear to me from the experiments that all the components together ensure the success of the method, since the experiments demonstrate examples that can be solved by one “modality” at a time - the glass door is solved by proprioception and depth sensing (similar to Fu (2022)), the path tracking experiment is similar to Wellhausen (2019, not cited either: https://ieeexplore.ieee.org/document/8627373) or Frey (2023), who solved it with vision only. It would be necessary to provide a further analysis and demonstration on how these different modalities and the overall system design comes together and effectively justify the success of the approach.

Minor text fixes:
- _“Legged navigation is typically examined within open-world, off-road, and challenging environments“_ -> I’m sure that examined is not the right word to use here, but I’m not sure what was the intention of the sentence to suggest an alternative
- _“insimulable”_ -> does not exist, I suggest to write “hard to simulate” instead
- _“open-loop local planner”_ -> a local planner is closed-loop by construction, as it uses online sensing to compute local paths
- _”A comprehensive legged navigation framework Integrating multi-modal”_ -> “integrating” should not be capitalized
- _“As demonstarted in Fig. 3(b)”_ -> should be “demonstrated. There are way too many other typos in the figures and the text, please revise.
- Please do not use red and green together for the figures. They are hard to distinguish for colorblind people.
limitations: limitations: 1
review_summary: review_summary: I believe the paper needs a general revision: the method is presented in a convoluted manner, important pieces are not explained, and it is not clear if the experiments are really assessing the important aspects and contributions of the work. Apart from presentation issues that were also raised in terms of the wording and typos.
title: title: Review of TOP-Nav
paper_summary: paper_summary: This paper presents a method for integrating perception in the form of terrain classification, obstacle identification, and proprioception for the purposes of improving legged locomotion over challenging and potentially unseen terrain.  The proposed framework is validated in simulation and in real-world experiments and is compared against existing approaches. The contributions are further validated via an ablation study examining the contribution of each of the proposed elements.
main_review: main_review: The proposed method for integrating terrain awareness, obstacle avoidance, and proprioception into reinforcement learning of legged locomotion policies provides an interesting framework for online and continual learning for legged robots. While the authors validate the framework in simulation and in real-world experiments, the thresholds chosen for the evaluation metrics feel arbitrarily chosen. Furthermore, the methodology of the paper is poorly written and hard to understand, making the review of the contribution difficult and the comparison against existing works challenging. 

Major Comments:

1. Section 4 is has many grammatical errors and the explanations behind the presented concepts are confusing. For example, statements in Section 4.2 such as "along the y-axis" don't make sense when the coordinate frames have not been introduced, y_base and y in Eq.3 are not introduced, etc. Furthermore, there are mathematical inaccuracies in Section 4. For example, Eq.3 has x as an input, but this does not appear in the evaluation of the right-hand side. Overall I would recommend editing Section 4 so that the presented concepts have a more well-defined motivation and are more clearly explained without requiring the Appendices.

2. The coordinate frames of M are not discussed. Are these cost maps in front of the robot or is the robot centered within them? If they are in front of the robot, how are network predictions for things like the terrain estimator smoothed as the robot moves or as new images are considered? 

3. Eq.4 introduces U and T, but both of these values are not used in the paper and only appear in the Appendices.

4. In Section 4.4 it is unclear when the transition between M_To and M_Tp happens. There are other confusing issues in this section as well, such as what the domain and co-domain of the T_P and Norm functions are, why the traversability score T_P is not a binary variable (0 for intraversable, 1 for traversable), or how these local maps M are updated as the robot moves.

5. In Section 5, the thresholds for the evaluation metrics seem to be arbitrarily chosen. For example, why is the timeout for SR (Success Rate) set to 20s? To bolster the authors' claims that their proposed method is in fact better than existing approaches, I would like to see a graph demonstrating the success of all methods as these thresholds are varied (i.e. a graph where the x-axis is timeout in seconds and the y-axis is SR (Success Rate)). The same holds for the UT (Unstable Time) and VFT (Velocity Tracking Failure).

6. The limitations of the proposed methodology are not discussed.

Minor Comments:

 1. In Eq.4 it is incorrect to state U represents uncertainty. Rather, it represents entropy. The authors' may use this as a proxy for uncertainty, but this should be stated clearly.

2. The notation T is overloaded in Eqs. 4 and 5. I would recommend replacing the T in Eq.4 with another variable name.

3. "Traversability" is poorly defined. Intuitively, a region of the robot's environment is either traversable or it is not (1 or 0, respectively). Representing traversability as a continuous variable is thus confusing (i.e. How is a region 0.6 traversable?). Furthermore, this "traversability score" not only varies between robots (i.e. large vs small quadrupeds, bipeds vs quadrupeds), but also has intra-gait variation (i.e. trotting vs static gait). If the RL policy is not constrained to a particular gait, wouldn't traversability vary for the same patch of terrain? For example, if a quadruped tries to run across ice, it would fall (intraversable), but if it walked across slowly it would successfully cross the ice (traversable).
questions: questions: 1. More clearly explain the methodology in Section 4 and fix grammatical errors throughout the paper.

2. Evaluate the effect of the evaluation metric thresholds on the proposed method and comparisons.

3. Include a more thorough discussion on the limitations of the proposed methodology.

4. Address minor comments.
limitations: limitations: 1
review_summary: review_summary: The proposed framework in well validated, however grammatical errors, poorly communicated methodology, and evaluation metric thresholds need to be addressed.
title: title: Los of impressive results but lots of clarification needed.
paper_summary: paper_summary: This paper introduces a framework for estimating traverabilitiy for a legged robot. The method can be updated online with historical experiences.
main_review: main_review: This paper presents a useful idea and show a lot of impressive results. The experiments, especially in the attached video, do a good job of showing the performance of the system. A large amount of integration work is needed to execute this method on a real robot which makes the results very impressive. There is also a good comparison to baselines and sufficient ablation studies to justify design choices. However, I feel that too much is written about the system as a whole when in reality the main contribution is the terrain estimator. As a result, lots of important details about the terrain estimation are missing or confusingly distributed between the main paper and the appendix. For example, I would suggest the authors change Section 3 to discuss less the locomotion policy and more the terrain sensing such as input signals. 

Therefore, I mainly have implementation questions and requests for clarification which I will write in the following section. I also recommend reducing content in the paper about the system and instead moving appendix details on the terrain estimator into the main body of the paper.
 
The main concern I have about the method overall is: how well does the historical terrain experience list scale with large terrain? How often is new terrain information added to the list? What will happen if the robot's position estimate drifts over time?

Finally, there is no limitations section in the paper. My understanding is the paper can be rejected on the grounds of not including a limitations section so I would strongly urge the authors to look at other submissions to CoRL and add a limitations section.
questions: questions: 1. Add a limitations section.
2. Please address the question of scalability for the terrain experience list and explain how this method works over larger distances.
3. Please clarify the following details:
 - What exactly are the proprioceptive signals used? Joint velocities? IMU? Please explicity describe all sensing including dimensionality.
 - How is the proprioception advisor signal mapped to the Mp like in Figure 10? Why does a collision in the front show low cost behind the wall?
limitations: limitations: 1
review_summary: review_summary: I lean towards accept but there are some issues with the paper which need to be addressed.