Learning to See Physical Properties with Active Sensing Motor Policies
Authors: Gabriel B. Margolis
Xiang Fu
Yandong Ji
Pulkit Agrawal
Venue: CoRL 2023 Poster (2023)
Keywords: Locomotion
Vision
Navigation
Reinforcement Learning

Abstract: To plan efficient robot locomotion, we must use the information about a terrain’s physics that can be inferred from color images. To this end, we train a visual perception module that predicts terrain properties using labels from a small amount of real-world proprioceptive locomotion. To ensure label precision, we introduce Active Sensing Motor Policies (ASMP). These policies are trained to prefer motor skills that facilitate accurately estimating the environment’s physics, like swiping a foot to observe friction. The estimated labels supervise a vision model that infers physical properties directly from color images and can be reused for different tasks. Leveraging a pretrained vision backbone, we demonstrate robust generalization in image space, enabling path planning from overhead imagery despite using only ground camera images for training.

Learning to See Physical Properties with Active Sensing Motor Policies
Gabriel B. Margolis
Yandong Ji Improbable Al Lab, Massachusetts Institute Technology
Xiang Fu
Pulkit Agrawal
https://gmargolt github.
Abstract: To plan efficient robot locomotion, we must use the information about terrain's physics that can be inferred from color images. To this end, we train a visual perception module that predicts terrain properties using labels from small amount of real-world proprioceptive locomotion. To ensure label precision, we introduce Active Sensing Motor Policies (ASMP). These policies are trained to prefer motor skills that facilitate accurately estimating the environment's physics, like swiping a foot to observe friction. The estimated labels supervise a vision model that infers physical properties directly from color images and can be reused for different tasks. Leveraging pretrained vision backbone, demonstrate bust generalization in image space, enabling path planning from overhead imagery despite using only ground camera images for training.
Introduction
In recent years, legged locomotion controllers have exhibited remarkable stability and control across wide range of terrains such as pave- ment, grass, sand, ice, slopes, and stairs [1, 2, 4, 5, 6, 8]. ofate-of-the-att approaches use sim-to-real learning with a combination of pro- prioception and depth sensing to perceive the ground beneath the robot or obstacles around [5, 7, 8, 9, 10, 11, 12, 13, 14] but have dis- carded a valuable signal about terrain's physi- cal properties that is conveyed by color images. To utilize this information, some works learn to predict locomotion performance or interac- tion properties from terrain appearance using data collected in the real world [15, 16, 17, 18]. However, the learned representations in these works are task- or policy-specific. Instead, we propose directly predicting the terrain's 's physi- cal properties (e.g. friction, roughness) that (a) can be simulated and (b) are invariant to the pol- icy and task. With this approach, we can learn a cost map from simulated rollouts to inform traversal planning when performing a new task (like payload dragging) or optimizing new ob- jective (like a preference for speed vs energy efficiency).
Active Sensing
Motor Policy
Traversul
Labels
Pretruined Backbone
Egocentric
Tixai
Images
Features
Dense
Physics
Physies Adupter
Infer
Aerial
Images
Grounded
Perception
Module
Nomtnal Plan
Cost Function
from Simulator
Paylond Plan
Figure 1: Learning to see how terrains feel. We
propose (1) learning optimized gait for collect- ing informative proprioceptive terrain labels that (2) used to supervise training for vision mod- ule, which can (3) be used for navigation planning with new tasks and image sources.
A natural way of estimating the terrain's physical parameters during data collection is by training a neural network to predict them from the proprioceptive sensor history, supervised by the ground truth labels available in simulation [4, 5]. We discovered that the estimates obtained through this
7th Conference Robot Learning (CoRL. 2023), Atlanta, USA.

---

Passive Estimation L00 = E. Rdd Is, R.4 1n e.1l"
Active Estimation in Sensing Polictes (.0) E. log x to EaMp 10-
datie Eslonttion Rovard
St..
Policy
a m rSp a)
S...- H
Policy
ag m ns,, ag a,le, 2
RL Lara
Only
(SAll - Exnoattinn)
Estimnttr
Estimator
Supervivel Exiimation Lona
Sayerviod
Propgie Actemdten
Privileged
Privileged
Oraele
Oracke
Figure 2: Active Sensing Motor Policies optimize for estimation. Unlike passive methods (left), which estimate the state only to extent that observable as byproduct of control relevance, Active Sensing Motor Policies (right) directly incentivize improved state estimation through the advantage function. This incentivizes a policy to adopt information information-gathering behaviors, like inten- tionally swiping the robot's foot during legged locomotion, to improve the estimate quality.
approach can be imprecise because the locomotion behavior often makes the terrain properties hard to predict. Therefore, unlike prior works terrain perception that predict the terrain character or traversal cost from passive data [15, 16, 17, 18], we propose training specialized data collection policy that directly optimizes for terrain property estimation. This Active Sensing Motor Policy (ASMP) learns emergent locomotion behavior, such dragging the feet on the ground to better estimate friction, and improves the informativeness of its proprioceptive traversals.
We use the improved data obtained through ASMP as self-supervision to learn a visual perception module that predicts terrain material properties. The same model can inform efficient plans for nominal locomotion and for dragging objects by considering the impact of terrain properties traversal cost. Because the robot is low the ground, its onboard cameras only provide enough range for local planning. Sometimes the robot is in a position where it must plan its motion using only the information in front of it, but in other cases, it might have access to some global information about the environment. Therefore, we also consider the scenario of a teamed drone that flies above the legged robot and provides an extended view of the environment. Despite being trained solely with images captured from an onboard camera, our resulting model can also be evaluated to predict terrain properties using images from various cameras and viewpoints, which allows this type of global planning succeed.
2 Method
Our approach consists of the following stages, which are also illustrated graphically in Figure I:
We estimate the terrain dynamics parameter, et. from the proprioceptive sensor history during an initial blind traversal. Our Active Sensing Motor Policy (ASMP) crucially provides better-calibrated estimates than the baseline policy. In our experiments, the estimated parameter e, is the ground friction coefficient, the ground roughness magnitude, or both. (Section 2.1)
1. Active Sensing:
2. Self-Supervised Vision Learning: Using labels of e, recorded from the real-world traversal of the robot, we learn a function, é = ((), that predicts the per-pixel value of e, for a given image I. The labels for training are only available at the pixels corresponding to the places the robot traversed, but the resulting model can be queried to predict the terrain parameter any pixel. (Section 2.2)
Cost Function Learning: inform planning, we learn cost functions that relate the terrain dynamics to various performance metrics. First, we create terrains with range of e, values in simulation. Then, perform rollouts in simulation to measure cost function C(ex) that relates dynamics parameters to performance. We lear separate cost function for each task. (Section 2.3)
4. Dynam Path Planning: Combining (2-3), compute cost maps directly from color images and use them for path planning. (Section 2.4)

---

2.1 Active Sensing Motor Policies: Learning Whole-Body Active Estimation
In learning control policies under partial observations, it is commonplace train with an im- plicit [1, 2, 3, 5, 8] or explicit [4, 19] incentive to form representations within the policy network that correspond to the unobserved dynamics parameters. Consider the concurrent state estimation frame- work of Ji et al. [4], under which state estimation network is trained simultancously with the policy network to predict the unobserved parameters. The predictions of the state estimation network are concatenated with the rest of the observation to construct the policy network input. This approach optimizes two-part loss consisting of the standard policy objective and the state estimation error:
L(0, 8*) = E, log To T e) + lle - ēgr((s)))
This has been empirically shown to yield better policy performance in environments with random- ized dynamics or unobserved state variables [4].
In the formulation above, the estimation error is used to update the state estimator weights f', but not the policy weights 6. This does not incentivize the policy to adjust its actions improve estimation performance beyond what is required for control. Typically, this is problem because it allows the policy to maximize its performance at the current control task. However, our end goal is to use the output of the state estimator to train visual perception module that may be reused with other controllers and tasks. To support this, the labels should be as accurate as possible even when that is not necessary for control. To obtain the most accurate perception module, we would like a mechanism to improve the state estimate quality of the proprioceptive data collection policy as much as possible by adapting the policy's s behavior. To this end, we propose Active Sensing Motor Policies in which the policy =e trained with an additional estimation reward: Test = c exp (lle - él Figure illustrates the policy architecture.
In practice, we observe that an Active Sensing Motor Policy that is rewarded for estimating the ground friction coefficient slides one foot along the ground or swipes it vigorously to improve the friction coefficient observability in the state history.
2.2 Grounding Visual Features in Physics from Real-world Experience
We collect paired proprioceptive and vision data from the state estimation policy in the real world in order to learn about the relationship between visual appearance and terrain physics. Specifically, we collect data of the form (I, é, x), where I is a camera image. è are the estimated dynamics parameters and x the position and orientation of the robot in a fixed reference frame. We obtain x by training an additional 2D output of the final MLP layer in our learned state estimator to predict the displacement in the ground plane of the base from its location the previous timestep. Ax, and then integrate the estimated displacements. The integrated estimates will drift over time, but will only rely them algorithm to estimate the robot's state.
short time window. This alleviates the need for separate odometry
Using the camera intrinsic and extrinsic transform, we project the relative positions of the robot in the past and future into each camera image frame. We restrict the positions to those between m and 5 5m from the robot along the traversal path so that they are neither too far away to see nor so close to be obstructed from view by the robot's own body. We label each of the projected robot positions with the estimated dynamics parameters é that the robot felt when it walked there. This yields a corresponding label image I for each color image I where the traversed pixels are labeled with their measured dynamics.
For each color frame I, we use the pretrained convolutional backbone [20] to compute a dense feature map. Similar to the procedure that Oquab al. [21] used for depth estimation, discretize the labels é, into 20 bins and train a single linear layer with cross-entropy loss where the inputs the features of one patch and the outputs are the logits of the patch's é, label from proprioception.
3

---

2.3 Cost Function Learning: Connecting Physics Parameters to Affordances
The impact of terrain properties on robot per- formance is task-dependent: for example, robot dragging an object may face distinct con- straints that inhibit its traversal on some ter- rains, compared to robot without any payload. To use our vision module for planning, establish mapping between terrain properties and robot performance for each task. We pro- pose a simple procedure for extracting a task cost function from simulated data to demon- strate that our perception module can be useful in planning for multiple tasks, which we refer to as "operating modes". We sample simulated terrains with a variety of terrain properties and command a locomotion policy from prior work [19] to walk forward at m/s. We record the actual resulting velocity achieved on each terrain. We evaluate the mean realized velocity
Affordance Measurements
Operating Modes
Mean Velocity
must
Lecomotion Drugping Psyload
Friction Coefficient
Figure Locomotion affordances. We mea- sure the dependence of locomotion performance (1 m/s) on terrain friction in two different oper- ating modes. In free locomotion, the controller maintains the target velocity across range of fric- tion coefficients, except for the lowest friction. In contrast, when dragging a weighted box, the robot slows down the terrain friction increases.
for multiple operating modes: (I) locomotion, (2) payload dragging. We construct cost function for each operating mode as the average time spent traversing meter of given terrain. Mini- mizing this cost function during path planning will yield an estimated shortest-time path. While we focus on oime-optimal payload dragging as an example, (1, 2) could be any combination of task and metric as long as their relation to terrain properties can be evaluated in simulation.
2.4 Integrated Dynamics-Aware Path Planning from Vision
Our perception module (Section 2.2) runs in real-time (2 Hz) using onboard compute. Although was trained using images from a 360-degree camera, the resulting pixel-wise friction estimator can be evaluated in images from other cameras including the robot's onboard fisheye camera and an overhead drone. This is useful because the perception module can remain useful when deployed on viewpoint.
new robot evaluated from
One possible scenario for carrying ground objects across a long distance is that of a drone-quadruped team. these cases, we can directly evaluate our grounded vision module in overhead images to obtain a pixel-wise friction mask. Then, considering the robot's operating mode, we compute the cost associated with each pixel using the corresponding cost function determined from simulation (Section 2.3). Given this overhead cost map, we use the A* search algorithm [22] to compute the minimum cost traversal path for the current operating state.
2.5 System Setup
Robot: We use the Unitree Gol robot, 12-motor quadruped robot stands 40 em tall. It has an NVIDIA Jetson Xavier NX processor, which runs the control policy and the vision module. For payload dragging experiments, the robot's body is connected to an empty suitcase using a rope.
use a Insta360 X3 360 action camera mounted on robot to collect images for training the perception module. This camera provides 360 field of view. Before the image data used for training,
360 Camera:
the Insta360 app to perform image stabilization, which takes about two minutes for data collected from ten-minute run.
We perform policy training, video postprocessing, and vision model training on a desktop computer equipped with an NVIDIA RTX 2080 GPU.
Training Compute:
Drone Camera: For planning from overhead images, record terrain videos using DJI Mini 3, consumer camera drone.

---

1.0
Velocity Tracking Reward
Friction Estimation MSE
Preeéve Fatimation Estimation
Denstyy
Predictions
94
Ground Iruth
15
2.5
Density
0.6
0.5
1,0
1.5
2.0
2.5
3.0
0.4
1.5
Dendity
-veddccions
0.24
Step
Stop
0.0
1.5
0.5
2.0
1k:
24
34
4k
14
2k
3k
4k
Friciio Edtimationn Faror
Friction
(a) Performance and estimate quality during training.
(b) Distribution of friction estimates convergence.
Figure 4: Learning active estimation. Active Sensing Motor Policies (Active-SE) automatically learn motor skills (e.g. dragging the feet) that improve observability of the environment properties.
3 Results
3.1 Interaction among Estimation, Adaptation, and Performance
Obxerving supervised internal state estimates improves proprioceptive locomotion. Affirming the results of Ji al. [4], train state estimation network using supervised learning predict privi- leged information (the ground friction coefficient and terrain roughness parameter) from the history of sensory observations. When the policy is allowed to observe the output of this state estimation network (Passive-SE), the policy training is more stable and results in a more performant final policy than when the state estimate is not observed (No-SE) (Figure 4).
Observing passive state estimates can degrade the state observability. We analyze the error dis- tribution of the learned state estimator in Passive-SE and No-SE policies (Figure 4). It may be surprising that the friction estimation error of the more-performant Passive-SE policy is worse than that of the less-performant No-SE policy. We suggest mechanistic explanation for this be- havior: Supposing some irreducible sensor noise, two terrains of different frictions will only be distinguishable if they make the robot slip in sufficiently different ways. However, a control policy with better adaptive facility is more likely avoid slipping across wide range of ground frictions. Therefore, in the more adaptive policy, slip will occur less intensely, and a result, the observability of the ground friction coefficient will degrade.
Our method, ASMP, produces the best privileged state observability. We train an active sensing mo- tor policy (Active-Si SE) to intentionally measure the friction as described in Section 2.1. (The full reward function for each policy we trained is provided in the appendix.) We find that the Active-SE policy provides the most accurate friction estimates among the three architectures (Figure 4). There- fore, we will further show, it is the superior policy for supervising a task-agnostic physical ground- ing for vision.
3.2 Learning to See Friction
Evaluation in Simulated Environment. We collect five minutes of simulated data on four terrains:
ice, gravel, brick, and grass, assigning them arbitrary friction coefficients of = (0.25, 1.17, 2.08, 3.0) respectively. Figure 10 compares the resulting visual perception module learned from the policies performing passive vs. active estimation. Qualitatively, the vision module learned from passive data learns to see ice but fails to distinguish between higher-friction terrains (gravel, brick, and grass). This makes sense Figure 3 shows that frictions in this range have less influence on the performance of locomotion. In contrast, the vision module trained on data from our Active Sensing Motor Policy correctly learns to distinguish all four terrains. Quantitatively, ASMP results in lower dense prediction loss on images from a held-out test trajectory (Figure 10, Appendix).
Real-world Training. We collect 15 minutes of real-world traversal data spanning diverse terrains: grass, gravel, dirt, and pavement, Following the procedure in Section 2.2, we project the traversed
5

---

Measured ASMP (Ours) Passive (Baseline) Vision (Train) Vision (Test)
P (Friction Coefficient)
Grass
Pavement
Dirt
Gravel
Figure 5: Real-world friction sensing performance with proprioception and vision. Measured values are directly measured by dynamometer. The predictions from our proposed ASMP (Ours) agree more strongly with the dynamometer measurements than the baseline Passive (Baseline). Vision (Train) shows the generalization of visual prediction to un un-traversed patches in the train- ing images from the onboard camera; (Vision (Test)) shows the generalization to unseen patches and viewpoints by evaluating drone footage. We manual segmentation maps (Appendix Fig- ure 9) to match pixel predictions to terrains. Error bars indicate one standard deviation.
points into the corresponding camera images and train linear head top of convolutional back- bone trained for segmentation [20] to predict the terrain friction estimate for each traversed patch.
To evaluate estimation performance in the real world, we manually label image segments in a subset of train and test images containing grass, pavement, dirt, or gravel and compute the distribution of proprioceptive and visual friction predictions for each (Figure 5). To obtain a ground truth friction dynamometer to measure the weight of payload made of robot foot
value for comparison, material and its drag force across each terrain. The proprioceptive estimates from ASMP much closer to the dynamometer measurements than the estimates from the passive baseline. They do not match perfectly, suggesting small but measurable sim-to-real gap in the robot dynamics or terrain modeling. They agree with the dynamometer measurements on the ordering of terrains from most to least slippery. The grounded vision module is close to the distribution of proprioceptive estimates for both train and test images, with increased variance in test images.
3.3 Integrated Planning
Cost Function Evaluation. We define cost metric for the locomotion policy from [19] as the distance traveled per second when commanded with speed of 1.0 m/s. We evaluate this metric in simulation by averaging the performance of 50 agents simulated in parallel for 20s terrains of different friction coefficients ranging from a lower limit of 0.25 to upper limit of μ 3.0. This procedure is performed once with the robot in nominal locomotion and again with the robot dragging 1.0 kg payload. Figure shows the measured result; both tasks yield poor performance on extremely slippery terrain, but on higher terrains, the robot dragging a payload slows down while the free-moving robot adapts to maintain velocity. Knowledge of the ground's physical properties motivates difference in high-level navigation decisions between the two tasks.
Path Planning and Execution. We plan paths for locomotion and payload dragging and execute them via telcoperation to evaluate whether the predicted preferences hold true in the real world. We fly a drone over the same environment where the vision model was trained and choose a bird's med's-eye- view image that includes grass and pavement. We estimate the friction of each pixel and from this we compute the associated cost for locomotion and payload dragging. Then we use A search to compute optimal paths. The optimized paths and traversal result shown in Figure In agreement with the planning result, it is preferred to remain the sidewalk while dragging the payload and cut directly across the grass when in free locomotion.

---

Locomotion Dragging Payload
(b)
(a)
Operating Mode
Metric
Cross Grass
Stay Sidewalk
Dragging Payload
Time (s)
48 +
45
Locomotion
Time (s)
23 + 1
26 + 0
Figure 6: Path planning in overhead images. (a) We use the learned vision module to plan navi- gation in overhead images of terrain. (b) The vision module is only trained using first-person views from the robot but can infer the terrain friction with a different camera model and viewing pose. (c) We teleoperate the robot across both planned paths in both locomotion modes. The preference among paths in the real world matches the planning result from our pipeline.
Related Work
Self-supervised traversability estimation has been studied previously for the navigation of wheeled and legged robots. Some works have focused on the direct estimation of a traversability metric, scalar value quantifying the cost of traversing a particular terrain [16, 17, 23]. These approaches are specialized to the robot's traversal capability at the time of data collection, implying that a change the policy or task may necessitate repeated data collection train a new vision module.
Other works have demonstrated self-supervised terrain segmentation from proprioceptive data [15, 24, 25). Wu et al. [24] demonstrated that proprioceptive data from a C-shaped leg equipped with tactile sensors may be sufficient to classify different terrains. Wellhausen et al. [15] took supervi- sion from the dominant features a six-axis force-torque foot sensor during traversal and trained a model to densely predict "ground reaction score" from color images to be used for planning. Eysakowski et al. [25] also demonstrated that terrain classification from proprioceptive readings could be performed an unsupervised manner on a full-scale quadruped and showed that this infor- mation could be used as additional signal to improve localization. Our work differs from these in that (1) we do not use any dedicated sensor in the foot but predict the terrain properties using only standard sensors of the robot's 's ego-motion, and (2) thanks to our Active Sensing Motor Policies, we can directly predict the terrain properties instead of a proxy score, which allows us to compute the cost function in simulation for multiple scenarios as in Section 2.3.
Another possibility is directly predicting which locomotion skill to execute from visual information[18. 26]. Loquercio et al. [26] learned to predict the future latent state of the policy [2] from a front-facing camera image to improve low-level control performance in stair climb- ing. An advantage of their approach is that it does not require the choice of an explicit terrain parameterization, but this comes at the cost that its visual representation is specialized to the la- tent of a single motor policy, so cannot reused for new policies or operating states, and pre- dicting the next latent is only meaningful for egocentrie images, so it cannot be used for novel viewpoints, as in drone-quadruped teaming or planning from satellite imagery. Yang al. [18] trained semantic visual perception module for legged quadrupeds using human demonstrations.
7

---

The resulting system imitated an operator's 's re- sponse to different terrains, controlling velocity and gait. This relies on a human operator to pre-
Eatimation
Priction
Roughness
Torque
Mode
Loss
Los
Penalty
Fassive
1.00
1.00
-0.34
Priction
0.47
1.06
-0.87
dict the terrain properties during the demonstra- tion. Other work has learned general navigation through supervised learning diverse robotic platforms, including legged robots [27, 28, 29]. These works train an omni-policy for all robots and environments, enabling interesting zero- shot generalization but not explicitly adjust- ing for embodiment, operating state, or camera viewpoint variation.
Roughneee
0.99
0.72
-0.84
Joint
0.49
0.80
-1.18
Fr.+Ro.
Figure 7: ASMP for multiple physical param- eters. Friction and roughness estimates are im- proved by ASMP, even when both parameters jointly targeted. We estimation loss for pas- sive estimation (None (Passive)), active esti- mation of each parameter separately (Friction, Roughness), and active estimation for both pa- rameters in a single policy (Joint tr.+Ro.) Variation in torque reflects that change in mo- tor strategy enabled the improved estimation.
Several works on wheeled robots visually es- timate the geometry or contact properties of the terrain through self-supervision or hand- designed criteria and then compute the traversal
cost from these metrics [30, 31, 32, 33, 34, 35, 36, 37, 38]. Compared to legged robots, wheeled robots have limited variety of traversal strategies. Consequently, the question of selecting a lo- comotion controller gather the most informative self-supervision data has not been directly ad- dressed. Active perception, in which robot agent acts to increase environmental observability, suggests a solution. This approach has been applied to vision systems [39, 40, 41], and more re- cently has been extended to include physical interaction [42, 43, 44, 45]. This inspired our approach to the controller selection issue in labeling vision with proprioception for legged robots.
Discussion and Limitations
Our work assumes a mapping between the estimated terrain properties and the robot's performance. Friction affects the slip of the robot's feet against the ground and the drag force of payloads and other objects, so is an interesting source of performance variation for practical locomotion tasks. To account for other parameters besides friction that vary in the environment, our framework can potentially be extended to include them. For example, Figure 7 shows that ASMP successfully enhances the estimation of a terrain roughness parameter in addition to friction.
In general, ASMP may applied under two conditions: (1) a history proprioceptive readings sufficient to infer the parameter of interest, and (2) the parameter of interest can be effectively simulated. If these conditions are not met, a different technique besides ASMP may be necessary to collect training data. Additionally, to train our vision module, we assume terrains with different properties are visually different. If some parameters do not have an impact on the terrain's 's visual appearance, it may be impossible to learn a vision module of the form we propose for those parame- ters. If the mapping varies quickly over time, future work could explore representing uncertainty or performing fast online adaptation the estimates to the current environment based on new proprio- ceptive information. Finally, our pipeline does not yet account for terrain geometry and occlusions. In the future, ASMP may also be useful to complement data from other sensors like LiDAR to achieve the most detailed perceptual representation that includes geometric information.
6 Conclusion
Proprioceptive self-supervision is promising data source for robots to learn about the relationship between vision and physics. In this work, exposed that the quality of proprioceptive supervision can be strongly influenced by the style of the motor policy acquired through reinforcement learning. We proposed a novel technique, Active Sensing Motor Policies, and show that it improves the pro- prioceptive estimation quality and the corresponding performance of a grounded vision module that reusable for new sensor configurations and physical tasks.
8

---

Acknowledgments
We thank the members of the Improbable AI lab for the helpful discussions and feedback on the paper. We grateful to MIT Supercloud and the Lincoln Laboratory Supercomputing Center for providing HPC resources. This research was supported by the DARPA Machine Common Sense Program, the MIT-IBM Watson Al Lab, and the National Science Foundation under Cooperative Agreement PHY-2019786 (The NSF Al Institute for Artificial Intelligence and Fundamental Inter- actions, http://iaifi.orr/) We acknowledge support from ONR MURI under grant number N00014- 1- 22-1-2740. This research was also sponsored by the United States Air Force Research Laboratory and the United States Air Force Artificial Intelligence Accelerator and was accomplished under Cooperative Agreement Number FA8750- 10000-19----000 1000. The views and conclusions contained in this document are those of the authors and should not be interpreted representing the official policies, either expressed implied, of the United States Air Force or the U.S. Government. The U.S. Gov- ernment is authorized to reproduce and distribute reprints for Government purposes, notwithstanding any copyright notation herein.
Author Contributions
Gabriel Margolis ideated, implemented, and evaluated Active Sensing Motor Policies and shared ideation and implementation of the vision module and overall experimental design.
Xiang Fu shared ideation and implementation of vision module and overall experimental design.
Yandong Ji contributed ideas and supported infrastructure development during the project.
Pulkit Agrawal advised the project and contributed its conceptual development, experimental design, positioning, and writing.
References
[1] J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning quadrupedal locomo- tion over challenging terrain. Science robotics, 5(47):eabc5986, 2020.
[2] A. Kumar, Fu, Pathak, and J. Malik. RMA: Rapid motor adaptation for legged robots. Robotics: Science and Systems, 2021.
[3] G. B. Margolis, G. Yang, K. Paigwar, T. Chen, and P. Agrawal. Rapid locomotion via rein- forcement learning. Robotics: Science and Systems, 2022.
[4] G. Ji, J. Mun, H. Kim, and J. Hwangbo. Concurrent training of control policy and state estimator for dynamic and robust legged locomotion. IEEE Robotics and Automation Letters, 7(2):4630 4637, 2022.
[5] T. Miki, J. Lee, Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning robust per- ceptive locomotion for quadrupedal robots in the wild. Science Robotics, 7(62))eabk2822, 2022.
[6] Y. Ji, G. B. Margolis, and P. Agrawal. Dribblebot: Dynamic legged manipulation in the wild. arXiv preprint arXiv:2304.01159 2023.
[7] A. Agarwal, A. Kumar, J. Malik, and D. Pathak. Legged locomotion in challenging terrains using egocentric vision. In Conference on Robot Learning, pages 403-415. PMLR, 2023.
[8] Choi, G. Ji, Park, H. Kim, J. Mun, H. Lee, and J. Hwangbo. Learning quadrupedal locomotion on deformable terrain. Science Robotics, 8(74)eade2256, 2023.
[9] D. Hoeller, L. Wellhausen, Farshidian, and M. Hutter. Learning a state representation and navigation in cluttered and dynamic environments. IEEE Robotics and Automation Letters, (3):5081-5088, 2021.

---

[10] G. B. Margolis, T. Chen, K. Paigwar, X. Fu, D. Kim, S. Kim, and P. Agrawal. Learning to jump from pixels. Conference on Robot Learning, 2021.
[11] R. Yang, M. Zhang, N. Hansen, H. Xu, and X. Wang. Learning vision-guided quadrupedal lo- comotion end-to-end with cross modsl transformers. arXiv preprint arXiv:2107.03996 2021.
[12] I. M. A. Nahrendra, B. Yu, and H. Myung. Dreamwaq: Learning robust quadrupedal lo- comotion with implicit terrain imagination via deep reinforcement learning. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 5078-5088 IEEE, 2023.
[13] S. Kareer, N. Yokoyama, D. Batra, S. Ha, and J. Truong. Vinl: Visual navigation and loco- motion over obstacles. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 2018-2024. IEEE, 2023.
[14] Truong, A. Zitkovich, S. Chernova, D. Batra, T. Zhang, J. Tan, and W. Yu. Indoorsim-to- outdoorreal: Learning to navigate outdoors without any outdoor experience. arXiv preprint arXiv:2305.01098, 2023.
[15] L. Wellhausen, A. Dosovitskiy, R. Ranftl, K. Walas, C. Cadena, and M. Hutter. Where should i walk? predicting terrain properties from images via self-supervised learning. IEEE Robotics and Automation Letters, 4(2): 15011-59611111. 2019.
[16] M. G. Castro, S. Triest, W. Wang. J. M. Gregory, F. Sanchez, J. G. Rogers III, and S. Scheren. How does it feel? self supervised costmap learning for off-road vehicle traversability. arXiv preprint arXiv:2209. 10788, 2022.
[17] J. Frey, M. Mattamala, N. Chebrolu, C. Cadena, M. Fallon, and M. Hutter. Fast traversability estimation for wild visual navigation. Robotics: Science and Systems, 2023.
[18] Y. Yang, X. Meng, W. Yu, T. Zhang, J. Tan, and B. Boots. Learning semantics-awe aware locomo- tion skills from human demonstration. In Conference on Robot Learning, pages 2205-2214. PMLR, 2023.
[19] G. B. Margolis and P. Agrawal. Walk these ways: Tuning robot control for generalization with multiplicity of behavior. In Conference on Robot Learning, pages 22-31. PMLR, 2023.
[20] Q. Yu, J. He, X. Deng, X. Shen, and L.-C. Chen. Convolutions die hard: Open-vocabulary segmentation with single frozen convolutional clip. arXiv preprint arXiv:2308. 024887 2023.
[21] M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniee, V. Khalidov, Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07/99. 2023.
[22] P. E. Hart, N. J. Nilsson, and B. Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE transactions on Systems Science and Cybernetics, 4(2):100-107, 1968.
[23] 7. Fu, A. Kumar, A. Agarwal, H. Qi, J. Malik, and D. Pathak. Coupling vision and propriocep- tion for navigation of legged robots. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 17273-17283, 2022.
[24] X. A. Wu, T. M. Huh, R. Mukherjee, and M. Cutkosky. Integrated ground reaction force sensing and terrain classification for small legged robots. IEEE Robotics and Automation Letters,
2016.
[25] M. Lysakowski, M. R. Nowicki, R. Buchanan, M. Camurri, M. Fallon, and K. Walas. Un- supervised learning of terrain representations for haptic monte carlo localization. In 2022 International Conference on Robotics and Automation (ICRA), pages 4642-4648. IEEE, 2022.
10

---

[26] A. Loquercio, A. Kumar, and J. Malik. Learning visual locomotion with cross-modal supervi- sion. arXiv preprint arXiv:221 1. .03785, 2022.
[27] S. Levine and D. Shah. Learning robotic navigation from experience: principles, methods and recent results. Philosophical Transactions of the Royal Society B, 378(1166))22214447, 2023.
[28] D. Shah, A. Sridhar, A. Bhorkar, N. Hirose, and S. Levine Gnm: A general navigation model to drive any robot. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 7226-7233. IEEE, 2023.
[29] D. Shah, A. Sridhar, N. Dashora, Stachowicz, K. Black, N. Hirose, and Levine. Vint: A foundation model for visual navigation. arXiv preprint arXiv:2306. 2023.
[30] R. Hadsell, P. Sermanet, Ben, A. Erkan, Han, B. Flepp. U. Muller, and LeCun. Online learning for offroad robots: Using spatial label propagation learn long-range traversability. In Proc. of Robotics: Science and Systems (RSS), volume 11, page 32. Citeseer, 2007.
[31] R. Hadsell, P. Sermanet, J. Ben, A. Erkan, M. Scoffier, K. Kavukcuoglu, U. Muller, and Y. Le- Cun. Learning long-range vision for autonomous off-road driving. Journal of Field Robotics, 26(2):120-144, 2009.
[32] D. Stavens and S. Thrun. A self-supervised terrain roughness estimator for off-road au- tonomous driving. arXiv preprint
6872, 2012.
[33] S. Palazzo, D. C. Guastella, L. Cantelli, P. Spadaro, F. Rundo, G. Muscato, D. Giordano, and C. Spampinato. Domain adaptation for outdoor robot traversability estimation from rgb data with safety-preserving loss. In 2020 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 10014-10021. IEEE, 2020.
[34] H. Lee and W. Chung. A self-training approach-based traversability analysis for mobile robots in urban environments. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 3389-3394. IEEE, 2021.
[35] X. Xiao, J. Biswas, and Stone. Learning inverse kinodynamics for accurate high-speed off- road navigation unstructured terrain. IEEE Robotics and Automation Letters, 6(3):6054- 6060, 2021.
[36] A. Shaban, X. Meng, J. Lee, B. Boots, and D. Fox. Semantic terrain classification for oof- autonomous driving. In Conference on Robot Learning, pages 619 629. PMLR, 2022.
[37] A. J. Sathyamoorthy, K. Weerakoon, T. Guan, J. Liang, and D. Manocha. Terrapn: Unstruc- tured terrain navigation using online self-supervised learning. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7197-7204. IEEE, 2022.
[38] X. Meng, N. Hatch, A. Lambert, A. Li, N. Wagener, M. Schmittle, J. Lee, W. Yuan, Z. Chen, Deng, et al. Terrainnet: Visual modeling of complex terrain for high-speed, off-roat navi- gation, arXiv preprint arXiv:2303. 15771, 2023.
[39] R. Bajesy. Active perception. Proceedings of the IEEE, 7668):666-1055, 1988.
[40] D. Jayaraman and K. Grauman. Look-ahead before you leap: end-to-end active recognition by forecasting the effect of motion. Computer Vision-ECCVV 2016: 14th European Conference, Amsterdam, The Netherlands, October 1-14, 2016, Proceedings, Part V 14, pages 489-505. Springer, 2016.
[41] S. K. Ramakrishnan. D. Jayaraman, and K. Grauman. Emergence of exploratory look-around behaviors through active observation completion. Science Robotics, cc30):caaw6326, 2019.

---

[42] J. Bohg, K. Hausman, B. Sankaran, O. Brock, D. Kragic, S. Schaal, and G. S. Sukhatme. Inter- active perception: Leveraging action in perception and perception in action. IEEE Transactions on Robotics, 33(6):1273-1291, 2017.
[43] H. Van Hoof, O. Kroemer, H. B. Amor, and J. Peters. Maximally informative interaction learning for scene exploration. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5152-5158. IEEE, 2012.
[44] V. Chu, I. McMahon, L. Riano, C. G. McDonald, Q. He, J. M. Perez-Tejada, M. Arrigo, T. Darrell, and K. J. Kuchenbecker. Robotic learning of haptic adjectives through physical interaction. Robotics and Autonomous Systems, 63:279-292, 2015.
[45] D. Pathak, P. Mahmoudich, G. Luo, P. Agrawal, Chen, Y. Shentu, E. Shelhamer, J. Malik, A. A. Efros, and T. Darrell. Zero-shot visual imitation. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 2050-2053, 2018.
12


Reviews:
[{"review_id": "xITU3VpKrxd", "reviewer": "Reviewer_dDBh", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The work presents a learning-based system to plan a navigation path for a legged robot in multi-terrain outdoor environments. The core idea is to learn a cost-map of the environment for planning the most efficient route to take. To obtain the cost-map, a three-stage approach is devised: 1) an active sensing motor policy is trained to optimize the system identification error for friction so that the policy takes actions to best estimate the friction of the environment, 2) the active sensing motor policy is then deployed in the real-world to collect a set of image data with sparse friction labels, which is then used to train a model to predict per-pixel friction coefficient, 3) the target locomotion controller is tested in simulation with different friction coefficients to estimate the 'affordance' of the controller at different frictions, which is then used to map the dense friction prediction into an affordance/cost map for the path planning. \n", "summary_of_recommendation": "The paper presents a solid pipeline with interesting ideas and results. However it's missing some key analysis and comparisons. With these resolved I believe it can be a good contribution to the venue.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "1) It would be helpful to add some discussions regarding whether a good mapping between friction and affordance exists.\n2) Add some analysis/discussion on scalability to higher-dimensional physical parameters identification.\n3) Add some comparisons to demonstrate the value of going through friction as an intermediate step vs directly learning the affordance. i.e. use the final policy to collect the same amount of data on the same route used in the current experiment for learning friction, but this time estimate the realized velocity and construct a model to predict per-pixel cost directly from the data. This would also enable the method to handle cases where friction alone cannot predict affordance well.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The idea of learning a policy to actively explore the environment for better system identification is interesting and the presented policy indeed learns interesting behavior. The entire pipeline to obtain the cost-map seems solid and results show intuitively meaningful behavior on real hardware.\n\nOne weakness of the work is that it currently heavily focus on friction coefficient and assumes that a good mapping between friction and affordance exists. However, this may not be true for other terrains such as stairs and it's not clear how well the method can scale up to higher dimensional physical parameters.\n\nIn addition, I think one important baseline is missing in the current evaluation, where one directly learns the affordance in real-world, using the same strategy for learning the visually grounded friction model in this work (similar to [1]).\n\n[1]. Where Should I Walk? Predicting Terrain Properties from Images via Self-Supervised Learning.", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}, {"review_id": "qRSBz4CjNDW", "reviewer": "Reviewer_dYv6", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper presented a method to train a vision model with physical grounding (image -> friction coefficient of surface) and an application of such a vision model in task specific legged robot navigation path planning.\n\nThe work is done in stages: 1) train an active sensing legged robot locomotion policy with auxiliary task of state estimation and environment physical parameter (friction coefficient) estimation; 2) roll out policy in real world and collect images and physical parameter estimation output as training data to train a physically grounded vision model that estimate friction of coefficient from images input; 3) apply the grounded vision model for navigation planning.\n\nMain contribution of the papers are\n* Pipeline for training grounded vision models from self-supervision data and apply output to task planning.\n* Showed that physical property can be translate to task specific affordance.\n* Reward design for co-trained active sensing policy.", "summary_of_recommendation": "The paper has clear novel points and the procedure and results are well documented and easy to follow/understand. There are two points that can be addressed to strengthen the claim of the author.\n\nI recommend accepting the paper if the author can address the two questions raised in the rebuttal questions.\n\n", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "1) the analysis of generalization to OOD data (bird-eye view) is not clear: only one example is provided and no intermediate results(such as the vision model output) are shown.\n\n2) the performance of the grounded vision model in terms of estimation accuracy is not discussed. This is important because the ASMP is trained in sim environment and the grounded vision model uses the data generated from it.\n", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper extends the work of co-trained state-estimator to environment physical properties and generalizes the application of estimated value to task planning. The work also demonstrated that the physical property estimation model can be used for task planning of multiple different tasks. Both active sensing policy training and the observation of physical properties generalize better are novel points proposed in the paper.\n\nThe paper is well written and easy to follow with a good amount of description and figures to help illustrate the ideas and procedure.\n\nThough the support of the proposed points can be strengthened (see question for rebuttal section).\n", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "hLoJdPhaol", "reviewer": "Reviewer_zcuB", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes learning terrain properties via active sensing motor policies (ASMP). ASMP uses a reward term that encourages the policy to produce actions that improve the accuracy of state estimation. In this work, only terrain friction is considered for state estimation. Given the estimated friction parameters, a robot trajectory is projected on to the observations (RGB images), thereby associating each image patch with the corresponding predicted friction parameter. Then, a linear layer is trained to predict the friction parameter from the image patch. The policy is trained in simulation and results show that it produces more accurate friction estimation than the baselines.  For planning, the predicted friction is converted to cost and an off-the-shelf planner computes the optimal path. Finally, the paper conducted a real experiment where a robot would prefer sidewalk over grass if it carries a payload.", "summary_of_recommendation": "The paper is well-written and the idea is interesting and makes intuitive sense. I have some questions regarding the advantages of this approach compared to end-to-end policy optimization , its general applicability for other state estimation tasks, and the sim2real gap. I think this is critical to understand the potential of this approach.\n\n**Post rebuttal**\nThe authors have done a great job of conducting additional experiments and discussing the pros and cons of the proposed approach. I think the findings are valuable to the community. I recommend accepting the paper.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "- How much sim2real gap is there? Could you elaborate what may hinder the sim2real transfer? It will be more convincing if the paper conducts an experiment that compares the friction parameter estimation in the real world.\n- Why friction? Since there will still be error in the friction estimation and there are other factors , the corresponding costmap may not be accurate. Would it be possible to estimate the terrain cost with RL?\n- What are the tradeoffs between passive and active sensing? While active sensing could be more accurate, it may also lead to unexpected behaviors or even accidents. Passive sensing is safer since it is more controllable. \n- It is unclear if the active sensing approach has significant advantages in the real world since the paper only focuses on friction estimation. A more end-to-end approach may potentially work better because it considers more environmental factors and optimizes the final locomotion policy. Could you elaborate on this?\n- How is the learning-based odometry trained? Is it better than the default odometry?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths:\n\n- Using RL to improve the estimation of terrain property is interesting and makes intuitive sense. The supplementary video shows an interesting behavior learned by the policy.\n- The paper is well-written and easy to follow. \n- The figures are illustrative and informative.\n- The limitation section is informative.\n\nWeakness:\n\n- It is unclear how effective the proposed method is for estimating other terrain properties. \n- It is unclear which one is better: optimizing the terrain properties or optimizing the final policy directly. This seems to boil down to the reward design. The advantage of directly optimizing the final policy is that it can be trained with real-world data, and it considers all environmental factors. On the other hand, the proposed approach has to be trained in simulation, and it is unclear how big the sim2real gap is. It would be more enlightening if the paper elaborates on this and includes comparisons with some recent baselines (e.g., [1, 2, 3, 4, 5]).\n", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "RtMYKO9IUa", "reviewer": "Reviewer_knX2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The work \"Learning Physically Grounded Robot Vision with Active Sensing Motor Policies\" proposes an approach to train quadrupedal locomotion policies optimized for environment parameter estimation. They demonstrate this by training a controller which optimizes for foot friction coefficient estimation and learns to drag its foot to improve the observability of the friction. The friction estimates are then used to train a network that directly predicts the friction coefficient from images.", "summary_of_recommendation": "The direction of the research and the methodology are very interesting and relevant. The presented ASMP method can be used for accurate parameter estimation and the authors demonstrate this with friction coefficient estimation. Furthermore, they hint towards the generalizability by showing (simulated) results for terrain roughness estimation. Overall the method and the experiments are convincing. \nThe application of the method, path planning when dragging a payload, is less convincing though. Nevertheless this reviewer recommends accepting the manuscript.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "- Why is a separate policy better than having a locomotion policy that continuously estimates the friction values (explain the statement in line 29)?\n- Using published friction values (or measuring them accurately), how close is the estimate to the ground-truth value in the different real-world scenarios?\n- Is there any measure of how consistent and homogenous the estimated friction coefficients are in areas of similar materials?  This is especially interesting in the context of planning as any (false) variations in the friction coefficient might lead to unnecessarily complex paths.\n- How is friction related to traversal time?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The rebuttal period has very much strengthened the paper and the authors have adequately responded to the open questions.\n\nSTRENGTHS:\nThe task of environmental parameter estimation is generally a very important one to work on, and by explicitly estimating the friction, the approach presented here is more versatile than, for example, RMA, which has an implicit representation in a latent space and does not optimize for observability. The approach shown here is somewhat similar to drivers trying out the traction on snow or ice at a slow speed to calibrate their perception of the road and adjust accordingly. Therefore, the reviewer believes that the task is highly relevant to robotics and the approach presented is potentially impactful. \n\nThe paper clearly demonstrates that the presented ASMP approach enhances the estimation of the ground friction coefficient. The real-world experiments comparing the friction estimates from an active and a passive policy with the measured ground-truth value prove the effectiveness of ASMP to improve the observability of the given ground parameter. Additionally, the good agreement between ground-truth friction coefficient (obtained through a dynamometer) and the ASMP estimate highlight the accuracy of the overall method. The real-world experiments also highlight that the vision system trained on the ASMP observations is outputting coherent friction values based on the camera images.\n\nThe authors hint at the generalizabilty of the method to other parameters beyond friction estimation and show simulation results of terrain roughness estimation. This is a key point as ASMP can, in theory, be applied to almost any mobile robot that needs to estimate a certain parameter of its environment. The reviewer therefore believes that the method potentially has significant impact.\n\nWEAKNESSES:\nThe experimental validation of the friction coefficient estimation is convincing, however the demonstrated application in pulling a crate across various terrains is not. First, as pointed out by other reviewers, the object that is being pulled by the robot also interacts also with the ground and hence is subject to some friction as well. In the results shown, only the robot foot friction is estimated - and there is no neccessity that the object being pulled by the robot behaves similar. Also, unless the robot is severly underpowered for the task, a change in friction should not affect the pulling performance too much as less friction for the legs also likely (but not neccessarily!) corresponds to less friction for the pulled obstacle. \n\nFor very similar reasons, it is unclear (perhaps even undefined) how friction of the robot foot and traversal time for a given course are related. This reviewer believes that the parameter estimation performance of the method is its key feature, and less the application to path planning shown in the paper.", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces active sensing motor policies (ASMP) for learning terrain properties, focusing on terrain friction estimation using actions that enhance state accuracy. After estimating friction from RGB images, a linear layer predicts friction from the image patch. Simulations show ASMP has better friction estimation compared to baselines. For planning, friction estimates are used to calculate costs, which is then used by a standard planner to determine the optimal path. In real-world tests, robots carrying payloads favored sidewalks over grass.

The reviewers unanimously agreed on the quality of this work, including sufficient real-world evaluations. There are some remaining concerns, e.g. reviewers are not sure if the foot friction prediction is helpful for optimizing crate-pulling behavior. Additionally, the authors are encouraged to take in remaining editorial feedback and improve the paper for the camera-ready version.
Decision: Accept (Poster)