Learning Robotic Locomotion Affordances and Photorealistic Simulators from Human-Captured Data

Authors: alejandro  escontrela
justin  kerr
kyle  stachowicz
pieter  abbeel
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-07 12:46:25

Abstract: Learning reliable affordance models which satisfy human preferences is often hindered by a lack of high-quality training data. Similarly, learning visuomotor policies in simulation can be challenging due to the high cost of photo-realistic rendering. We present PAWS: a comprehensive robot learning framework that uses a novel portable data capture rig and processing pipeline to collect long-horizon trajectories that include camera poses, foot poses, terrain meshes, and 3D radiance fields. We also contribute PAWS-Data: an extensive dataset gathered with PAWS containing over 10 hours of indoor and outdoor trajectories spanning a variety of scenes. With PAWS-Data we leverage radiance fields' photo-realistic rendering to generate tens of thousands of viewpoint-augmented images, then produce pixel affordance labels by identifying semantically similar regions to those traversed by the user. On this data we finetune a navigation affordance model from a pretrained backbone, and perform detailed ablations. Additionally, We open source PAWS-Sim, a high-speed photo-realistic simulator which integrates PAWS-Data with IsaacSim, enabling research for visuomotor policy learning. We evaluate the utility of the affordance model on a quadrupedal robot, which plans through affordances to follow pathways and sidewalks, and avoid human collisions. Project resources are available on the [website](https://pawslocomotion.com).

Main paper: 
 introduction: 
 Ensuring that robots navigate in the wild while satisfying human preferences is crucial to ensure safe deployment. Without considering navigation affordances, our robots may perilously ignore crosswalks, stray off of sidewalks, trespass on forbidden ground, or damage themselves on hazards. However, current algorithms for learning affordances and training visuomotor policies often suffer from a lack of high-quality training data. Additionally, defining effective affordance labels from human data is challenging, as the naive approach of assigning high affordances solely to traversed areas leads to sparse labels with many false negatives. Further, simulators for training visuomotor policies tend to provide low-fidelity renders.In this work we propose PAWS, a method for instrumenting a person with cameras to capture ego view of natural human locomotion in the wild for affordance model training. PAWS can record long-horizon trajectories of indoor or outdoor scenes and process them to obtain camera poses, foot positions, radiance field reconstructions, and terrain meshes. Using this method we contribute PAWS-Data, a dataset of over ten hours of the aforementioned data spanning diverse scenes including hikes, cityscapes, and indoor environments. We also integrate PAWS-Data with IsaacSim (Mittal et al., 2023) to develop a fast and realistic simulator which can generate thousands of photo-realistic images per second across thousands of environments.Neural Radiance Fields have exploded in popularity as a photo-realistic reconstruction method (Mildenhall et al., 2020) , however they are prohibitively expensive to train and render at scale. Recently, Gaussian Splatting (Kerbl et al., 2023) significantly reduces the cost of rendering by eliminating the need for expensive ray casting operations, opening the door for real-time photo-realistic reconstruction in robot data pipelines. We take advantage of this fact to reconstruct photo-realistically renderable environments from human locomotion data.We utilize the PAWS-Data radiance fields to train a Locomotion Affordance Model (PAWS-LAM) on a large augmented dataset by rendering from novel views while retaining visual fidelity. Affordance labels are generated using DINO features to label parts of the scene which are semantically similar to areas traversed by the user. In our experiments, we demonstrate that viewpoint augmentation significantly enhances the prediction accuracy for trained models, while semantic-aware affordance labels accurately capture human preferences for navigation. We also find that a foundation model backbone enables generalization to different environments beyond the training data. We apply PAWS-LAM to a quadruped navigation task, and combine PAWS-LAM with a simple planner to produce robot trajectories which are safe and efficient. 
approach: 
 In this section we discuss the three contributions that make up our method: PAWS: a data collection system for capturing and processing a large quantity of trajectories, PAWS-LAM: A robotic locomotion affordance model that leverages learned radiance fields and semantic features to produce high 2 quality affordance predictions, and PAWS-Sim: which leverages the PAWS-Data dataset and a massively parallel simulators to achieve fast photo-realistic rendering across thousands of environments.PAWS uses three cameras mounted on the user's torso to capture three stereo camera streams. Two cameras are mounted on the shoulder and angled such that they capture a wide field of view while attending to the terrain ahead. A third camera is pointed directly down towards the user's feet, and is used for two functions: to track the fiducial markers placed on the user's shoes, and to provide a clear top down view of the terrain for later use when training 3D Gaussian radiance fields. Figure 2 demonstrates an overview of the camera setup and our data processing steps. We use DROID-SLAM (Teed and Deng, 2021) to provide an estimate of the camera poses and to generate a colored point cloud of the environment. Importantly, we find that initializing DROID-SLAM disparities with stereo depth values significantly improves the quality of estimated poses. We also compute human segmentation masks with Mask2Former (Cheng et al., 2022) to mask out humans for later processing steps. Given the estimated camera poses, we can estimate foot poses in the global frame using the relative poses provided by the fiducial markers (Olson, 2011) . Re-projecting depth from the known camera poses allows us to generate an accurate mesh of the environment, capturing details such as stairs and rough terrains. Finally, the RGB, Depth, camera poses, and point cloud are used to train a 3D Gaussian splatting model of the scene using Splatfacto (Ye et al., 2023; Tancik et al., 2023) . Figure 3 visualizes the data products produced by PAWS. PAWS-Data consists of 10 hours of diverse indoor and outdoor trajectories, a subset of which are visualized in Figure 6 .Dataset Augmentation Given a dataset consisting of radiance fields and camera poses, we significantly expand the dataset size by rendering from novel viewpoints, a process we refer to as Viewpoint Augmentation. Namely, we sample a batch of new camera poses by perturbing the positions and orientations of the original camera poses by up to 3 meters and 30 degrees, respectively. We then rasterize images from the sampled camera poses using the learned 3D Gaussian radiance fields, which allow for fast batch rendering. Figure 1 visualizes a radiance field along with the original camera poses, augmented poses, and affordance labels defined in the next section.Affordance Label Generation Given a dataset of camera images, camera poses, and foot positions, a naive approach to generate labels would be to simply project the foot position into the camera image. While these labels accurately capture where the human navigated, they are sparse and falsely label possibly navigable areas as having low affordance. Instead, we propose to label regions which are Gaussian Splatting Render semantically similar to those traversed by the human. We do so by computing DINOv2 features (Caron et al., 2021) for the M patches the human stepped on in the scene. We then compute a M × N × N cosine similarity matrix, comparing all M traversed ground features to the N × N features in the input image. We take the minimum along the footstep feature dimension and threshold by a c DINO to get affordance patches, and upsample with bi-linear sampling to obtain affordance labels. Figure 5 compares the naive dilated sparse affordance labels to the affordance labels produced by PAWS. Model Architecture Given the affordance dataset from the previous section, we fine-tune a foundation model backbone on the task of affordance prediction. Specifically, we attach a simple token projection head to a pre-trained DINOv2 (Caron et al., 2021) model and train the model to minimize the cross entropy loss against the affordance labels. In Section 3.1 we evaluate different choices of backbone and model size.PAWS-Data provides camera poses, foot poses, meshes, and radiance fields that accurately match the scale of objects in the real world. Additionally, 3D Gaussian radiance fields allow for fast rendering, allowing for thousands of renders per seconds. We integrate these features into IsaacSim (Mittal et al., 2023) to develop an open source photo-realistic gym environment for running robots which can be adapted to any desired reward signal or user-captured scene. Namely, we create a convex decomposition of the 3D terrain mesh to allow for stable physics simulation. In env.step, we query the 3D Gaussians for each scene with a batch containing each robot's camera pose, camera intrinsics, and image resolution to obtain thousands of renders per second. While 3D Gaussian splats provide high-fidelity renders for a large portion of poses within the scene, deviating significantly from the original camera poses can lead to highly distorted and unrealistic renders. To combat this, we initialize the robots near the original camera poses, and terminate the episode once the camera pose has deviated significantly from the set of original camera poses, while still bootstrapping values. 
experiments: 
 In this section we evaluate PAWS-LAM and assess it's utility in a practical robotics setting. Specifically, we aim to answer the following research questions:Figure 6 : Example scenes in PAWS-Data. Aside from radiance fields, the PAWS dataset also contains terrain meshes, foot positions, and affordance labels. Artifacts in aerial renders can be attributed to the extreme viewing angle.• How does PAWS-LAM compare to alternative approaches to affordance learning?• How well does PAWS-LAM generalize to unseen environments?• Does PAWS-LAM learn affordances that facilitate safe quadruped navigation?We ablate different design decisions, including training with viewpoint augmentation and different backbone sizes. In Table Figure 7b , we ablate PAWS-LAM trained with and without viewpoint augmentation. We find that disabling viewpoint augmentation drastically reduces performance on the validation set, as measured by pixel accuracy and mean intersection over union (mIoU). We attribute this to the decreased dataset size, as well as a smaller set of distinct camera poses, which both contribute to overfitting. Figure 7a visually demonstrates the failure modes of training without viewpoint augmentation. Specifically, models trained without viewpoint augmentation fail to capture the nuanced human preferences encountered in complex scenarios, such as predicting the entire road as navigable while only sidewalks or crosswalks should be traversed. Additionally, increasing the size of the pre-trained backbone from 21M parameters (ViT-S/14) to 86M parameters (ViT-B/14) significantly enhances model performance.Additionally, in Table Figure 7c we evaluate PAWS-LAM's ability to generalize to environments unseen during training. We evaluate a model trained solely on outdoor data on a held-out dataset of indoor trajectories and find that PAWS-LAM generalizes favorably, displaying a slight decrease in performance. Further, models trained without viewpoint augmentation generalize poorly to the indoor dataset, which we hypothesize is due to the model overfitting to a small set of DINOV2 features which do not generalize to indoor environments. We now propose one potential downstream application of our data collection interface and splatbased rendering pipeline: learning affordance models that transfer across embodiments, from human data to robot locomotion. We propose a simple pipeline using our affordance model to optimize a robot's high-level plan to ensure that it is physically reasonable and matches human affordances such as sticking to marked paths or avoiding other humans.We use a Unitree GO1 quadruped as our target platform, equipped with two depth cameras to capture a wide field of view of the scene. For each camera, we compute visual affordances using PAWS-LAM, then project the predicted affordance probabilities directly onto point clouds from the depth cameras. This map is then flattened and used as a 2-dimensional cost map, through which any planner can be used to to generate a high-level plan. For simplicity, we use a grid-based Dijkstra's algorithm, but the generally proposed method is applicable to any sufficiently flexible planning mechanism. Figure 8a visualizes an example plan through the affordance map.As shown in Figure 8 , we find that the resulting system is able to match human affordances in an intuitive manner, sticking to paths whenever possible and following forks in the road. Importantly, the robot's camera perspective is not contained in the initial dataset. Suggesting that the viewpoint augmentation used for PAWS-LAM helps to obtain high downstream performance on a non-humanoid embodiment. Please see our supplementary materials for a reference implementation of the planning algorithm and videos of our robot following human-interpretable affordances. 4 Work 
large scale photo-realistic reconstruction: 
 Scene reconstruction has long been studied, classically in context of SLAM SfM reconstruction (Newcombe et al., 2011; Schönberger and Frahm, 2016; Mur-Artal et al., 2015) . More recently, advances in neural rendering have drastically improved the capabilities of these systems for rendering visual outputs. Neural Radiance Fields (NeRF) (Mildenhall et al., 2020) can render highly photorealistic from novel views, and an explosion of extensions expand on the speed (Müller et al., 2022) , quality (Barron et al., 2021; 2022; 2023) , scene scale (Tancik et al., 2022) , and tooling (Tancik et al., 2023) . Recently, 3D Gaussian Splatting (3DGS) (Kerbl et al., 2023) proposes a method of representing a scene with a collection of oriented 3D Gaussians, which can be rasterized onto an image quickly with modern GPU hardware. This work uses 3DGS to reconstruct the surroundings of human capture trajectories for fast rendering at volume.For mobile robots to efficiently navigate the world while respecting both the laws of physics and social rules, it is important that they have some notion of affordances describing which states they can enter and which actions they can perform. Traditional legged locomotion policy learning architectures learn these affordances in simulation with domain randomization (Feng et al., 2023; Lee et al., 2020; Margolis et al., 2023) relying on either blind models (Kumar et al., 2021) or depth information (Agarwal et al., 2023) . However, many more subtle cues cannot in fact be recovered by depth information alone: a concrete road and a sand pit might appear identical when viewed as a depth image (Loquercio et al., 2023) , and they have difficulty resolving transparent or translucent objects. Furthermore, using RGB cameras instead of depth can allow socially-compliant behavior rather than purely physical behavior: walking on paths and in crosswalks rather than through a grass field, for example.Terrestrial navigation research has also studied the problem of traversability estimation from visual data, particularly in the off-road driving case (Jung et al., 2024) . In contrast to these works, we do not require hand-tuned heuristics nor are our learned affordances limited to only physical parameters:PAWS can learn rich, interpretable human affordances from features in in both dense urban environments and outdoor settings, and can be used to provide a dense traversability score rather than just a 0-1 traversability map if desired.Developing high-fidelity simulators is a large and active area of research, spanning applications in autonomous driving (Amini et al., 2021; Dosovitskiy et al., 2017) , UAV research (Shah et al., 2017; Guerra et al., 2019) , and robotics (Mittal et al., 2023; Gan et al., 2020) . While these simulators can achieve high-quality renders, they often require high compute budgets which decreases sample throughput. Additionally, designing simulation can be time consuming, and even the highest quality simulators may not fully overcome the simulation to reality (Sim2Real) gap. These challenges have spurred researchers to find new solutions to the challenges of overcoming the Sim2Real gap.  
discussion: 
 We present PAWS, a self-supervised data collection method for obtaining large datasets of scenes from human captured data. Accompanying PAWS is PAWS-Data, a dataset of over 10 hours of indoor and outdoor trajectories gathered with PAWS, containing RGB + Depth, camera poses, foot poses, terrain meshes, and radiance fields. We also introduces PAWS-LAM, which leverages PAWS-Data to train a locomotion affordance model capable of modeling the nuanced human preferences of outdoor navigation. We ablate PAWS-LAM and show that it benefits from our novel affordance dataset augmentation approach, our proposed affordance label generation method which drastically reduces false negatives, and the use of a foundation model backbone. Additionally, PAWS-LAM generalizes well to unseen environments, which we showcase by evaluating on indoor scenes using a model trained solely on outdoor data. We show how PAWS-LAM can be used for robotic navigation by using the affordance model to optimize a high-level plan to ensure the robot's trajectory satisfies human navigation preferences. PAWS-LAM generalizes to the robot's distribution of observed images, showcasing favorable generalization across embodiments. Finally, we introduce PAWS-Sim, a fast and photo-realistic simulator which integrates PAWS-Data IsaacSim to enable fast visuomotor policy learning.Limitations & Future Work While PAWS can complete trajectories for the camera pose, the foot goes out of frame for roughly half the user's gait, thereby limiting foot position estimation for that duration. Additionally, fast motions can produce motion blur, making it difficult to detect the fiducial marker. Approaches such as SelfPose (Tomè et al., 2020) estimate body positions from a headmounted camera and a pose detection network. Such approaches would potentially increase foot position availability and decrease hardware complexity. Further, integrating terrain dynamics estimation with our method would increase the physical fidelity of the simulator, further reducing the Sim2Real gap. 
 
 Appendix:  


Decision: The focus of this paper is on the development of a data collection and simulation pipeline for legged locomotion.

-- The strength of this paper is on the data collection pipeline. Generally the reviewers were favorable about that. One reviewer thought the data collection rig was too complex.

-- There was a question about whether the human footstep affordances collected in the data would generalize well to different robot morphologies. 

-- There was also a question about how well DINO v2 features would adequately capture footfall affordances and transfer them to novel scenes (reviewer ZQSK). Perhaps this is something that can be validated experimentally.

Reviews: review: title: title: Review
paper_summary: paper_summary: This paper presents a novel pipeline for locomotion data collection
main_review: main_review: The paper presents a full pipeline of a novel data collection device, a data processing pipeline, and a model for learning locomotion affordances. The data collection uses vision tools to reconstruct a 3D radiance field of the environment and human feet movements, therefore allowing the training of navigatable area predictors and robot navigation policies in simulation. The authors also released a new dataset of collected data.

This paper makes a clear contribution to the field of real-world navigation tasks. I can imagine the collected data and the simulator integration will be useful for various tasks.

Regarding the newly proposed affordance learning algorithm, it seems that the key insight here is the label generation process based on DINO features. However, I think more justification is needed here. Although this approach can reduce the amount of false negatives in labels, it also suffers from introducing additional false positives, because such semantic-similarity-based labels may fail to capture "the nuanced human preferences encountered in complex scenarios", such as the example of sidewalks mentioned by the authors in L129.

I agree that this might be a good enough strategy for generating low-cost labels with decent quality, but more visualizations and consensus analysis with human "groundtruth" labels should be included, to demonstrate the effectiveness/trade-offs of this approach.
questions: questions: I do not have specific questions. I encourage the authors to provide additional analysis of the semantic similarity-based labeling procedure.
limitations: limitations: 3
review_summary: review_summary: This paper is making solid contributions, but its scientific contribution is rather low, and it could benefit from more analysis of the data.
title: title: A pipeline for collecting real-world data, affordances, and usage in affordance prediction and downstream tasks
paper_summary: paper_summary: This paper proposes a pipeline for collecting human-captured real-world data and affordances. Such data can be used for affordance prediction and building a photo-realistic simulator. The learned affordances can be used for guiding a navigation policy.
main_review: main_review: The pipeline proposed in this paper is a complete procedure for collecting rich sensory data of natural scenes. Along with the data collection pipeline, the paper also proposes a photo-realistic simulator and affordance prediction models.

The experiments validate the effectiveness of having 3D scene reconstructions over 2D affordances, as well as the effectiveness of a larger backbone architecture. The on-robot study shows that learned affordance can be used in navigation tasks. 

Overall, the paper is clearly written and a solid contribution to the CoRL community. The completeness of the proposed pipeline is much appreciated. The key design choice of having 3D reconstruction is validated in experiments. 

The on-robot validation may bear some room for improvement: for example, in more diverse scenarios/scenes.

Another related work to this paper: Margolis, Gabriel B., et al. "Learning to see physical properties with active sensing motor policies." arXiv preprint arXiv:2311.01405 (2023).
questions: questions: The photo-realistic simulator is proposed but it is unclear how useful it might be for downstream tasks such as policy learning. It would be great (but not necessary, in this reviewer's opinion) to have experiments to validate if the simulator could be used for a downstream task.
limitations: limitations: 3
review_summary: review_summary: Overall a good paper -- experimental validation could be further improved.
title: title: Good data collection ideas, but limited technical contributions
paper_summary: paper_summary: The paper presents PAWS, a comprehensive robot learning framework designed to enhance robot navigation by leveraging human-captured data. The framework encompasses three main contributions: (1) PAWS Data Collection System: A portable data capture rig and processing pipeline for collecting long-horizon trajectories; (2) PAWS-Sim: Using Gaussian Splatting to construct a high-speed, photo-realistic simulation in IsaacSim using the PAWS-Data; (3) PAWS-LAM: A robot locomotion affordance model, which is fine-tuned on a dataset enriched through novel viewpoint augmentation with the DINOv2 model and viewpoint projection.
main_review: main_review: Strengths:
- Innovative Data Collection: The use of human-captured data to train robotic models ensures that the locomotion policies are aligned with human navigation preferences.
- Viewpoint Augmentation: This method significantly expands the dataset size and diversity, enhancing the robustness and generalization of the affordance model.
- High-Fidelity Simulation: PAWS-Sim provides a realistic simulation environment, crucial for developing and testing visuomotor policies efficiently.
- Semantic Affordance Labeling: Improves the accuracy of affordance models by focusing on semantic similarities, leading to better generalization and fewer false negatives.
- Comprehensive Dataset: PAWS-Data includes diverse environments, offering a rich resource for training and evaluation.

Weaknesses:
- Hardware Complexity: The data capture setup, which includes multiple cameras and fiducial markers, seems cumbersome and may be susceptible to motion blur during rapid movements.
- Simulation Constraints: The simulation environment may not adequately reflect the physical dynamics of real-world terrains, potentially limiting the applicability of the findings to real-world scenarios.
- Limited Technical Contributions: The paper predominantly utilizes existing methods, such as Gaussian Splatting, IsaacSIM, and DINOv2 to train an affordance model. There appears to be a lack of novel technical contributions.

While the ideas presented in the paper are interesting, there are multiple gaps that have not been addressed well --- the difference between the dynamics of a human and a robot are not considered. The most critical point is how can affordances be converted to a specific score associated with a particular robot/planning algorithm (what is traversable for a robot may not be traversable for a human and vice versa). The individual components used in the method are not novel --- projection of affordance labels using DINO features has been done in previous works, collecting data from people's footsteps and associating it with quadruped foothold traversibility has been researched before. The proposed method's main contribution seems to lie in combining such methods together. It would be helpful if the authors could clarify the contributions more clearly. based on the above, the potential impact of the work is also limited. 

For limitations, the authors have mentioned the reliance of fiducials and the lack of terrain dynamics. But there is a lack of limitations discussion regarding the method itself (such as how the affordance is used for planning). Previous works address affordance as a continuous score, while this work treats it as a binary score. Further, the method assumes that the robots that would make use of it would have similar affordances as the human -- i.e. ANYmal, Go1. it would be helpful to either address this as a limitation of the method, or specify it as the initial scope within the problem setup. Also, it would be suitable to mention the limitation of creating an elaborate rig for this data collection process as compared to the alternative of simply using a phone or readily available sensors that do not require a sophisticated hardware setup. 

In terms of comparisons, it would have been beneficial to compare also with other affordance models, such as "Fast Traversibility Estimation for Wild Visual Navigation" by Frey et al.
questions: questions: 1. Please provide a detailed explanation of the process used to convert the "affordance" (the output of the model) into a score for planning.
2. Could you elaborate on how motion blur affects the quality of reconstruction? Did you explicitly select keyframes to mitigate this issue?
3. The assumption underlying the data collection method requires further clarification, especially considering the differences in dynamics between humans and legged robots.
limitations: limitations: 2
review_summary: review_summary: I recommend Weak Reject because of the lack of technical novelty in the method, however I am willing to change my recommendation either way depending on the author responses and reviewer comments