Velociraptor: Leveraging Visual Foundation Models for Label-Free, Risk-Aware Off-Road Navigation
Authors: Samuel Triest
Matthew Sivaprakasam
Shubhra Aich
David Fan
Wenshan Wang
Sebastian Scherer
Venue: CoRL 2024 (2024)
Keywords: Field Robotics
Self-Supervised Learning
Visual Foundation Models
TL;DR: We propose a fully self-supervised traversability learning method that leverages visual foundation model features and demonstrate it in high-speed off-road navigation.

Abstract: Traversability analysis in off-road regimes is a challenging task that requires understanding of multi-modal inputs such as camera and LiDAR. These measurements are often sparse, noisy, and difficult to interpret, particularly in the off-road setting. Existing systems are very engineering-intensive, often requiring hand-tuning of traversability rules and manual annotation of semantic labels. Furthermore, existing methods for analyzing traversability risk and uncertainty are computationally expensive or not well-calibrated. We propose Velociraptor, a traversability analysis system that performs [veloci]ty-informed, [r]isk-[a]ware [p]erception and [t]raversability for [o]ff-[r]oad driving without any human annotations. We achieve this via the use of visual foundation models (VFMs) and geometric mapping to produce a rich visual-geometric representation of the robot's local environment. We then leverage this representation to produce costmaps, speedmaps, and uncertainty maps using state-of-the-art fully self-supervised techniques. Our approach enables intelligent high-speed off-road navigation with zero human annotation, and with about forty minutes of expert data, outperforms several geometric and semantic traversability baselines, both in offline and real-world robot trials across multiple challenging off-road sites.

Velociraptor: Leveraging Visual Foundation Models for Label-Free, Risk-Aware Off-Road Navigation
Samuel Triest,, Matthew Sivaprakasam Shubhra Aich', David D. Fan",
Wenshan Wang'. Sehastian Scherer Carnegie Mellon University Robotics Institute, FField Al
striest@andrew. cmu.edu
Abstract: Traversability analysis in unstructured environments is a challenging task that requires understanding of multi-modal inputs such as camera and LiDAR. Measurements from these sensors are often sparse, noisy, and difficult to interpret, particularly in the off-road setting. Existing traversability analysis systems are very aagiinerrin-innensiive often requiring hand-tuning of rules and manual anno- tation of semantic labels. Furthermore, existing methods for analyzing traversabil- ity risk and uncertainty are computationally expensive or well-calibrated. We propose Velociraptor, traversability analysis system that performs [veloci]ty- informed, [r]isk-[a] ware [p]erception and [t]raversability for [o]ff-[r]oad driving without any human annotations. We achieve this via the use of visual foundation models (VFMs) and geometric mapping to produce rich visual-geometric repre- sentation of the robot's local environment. We then leverage this representation to produce costmaps, speedmaps, and uncertainty maps using state-of-the-an fully self-supervised techniques. Our approach enables intelligent high-speed off-road navigation with zero human annotation, and with about forty minutes of expert data, outperforms several geometric and semantic traversability baselines, both in offline and real-world robot trials across multiple challenging off-road sites.
Keywords: Field Robotics, Supervide Learning, Visual Foundation Models
Introduction
High Speed
Autonomous off-road driving is a challenging problem in the domain of mobile robotics with applications in many domains such as forestry, defense, exploration and construction [1, 3, 4, 5, 6, 7, 8). Unlike urban driving and in- door scenarios, off-road terrain is highly un- structured, variable and uncertain. In order to field a high-performing off-road system, it is necessary to consider both geometric and vi- sual information [9]. Unfortunately, the process of transforming this information into represen- tations amenable for planning and control (i.e. traversability analysis) is highly complex.
Low
LIDAR
V/RM
Unn..
Figure 1: We propose Velociraptor, an off-road traversability system that enables high-speed nav- igation without requiring any human annotations.
In recent years, there has been much research effort in replacing hand-engineered traversability rules with deep learning [10, 11, 12, 9, 13, 15, 16, 17]. Learning traversability using deep neural networks promising in that networks can represent complex functions that can directly consume high-dimensional sensory data. However, current approaches generally rely large aand-annotated datasets [18, 19, 20, 9]. The annotation
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

process is often laborious, error- prone and brittle to changes in platform or environment, making it challenging and expensive to deploy autonomous mobile robots at scale. Additionally, noisy mea- surements, novel obstacles and sparse data at high speeds creates the need to capture and quantify uncertainty in order to accurately aggregate information and make risk-informed planning decisions [21, 22, 23, 24]. Especially for self-supervised systems, is important to be able to both recognize input regions with high epistemic aleatoric uncertainty and to react accordingly.
We present a method for learning uncertainty- aware traversability from visual and geometric features without semantic annotations or hand-designed traversability rules. This allows us to rapidly deploy our system, requiring only forty minutes of expert driving data and minimal in-field tuning. This is accomplished via the use of geometric analysis and VFMs to produce a rich geooe-gge map representation in bird's eye view (BEV) space for downstream traversability learning. From this, train BEV neural network produce costmap and speedmap via self-supervised techniques. Given these maps, model-predictive controller [25] is then used to direct the vehicle to a goal point safely, and at high speed. In our experiments, we find that our system outperforms common geo- metric and semantic baselines, and that the shared visual-geometric representation is critical to the performance of the sulf-supe traversability learning. Relative to our closest point of compari- son I. our work makes the following contributions: 1. a simple but effective method of combining VFM features and geometric analysis BEV for traversability learning, 2. self-supervised learning of costs and speeds, allowing for maximum autonomous speeds of around 8m/s and 3. improved uncertainty estimation, allowing for avoidance of out-of -utrof-distribution (OOD) obstacles.
2 Related Work
The dominant approach for off-road driving leverages semantic segmentation of image and/or Li- DAR data to inform a hand-designed traversability function. Work by Maturana et al. [20] uses point cloud data to project semantic labels from first-person view (FPV) into BEV space. The re- sulting semantic map then transformed into a costmap, and is then used by trajectory library planner for off-road navigation. Additional work by Shaban et al. [12] bypasses the projection step by performing semantic segmentation directly on point cloud data. They leverage this representation for local traversability and planning of a skid-steered robot. This work is extended by Meng et al. [9] to consume image data and output both semantic and elevation information. These methods rely on a designd traversability function to produce costmaps from their semantic representations.
To address the laborious process of hand-annotating semanties, there has been significant interest in self-supervision for traversability. We refer the reader to recent survey papers [7, 26, 27] for more complete analysis of learning for traversability. Most approaches fall into one of three categories:
Predicting Future Proprioception: Methods in this family rely on the fact that there is a correla- tion between what the robot sees now, and what will experience in the future. This is exploited by regressing some proprioceptive signal such as slip [13], bumpiness [28, 16, 15] or future trajec- tory [14, 29] to the perception representation (e.g. images, local maps). This enables the robot to avoid undesirable proprioceptive events (such as high slip or bumpiness) by predicting them from exteroception. Work by Kahn et al. [28] trains a model to predict bumpiness from first-person im- ages and a sequence of actions. This model is trained from random exploration data and enables preferences for smoother terrain such as sidewalks. Work by Wellhausen et al. [24] leverages a ground reaction force to enable legged robots to avoid challenging terrain from vision. Further work [16, 15] performs this prediction directly in BEV and demonstrate effectiveness on wheeled and legged platforms.
Imitation of Experts: Methods in this family typically rely on inverse reinforcement learning (IRL) [30, 31] [1]st to learn a cost function under which a set of expert data is made optimal. Pioneering work by Ratliff et al. and Bagnell et al. [32, 33] presents an application of IRL for global navigation which costmaps are learned from satellite imagery and applied to a large-scale mobile robot. Wulfmeier al. [34, 35] demonstrate that the gradient of the MaxEnt IRI. objective be used train deep
2

---

Feature Extraction
Feature Aggregation
Learning
Agoragate
IRL
WH
MV Registened Visual Map (BEV)
Stack Channels
Carreaa (SPM)
Visual Map (BEV) eey
Costrap
WIN
Vieual Feeturee
Point Cloud (30)
(BEV)
Map Features (hot Impuc
Rasare Speedraap
Odomatyy
Extraction
uncersincy
30 aggrogato
Registered Cloud (30)
Univeraaityy Nap
A high-level overview of our perception pipeline. At every timestep, the robot receives camera image, point cloud and state estimate. The image is first through deep visual feature extractor to produce image-space features. These features are used to decorate the point cloud and BEV-splatted produce single-frame visual map. These maps are aggregated over time using odometry. Concurrently, the pointclouds themselves are registered using odometry and have BEV geometric features extracted. Finally, the geometric and visual maps are stacked together and fed through a FCN to generate our cosmaps/peedmapp/ueerranng
Figure 2:
neural networks, and demonstrate its efficacy in semi-structured urban driving. Additional work applies MaxEnt IRI. to local navigation for wheeled and legged robots [36, 38, 11].
Learning Model Uncertainty: Additional work leverages model uncertainty as a proxy for traversability. Work by Fan al. [39] (STEP) presents a LiDAR-based traversability pipeline that reasons about distributions cost. Uncertainty propagated through hand-engineered traversabil- ity rules, and is transformed into scalar cost using Conditional Value-at-Risk (CVaR) [40]. Work by Schmid et al. [41] leverages the uncertainty of a learned perception model metric of traversabil- ity. Additional work by Cai et al. [42] learns a perception-conditioned traction distribution. This is leveraged by model-predictive controller to enable risk-aware planning via dynamics rollouts that sample from this learned distribution. This work extended [21] to include estimation of epistemic uncertainty and demonstrations on both wheeled and legged platforms.
Leveraging VFMs: There has been recent interest in leveraging VFMs for off-road traversability. Existing work V-Strong [29] leverages SAM [43] for contrastive learning of traversability in im- age space. Existing work WVN [13] leverages Dino [44] to regress proprioceptive slip estimate onto FPV embeddings. These estimates are then projected onto an elevation map for BEV-space traversability and demonstrated on a legged robot. We differ from prior work in that we first map VFM features into BEV, and then perform learning. This simplifies the process of learning multi- ple outputs, enables simple fusion with geometric LiDAR information and reduces the effect of the lossy FPV to BEV projection step. Furthermore, draw from all three families of self- self-supervised traversability by 1. learning predict future expert speeds from exteroceptive map, 2. learn- ing to predict cost from expert demonstrations via IRI., and 3. performing epistemic and aleatoric uncertainty estimation on our perception representation.
3 Visual-Geometric Mapping with VFMs
is necessary to produce perception representation for downstream control that enables a robot to
navigate desired goal locations safely and quickly, without the use of any prior information (e.g.
road maps, satellite imagery). For this work, will use local grid map this representation [45], which we will populate with information derived from image and point cloud observations. At

---

high level, our method produces local maps for learning, given streams of state estimates 2;, point clouds P, and images I,- First, we maintain an aggregated point cloud P P1z via registration from our state estimates Fis- Then we extract BEV-space geometric features (e.g. elevation, slope, etc.) from this registered point cloud to produce a geometric map M In parallel, we compute deep visual features from the current image I using VFMs. The current point cloud Pr is projected into the image frame assign spatial positions to the deep image features. These features then projected into BEV space and aggregated over time to produce visual map Mý Finally, these two maps stacked to yield a visual-geometric map M,. This process is shown in Figure 2. A summary of each component of the mapping system are presented below, with additional details in the supplemental.
3.1 Obtaining Geometric Features
In order to obtain geometric features, we register the pointclouds Pi# together using odometry 214- Then given a local map size and resolution, we perform a series of geometric analyses to extract several salient features (such as terrain height, slope, etc.). Note that this geometric analysis lever- ages a registered point cloud. This is preferable to computing single-frame maps and aggregating, as the computation of several geometric features is influenced by effects such as occlusion, which aggregated point clouds can mitigate somewhat.
3.2 Obtaining Visual Features
Visual foundation models such as Dínov2 [46] and SAM [43] represent an exciting opportunity for the off-road driving domain in that they provide continuous- valued, semantic-level features for a wide range of image data without requiring fine-tuning or annotation. However, it remains an open question as to how best to incorporate such information into navigation stack. We provide baseline approach for incorporating visual features based on LiDAR point cloud projection that has been used for semantics in prior work [20]. Additional details are provided in the supplemental.
In order to improve memory efficiency and speed of mapping, we perform a principal component analysis (PCA) on the FPV VFM images in the train set and map the top components. This is desirable because it maintains the most relevant features from the VFM, while significantly reduc- ing the number of feature channels and being efficient to compute. We then project the current LiDAR point cloud P, into the image frame to assign spatial positions to the reduced VFM features. Features with a lidar return are then mapped into BEV to produce a visual map (cells containing multiple points use the average embedding). which is aggregated over time with odometry.
An important point to note is that unlike the geometric mapping, the aggregation of visual features occurs in map-space (as opposed to projecting the aggregated point cloud into the current image). This is done to avoid the effect of occlusions and odometry error, which can occur when points are projected into images at different timesteps. Additionally, this process qualitatively reduces the effects of positional embedding artifacts on the aggregated map.
Learning in BEV from Unlabeled Data
We can leverage our BEV representation perform several label-free traversability tasks. In order to provide the most useful guidance to downstream controller, we elect to produce three maps, speedmap Sg, costmap Cg and uncertainty map U (Figure 2). The costmap informs the vehicle of where to drive by producing a single scalar value commensurate to the difficulty of traversal over that particular cell (i.e. trails have low cost, trees have high cost). The speedmap informs the vehicle of a notion of how drive in that for every cell by producing a recommendation how fast a cell can be safely traversed over (i.e. trails have high speeds, and tall vegetation has low speeds). The uncertainty map identifies unseen terrain features which should be avoided. In order to learn these maps, a lightweight fully-convolutional network (FCN) with shared backbone and separate

---

heads for costmap and speedmap prediction. Following prior work [32, 11], we train an ensemble of these networks in order to perform uncertainty estimation.
4.1 Learning Costmaps
Unlike geometric and semantic features, there is no obvious heuristic mapping from VFM features to cost. Thus, order to learn costmaps, we leverage inverse reinforcement learning (IRL). IRL is desirable in this application due to demonstrated success off-road traversability [33, 32, 35, 37, 11] and ability to consume arbitrary state features, which need be human-interpprn innnaaninterret.ble..
At high level, IRL seeks to learn cost function under which corpus of expert demonstrations is made optimal. We optimize an objective similar to LEARCH [32], in which positive samples are generated from the expert trajectory, and negative samples are generated from a fast trajectory optimizer (in our case, MPPI [47]). State visitation distributions D and D (discretized over grid states in the map) are computed for the expert and learner, and used via Equation 1 to update the network. We follow the example set by Ratliff al. [32] and train our network to output log-costs, which we exponentiate for the planner. We also small L.2 regularizer costs.
Σ [DE 5,ES
[[VE-
aC -D,
Vo
Σ TeEDg
((TE|0)=
+ Σ Co(M)..: =xM(foo)((foM) (1)
00
8:ES
4.2 Learning Speedmaps
In order to learn speedmaps, perform supervised learning of the expert speeds, conditioned our perception representation. We follow the example of Cai et al. [21] by learning categorical distribution over discrete speed bins, per cell. This has the advantage of correctly representing the support of the speed variable (R+), while maintaining the ability to represent arbitrary distribution shapes. Learning a distribution allows us leverage quantiles of this learned speed distribution as a risk tolerance parameter. This distribution is learned by maximizing the pag--obbbbilty of the expert speed v(x) for its corresponding map cell s(x). For safety purposes, we also introduce negative samples for cells where the expert did not drive and set their speed label to zero, resulting in the objective in Equation (where Sj denotes the current learned speed distribution map state s).
Sspeed
Σ
Σ RRITT
-logS (10==)))++
Σ #.ES
-logS :: (0)
TgEDE
4.3 Learning Uncertainty Maps
We also perform density estimation of our training input distribution to detect regions high epis- temic uncertainty. We observe that the IRL network is effective at detecting distinct geometric ob- stacles and evaluating the traversability of different types of terrain, and our offline results indicate generalizability to held-out test sites. However, we noticed that various out-of ditt---distribttion objects that don't resemble terrain didn't t trigger high-cost predictions from the network as expected. This undesirable, as in practice we found some of these objects to be dangerous (e.g. a stray tire or a pallet full of nails). While this may not be the case every time, within the scope of this work we adopt a policy of avoiding these objects if the robot can find low-cost terrain around it.
Offline, we randomly sample image embeddings from the train dataset, and use K-Means clustering to generate feature clusters (using cosine distance). Online, we leverage the residuals between the features and cluster centers and perform the following operation to determine uncertainty U.
min ||d Fall1, U 50 ke(1.2...,.K)
if TU otherwise
(3)
5

---

This operation is done in the map space after the visual features have already been projected out. the minimum distance of a map feature d to any cluster F& exceeds a threshold Tv, we assume its corresponding terrain/object uncertain and therefore assign it high cost, allowing us to avoid anomalies without any further training.
4.4 Risk-Awareness
Since we are able to produce distributions of both cost and speed, we are able to adapt the risk- tolerance of our learned maps. Note that both our speed and cost distributions can be non-Gaussian and multi-modal. In order to produce a risk-aware cost, we follow prior work [39, 11, 48) and use Conditional Value-at-Risk (CVaR) [40]. In order to produce a risk-aware speed limit, we simply take the y-th quantile of the learned speed distribution per cell (Equation 4).
(M) (VaR, E)). SM(M) = VaR, (M)
(4)
5
Experiments and Analysis
5.1 Offline Tests
We first evaluate our method on several held-out datasets. The first dataset is a representative sample of expert driving from the test site where the training data was collected (Dataset 1). The second dataset (Dataset 2) collected from a different test site (approx. 70 miles away) and contains higher speeds, more slopes and denser vegetation than Dataset 1.
In order to evaluate our learned control representation, we compare the optimal trajectory under MPPI to the expert trajectory (Equation 5, cost function J in supplemental). We use Modified Hausdorff distance on positions (pMHHD) speeds (sMHD), and both positions and speeds (MHD). This metric is standard prior work for costmap prediction [36, 37, 11, 9J. We report positional and speed distances separately somewhat (though not completely) separate out the contribution of the costmap and speedmap. We compare our method to several strong baselines in recent literature.
L MHD(TE,
J Cv.
(5)
Baseline Geometric Analysis from ALTER [49]: We chose the cost function from ALTER as geometry-only baseline. At a high level, ALTER weights baseline occupancy map with SVD decompositons of the points in each cell to encourage the vehicle to drive on smooth terrain when possible. Since ALTER does not produce speedmap, linear quantile regression was performed on the train set to predict expert speeds from this SVD feature. We use two variants of the ALTER cost function, one which uses the coefficients directly from Chen et al. [49] (ALTER orig), and one with modified coefficients that yielded better on-platform navigation performance (ALTER tuned).
Baseline 2 Semantics from GANav [50]: We chose GANav as representative semantic baseline. The FPV semantic predictions from GANav were then passed through our visual mapping pipeline in order to generate BEV semantic maps. Note that we follow the approach of Asgharivasi et al. [51] and map the semantic logits (which we then softmax in the aggregated map). These maps were then transformed into costs via a hand-designed reward function (which was validated and tuned give good navigation performance on platform). Speeds were computed by computing the class-conditioned speed quantiles on the train set.
Baseline 3: Simple Vismel-Geometric Fusion (ALTER + GANav): We also evaluate a simple fusion of geometric and visual costs. This achieved by combining the previous baselines via a cell-wise maximum for costs, and minimum for speeds.
In addition to the above baselines, evaluate four variants of Velociraptor. This is done ablate the usefulness of various visual features for the downstream navigation task. V-geom is trained

---

Table 1: Results for cost CVaR = O, speed quantile = 0.5
MIDDI
EMILD (2)
sMI
MUDD
ALTER ong (49)
1.62 1
0.31
1.98
2.90
0.51
3:37
ALTTER ofn roedl [49]
1.33 1
0.25 1
1.65 =
3.36
0.52 1
3.70 +
GA-Nav ofn (50)
3.70
0.43
4.01 =
4.24
0.76 +
4.70 +
ALTER orig GAA-Nv
2.56
0.37
3.89 =
0.76
4.37
V V-gemm [II]
1.26 0.18
0.23 0.04
1
2.34 0.27
0.37 1: 0.03
2.65
0.28
Vosemantics
1.16
0.20 0.001
1.35 0.08
2.26 1:00::0.10
0.46 0.004
2.63
0.21
v-Dinn
V-SAM
1.13 0.10
0.20 0.01
1.41 0.10
2.12 0.24
0.34 0.03
2
Table 2: Results for cost CVaR 0, speed quantile 0.9
(MHD(1)
(M
MHD
pMHD
<MHD
ALTER cia onig [49]
1.13
0.20
1.41
1.57
0.17
1.78 1:
ALTER cin tuned [19]
1.06
0.17 1
1.32
1.99
0.16
2.16 1
[50]
ALTER orig GA Nav
2.68
0.39
2.97
2.96
0.27
3.15
ALTER tuned + GA-Nav
2.27
0.31
2.55 +
2.02 -
0.20
2.23 1
V-geom [11]
1.05 0.13
0.18
1.31 = 0.13
1.64 0.12
0.16:
1.82 0.12
Vsemantics
0.07
V.Dino
0.83
0.03
0.14
0.00
1.08
0.02
1.48 1 0.02
0.15 0.001
1.68
1: 0.03
V-SAM
1.01 + 0.13
0.18 + 0.02
1.25
+ 0.13
1.39 + 0.13
0.15 +
1.77 + 0.13
with only geometric features, which is equivalent to Triest et al. [11] with the addition of the extra head for speed. V-semantics, V-Dino and V-SAM are trained with both geometric features and the features from GANav [50], DINOv2 [46] and SAM [43], respectively. All other training parameters for these models are held constant.
Quantitative results are provided in Tables I and 2. Table the speed risk level (quantile of speed distribution) is set at the mean value (0.5) for all methods. For Table 2, v. is set to 0.9. We report the mean and standard deviation over five trials for learning-based methods.
We find that is important to learn the cost and speedmaps as opposed to relying on existing heuris- ties, as evidenced by significant improvement based on the metrics. As expected, the addition of visual features allows for improved trajectory matching, with the features from Dinov2 appearing to be the most useful, based on superior performance across all metrics and datasets. As one might pect, considering distributions of cost and speed are important, especially given that speed is treated as an upper bound. This is evidenced by improvement across all methods when 0.9.
5.2 Large-Scale, High-Speed Tests
V-Dino
We evaluated our method and several baselines on a 4km loop in an off- road testing site. This loop contains many challenging traversability narios such as steep slopes, tall grass, standing water, and small, OOD ob- stacles. We measure performance via the number of safety operator cor- rections, average autonomous speed and speed-normalize ride bumpi- ness [16] (Corrections (long), Avg. Speed and Avg. Bump in Table 3). For safety, experiments were run with more conservative risk
Figure 3: Leveraging fine-grained visual features allows for more sophisticated costing of different types of grass and vegetation. We can observe that our method can distin- guish between low shrubs (a), marshy grass (b), and low, traversable grass (c) where geometric and semantic base- lines cannot. Planned trajectories are shown in yellow. Costmaps are cropped to focus on the relevant region.
parameters than the offline test (v. = 0,v, 0.25 for learned, and 0.5 for baselines).
Overall, we find that our learned costmap and speedmap enabled the robot to complete the course with fewer operator interventions at higher speeds. In particular, visual features were important for allowing the vehicle travel more quickly on open trails. A combination of geometry and continuous-valud visual features allowed for disambiguation of cost within semantic class (Figure 3). We found that all methods achieved similar amount of ride bumpiness. This is likely due to

---

Table 3: Hardware Results
Method
Corrections (long)
Avg. Speed
Avg. Bump
Corrections (short)
Alter cfn tuned [49]
10
4.64m/s
0.47
5
GA-Nav [50]
15
3.93m/s
0.48
V-Dino
6
5.06m/s
0.47
5
V-Dino + Unc
10
4.59m/s
0.48
Figure 4: Example scenarios with pnanetrically-ndiittinn objects (tarps, tire, puddle) obstructing the path in red (robot pose is yellow arrow). The uncertainty layer classifies them as OOD objects.
the resolution of the costmap, and the lack of explicit bumpiness minimization in our cost function.
Several additional scenarios are presented in the supplemental.
FPV
Cost
Speed
Unc.
5.3 Avoidance of OOD Objects
Alter
To isolate the effect of the OOD de- tection, we designed smaller course where the robot had to navigate goal directly straight ahead, with var- ious OOD obstacles (two tires, flat- tened sign, and a tarp) placed in its path. We evaluate the number of cor- rections for each method over three trials, the results of which are shown in the Corrections (short) column of Table (note that the short and long trials are separate). We find that the uncertainty layer adds additional cost to these obstacles without adding cost to previously seen terrain like trails and tall grass (Figures 4, 5).
V-Dino
GANav
V-Dino
Figure Examples from our large-scale navigation trial. Relevant obstacles are circled in red for FPV and BEV. In the top scenario, the robot must navigate around a short pile of debris that incurs risk of tire puncture. The geometric method fails because the low height of this obstacle. In the bottom scenario, the robot must navigate through trail that necessitates contact with with dense, overhanging veg- etation. The semantics-bra based method is overly conservative and does not plan through. both cases, our method suc- ceeds by learning from geometry and vision together.
Limitations and Future Work
While our method represents a promising step towards fully self-supervised, adaptive traversability, there are a number of limitations that must be addressed in future work. First, our method does address occlusions and consistency features different distances. This affects both the visual and geometric features, which affect downstream traversability. We expect that applying existing inpainting methods [52, 12, 9, 17] as well as longer-range planning, can alleviate this problem while remaining self-supervised. Second, our method of aggregating and projecting VFM features to BEV, while effective, is overly simple. We plan to extend our mapping procedure to more expressive map representations such as voxels. Lastly, our representation output is limited in its ability to provide fine-grained trajectory guidance. While we are able to provide maximum speed and cost, we cannot produce certain behaviors such as approaching logs from certain angles, or going at a minimum speed. We ameliorate this by incorporating ceysice-informet costs in our MPPI cost function [16, 53, 54], performing learning trajectory-leve features such speed and curvature.

---

Acknowledgments
The authors would like to thank Amirreza Shaban, Calvin Chung, Ali Agha and Cherie for their advice and insights during the paper writing and review phases, and lan Higgins for help in capturing drone footage of our hardware experiments. This work partially supported by Field AI, Inc. Research was sponsored by the Army Research Laboratory and was accomplished under Cooperative Agreement number W91 191NN-2112-0012 and W911NF-18-2-0211 and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.
The views
References
[1] S. R. Team. Stanford racing team's 's entry in the 2005 darpa grand challenge. Published on DARPA Grand Challenge, 2005.
[2] C. Bellicoso, M. Bjelonic, L. Wellhausen, K. Holtmann, F. Günther, M. Tranzatto, P. Fankhauser, and M. Hutter. Advances in real-world applications for legged robots. Journal of Field Robotics, 35(8):1111-122 2018.
[3] A. Agha, K. Otsu, B. Morrell, D. D. Fan, R. Thakker, A. Caatamaria-Navaree S.-K. Kim, A. Bouman, X. Lei, Edlund, et al. Nebula: Quest for robotic autonomy in challenging envi- ronments; team costar at the darpa subterranean challenge. arXiv preprint arXiv:2103. 11470, 2021.
[4] K. Afsari, Halder, M. Ensafi, S. Vetito, and J. Serdakowski. Fundamentals and prospects of four-legged robot application in construction progress monitoring. EPiC Series in Built Environment, 2:274-283, 2021.
[5] S. Scherer, V. Agrawal, G. Best, C. Cao, K. Cujic, R. Darnley, R. DeBortoli, E. Dexheimer, B. Drozd, R. Garg, I. Higgins, J. Keller, D. Kohanbash, L. Nogueira, R. Pradhan, M. Tatum, V. K. Viswanathan, S. Willits, S. Zhao, H. Zhu, D. Abad, T. Angert, G. Armstrong, R. Boirum, A. Dongare, M. Dworman, S. Hu, J. Jackel, R. Ji, A. Lai, Y. Hsuan Lee, A. Luong, J. Man- gelson, Maier, Picard, K. Pluckter, A. Saba, M. Saroya, E. Scheide, N. Shoemaker-Trey) Spisak, J. Teza, F. Yang, Wilson, H. Zhang, H. Choset, M. Kaess, A. Rowe, Singh, J. Zhang, G. A. Hollinger, and M. Travers. Resilient and modular subterranean exploration with team of roving and flying robots. Field Robotics Journal, pages 678-734, May 2022.
[6] M. Tranzatto, T. Miki, M. Dharmadhikari, L. Bernreiter, M. Kulkarni, Mascarich, O. Ander- sson, S. Khattak, M. Hutter, R. Siegwart, al. Cerberus in the darpa subterranean challenge. Science Robotics, :666)).. 2022.
[7] P. Borges, T. Peynot, S. Liang, B. Arain, M. Wildie, M. Minareci, S. Lichman, G. Samvedi, I. Sa, N. Hudson, et al. A survey on terrain traversability analysis for autonomous ground vehicles: Methods, sensors, and challenges. Field Robotics, 166-166771 2022.
[8] M. Boxan, A. Krawciw, E. Daum, X. Qiao, S. Lilge, T. D. Barfoot, and F. Pomerleau. Fomo: A proposal for multi-season dataset for robot navigation in for)." et montmorency. arXiv preprint arXiv:2404. 13166, 2024.
[9] X. Meng, N. Hatch, Lambert, A. Li. N. Wagener, M. Schmiule, J. Lee, W. Yuan, Z. Chen, S. Deng, et al. Terrainnet: Visual modeling of complex terrain for high-speed, off off-road navi- gation. arXiv preprint drX1v:2303.1577, 2023.

---

[10] D. D. Fan, A.-A Agha-Mo Mohammmohammadi, and E. A. Theodorou. Learning aisk-aware costmaps for traversability in challenging environments. IEEE Robotics and Automation Letters, 7(1): 279-286, 2021.
[11] S. Triest, M. G. Castro, P. Maheshwari, M. Sivaprakasam, W. Wang, and S. Scherer. Learning risk-aware costmaps via inverse reinforcement learning for off-road navigation. arXiv preprint arXiv:2302.00114 2023.
[12] A. Shaban, X. Meng, Lee, B. Boots, and D. Fox. Semantic terrain classification for off-road autonomous driving. Conference on Robot Learning, pages 619-629, PMLR, 2022.
[13] J. Frey. M. Mattamala, N. Chebrolu, C. Cadena, M. Fallon, and M. Hutter. Fast traversability estimation for wild visual navigation. arXiv preprint vrXiv:2305.08500, 2023.
[14] J. Seo, S. Sim, and I. Shim. Learning off-road terrain traversability with self-supervisions only. IEEE Robotics and Automation Letters, 2023.
[15] H. Karman, E. Yang. D. Farkash, G. Warnell, J. Biswas, and P. Stone. Sterling: Self-supervised terrain representation learning from unconstrained robot experience. In 7th Annual Conference on Robot Learning, 2023.
[16] M. G. Castro, S. Triest, W. Wang, J. M. Gregory, F. Sanchez, J. G. Rogers III, and S. Scherer. How does feel? self-supervised costmap learning for off-road vehicle traversability. arXiv preprint arXiv:2209. 10788, 2022.
[17] J. Frey, S. Khattak, M. Patel, D. Atha, J. Nubert, C. Padgett, M. Hutter, and P. Spieler. Roadrunner-leaming traversability estimation for autonomous off-road driving. arXiv preprint wwXi:22402119341 2024.
[18] M. Wigness, S. Eum, G. Rogers, D. Han, and H. Kwon. rugd dataset for autonomous navigation and visual perception in unstructured outdoor environments. In 20/9 JEEE/RS/ International Conference Intelligent Robots and Systems (IROS), pages 5000-5007 IEEE, 2019.
[19] Jiang, Osteen, M. Wigness, and Saripalli. Rellis-3d dataset: Data, benchmarks and analysis, 2020.
[20] D. Maturana, P.-W. Chou, M. Uenoyama, and S. Scherer. Real-time semantic mapping for autonomous off-road navigation. In Field and Service Robotics, pages 335-350. Springer, 2018.
[21] X. Cai, S. Ancha, L. Sharma, P. R. Osteen, B. Bucher, S. Phillips, J. Wang, M. Everett, N. Roy, and J. P. How. Evora: Deep evidential traversability learning for risk-aware off-road autonomy. arXiv preprint arXiv:2311 1.223111.624,, 2023.
[22] N. R. under out-of-distribution obstacles. In 2024 IEEE International Conference on Robotics and Automation. IEEE, 2024.
Ancha, P. Osteen. Deep evidential uncertainty estimation for semantic segmentation
[23] Kim, J. Seo, and Min. Evidential semantic mapping in off-road environments with uncertainty-aware bayesian kernel inference, 2024.
[24] L. Wellhausen, R. Ranftl, and M. Hutter. Safe robot navigation via multi-modal anomaly detection. IEEE Robotics and Automation Letters, 5(2):1326-1333, 2020. 1oi:10.1 109/LAA. 2020.2967706.
[25] G. Williams, Aldrich, and E. A. Theodorou. Model predictive path integral control: From theory to parallel computation. Journal of Guidance, Control, and Dynamics, 40(2):344-357, 2017.
10

---

[26] C. Sevastopoulos and S. Konstantopoulos. A survey of traversability estimation for mobile robots. IEEE Access, 10:96331-96377, 2022.
[27] F. Islam, M. Nabi, and E. Ball. Off-road detection analysis for autonomous ground vehicles:
A review. Sensors, 22(21):8463 2022.
[28] G. Kahn, P. Abbeel, and S. Levine. Badgr: An autonomous self-supervised learning- based navigation system. IEEE Robotics and Automation Letters, 6(2):1312-1319, 2021.
[29] S. Jung, J. Lee, X. Meng, B. Boots, and A. Lambert. V-strong: Visual self-superrised traversability learning for off-road navigation. arXiv preprint arXiv:2312. 16016, 2023.
[30] Abbeel and A. Y. Ng. Apprenticeship learning via inverse reinforcement learning. In Pro- ceedings of the twenty-first international conference Machine learning, page 1, 2004.
[31] B. Ziebart, A. L. Maas, A. Bagnell, A. K. Dey, al. Maximum entropy inverse reinforce- ment learning. In Aaai, volume 8, pages 1433-1438. Chicago, II., USA, 2008.
[32] N. D. Ratliff, D. Silver, and J. A. Bagnell, Learning search: Functional gradient techniques for imitation learning. Autonomous Robots, 27(1):25-53, 2009.
[33] A. Bagnell, D. Bradley, D. Silver, B. Sofman, and A. Stentz. Learning for autonomous navigation. IEEE Robotics & Automation Magazine, 17(2):4-84, 2010.
[34] M. Wulfmeier, P. Ondruska, and Posner. Deep inverse reinforcement learning. CoRR, abs/l507.048ma 50888888888 2015.
[35] M. Wulfmeier, D. Rao, D. Z. Wang. P. Ondruska, and Posner. Large-scale cost function learning for path planning using deep inverse reinforcement learning. The International Jour- nal of Roboties Research,
1073: 2017.
[36] M. Wigness, J. G. Rogers, and L. E. Navarro-Serment. Robot navigation from human demon- stration: Learning control behaviors. In 2018 IEEE international conference on roboties and automation (ICRA), pages 1150-1157. IEEE, 2018.
[37] Y. Zhang, Wang, R. Bonatti, D. Maturana, and S. Scherer. Integrating kinematics and environment context into deep inverse reinforcement learning for predicting off-road vehicle trajectories. arXiv preprint arXiv: 1810.07225, 2018.
[38] I. Gan, W. Grizzle, M. Fustice, and M. Ghaffari. Energy-based legged robots terrain traversability modeling via deep inverse reinforcement learning. IEEE Robotics and Automa- 8814, 14,12442222 2022.
fion Letters, 7(4):8807-88) 18800788844
[39] D. Fan, K. Otsu, Kubo, A. Dixit, J. Burdick, and A.-A. Agha-Mohammadi. Step: Stochastic traversability evaluation and planning for risk-aware off-road navigation. arXiv preprint 21Xi::2103.02828, 2021.
[40] R. T. Rockafellar, S. Uryasev, et al. Optimization of conditional value-at-risk. Journal of risk, 2:21-42, 2000.
[41] R. Schmid, D. Atha, F. Schöller, Dey, S. Fakoorian, Otsu, B. Ridge, M. Bjelonic, L. Well- hausen, M. Hutter, et al. Self-supervised traversability prediction by learning to reconstruct safe terrain. In 2022 JEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pages 12419-12425. IEEE, 2022.
[42] X. Cai, M. Everett, J. Fink, and J. P. How. Risk-aware off-road navigation via learned speed distribution map. arXiv preprint 2rX1v:2203.13429 2022.
[43] A. Kirillov, E. Mintun, N. Ravi, H. Mao, Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 4015-4026, 2023.

---

[44] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin. Emerging properties self-supervised vision transformers. In Proceedings of the IEEE/CVE interna- rional conference on computer vision, pages 9650-9660, 2021.
[45] P. Fankhauser and M. Hutter. A universal grid map library: Implementation and use case for rough terrain navigation. In Robot Operating System (ROS), pages 120. Springer, 2016.
[46] M. Oquab, T. Darcet, Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.
[47] G. Williams, N. Wagener, B. Goldfain, Drews, M. Rehg, B. Boots, and E. A. Theodorou. Information thcoretic mpe for model-based reinforcement learning, 2017 IEEE Interna- tional Conference on Robotics and Automation, pages 1714-1721, IEEE, 2017.
[48] M. Schmittle, R. Baijal, B. Hou, S. Srinivasa, and B. Boots. Multi-sample long range path planning under sensing uncertainty for off-road autonomous driving. arXiv preprint 1rXiv:2403.11298. 2024.
[49] E. Chen, C. Ho, M. Maulimov, C. Wang, and S. Scherer. Learning-on teeig--oott----iie drivee Self-supervised adaptation of visual offroad traversability models. arXiv preprint arXiv:2306.1 15226, 2023.
[50] T. Guan, D. Kothandaraman, R. Chandra, A. J. Sathyamoorthy, K. Weerakoon, and D. Manocha. Ga-nav: Efficient terrain segmentation for robot navigation in unstructured out- door environments. IEEE Robotics and Automation Letters, 11188155, 2022.
[51] A. Asgharivaskasi and N. Atanasov. Active bayesian multi-class mapping from range and semantic segmentation observations. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 1-7. IEEE, 2021.
[52] M. Stölzle, Miki, Gerdes, M. Azkarate, and M. Hutter. Reconstructing occluded elevation information in terrain maps with sule-supervised learning. IEEE Robotics and Automation Letters, 7(2):1697-1704, 2022.
[53] Han, A. Liu, A. Li, A. Spitzer, Shi, and Boots. Model predictive control for aggressive driving over uneven terrain. arXiv preprint arXiv:2311.1228. 2023.
[54] E. R. Damm, M. Gregory, E. S. Lancaster, F. A. Sanchez, D. M. Sahu, and T. M. Howard. Terrain-aware kinodynamic planning with efficiently adaptive state lattices for mobile robot navigation in off-road environments. In 2023 IEEE/RSJ International Conference on Intelli- gent Robots and Systems (IROS), pages 9918-9925. IEEE, 2023.
12


Reviews:
[{"review_id": "VbJ4Pvi4Gu", "reviewer": "Reviewer_XNej", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The main idea of this paper is utilizing pretrained visual foundation models as representations to predict targets such as maximum allowed speed, costmap and uncertainty, suitable for the task of high-speed off-road navigation", "summary_of_recommendation": "The paper comes with an interesting idea and shows real-robot hardware execution, which are positives worthy of presentation at CoRL. The only lacking part is that the evaluations are weak, and I'm looking forward to hear what the authors have to say in their rebuttal regarding that.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "1) Equation 1 is not complete. It seems the assumption is that the expert trajectories and the candidate trajectories rolled out by MPPI are of the same length, since there is no normalization term for different-length trajectories. Correct me if i'm wrong, but I think there should be a term where the costs are normalized by the length of the trajectories. \n2) In the IRL formulation, there seems to be no mention of a regularization term for the costs. How did the authors regularize the costs generated by the cost function regressor ? Was it weight_decay, L1 regularization, etc? \n3) In section 4, parameter count of the shared backbone FCN needs to be mentioned. \n4) Figure 2 is a bit confusing and needs a complete overhaul. Specifically, the FPV visual features are connected to the visual map via an arrow, but it needs to be more descriptive for the reader. Please mention what that arrow signifies (is it perfoming a Lift operation?). \n5) In Fig 2, it seems there are multiple timesteps of visual and geometric map frames, but the features themselves are generated by aggregating over time as mentioned in text. a) What is the network architecture used to handle these time series feature maps b) Why feed same information when it has already been aggregated over time ?  \n6) Need additional info about Datasets 1 and 2- How long was the dataset ? What was the speed profile like? Was the same expert driver used to collect both datasets ? \n7) \"Long\" and \"short\" corrections on Table 3 needs to be defined in text.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "quHxMni6ur", "reviewer": "Reviewer_Wz3t", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The manuscript introduces VELOCIRAPTOR, a method to learn traversability estimation in a self-supervised manner for autonomous navigation of off-road vehicles, which combines visual-foundation models and geometric data in vehicle's Bird's Eye View (BEV). The method also predicts uncertainty estimation along with cost map and speed map which is further utilized by the low level planner and controller. The proposed method doesn't require any human-annotated labels and hand engineering of the traversability estimation function, which makes this method suitable for the application of autonomous off-road navigation.", "summary_of_recommendation": "The manuscript is well organized and comprehensive. It provides cool evaluation demo for the application of off-road autonomous navigation. It has the merit of learning traversability in unsupervised manner, and experiment demo shows the qualitative effectiveness of the method. The scientific contribution is incremental, on the other hand the evaluation and application on real hardware looks promising but lacking quantitive analysis. Given the strengths and weaknesses, my review rate is between neutral and weak accept, hoping the authors can address the review concerns.", "technical_quality": 3, "clarity_of_presentation": 3, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "As mentioned in the weaknesses, mainly, the presentation of the method details can be enhanced and method generalization to various terrain discussion would be preferred. More relevant related-work survey. Additionally, without code release, quantitive analysis and model runtime evaluation for downstream real-world robot tasks are hard for research community benchmark.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IM0aaxNhvU", "reviewer": "Reviewer_K4L2", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes a system for self-supervised traversability estimation in off-road environments. The system uses past LiDAR scans and images to build a aggregated terrain feature map around the vehicle. In particular, geometric features are computed from aggregated LiDAR point cloud, and image features are extracted from Vision Foundation Models. The features are projected into BEV space by averaging, which are then decoded into costmap, speedmap and uncertainty maps. Experimental results show that the system outperforms geometric-only, semantic-only, and simple fusion baselines.", "summary_of_recommendation": "The paper is in general convincing but I would like to see more novelty in system design and implementation.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "Questions for rebuttal\n\n1. In Table 3, why does V-Dino+Unc performs worse on the long run?\n2. What is the motivation of using a heuristics-based uncertainty estimation method? How good is it in terms of false positives? I understand it is difficult to have a quantitative analysis, but it will be informative if you can show a video of a long run visualizing the three maps.\n3. What are the causes of the interventions?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pqmykUFFqM", "reviewer": "Reviewer_yWjL", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 4, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper presents a framework that combines visual foundation models with geometric mapping to learn maps for high-speed off-road navigation in a self-supervised manner without human annotations.", "summary_of_recommendation": "Overall, I think this paper is high-quality and clearly written, and should be accepted.", "technical_quality": 4, "clarity_of_presentation": 5, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "I have a few high-level questions for the author:\n\n1. The proposed method uses self-supervised learning where the only \"label\" comes from human demonstration. How sensitive is it to the quality of human demonstration? Would it be beneficial to further incorporate other information (e.g. body oscillations, slipping) into this framework?\n\n2. How would you extend this framework to a different mobile robot platform (e.g. another off-road vehicle of different dimensions, a legged robot, a wheel-legged robot)?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Strengths:

- Reviewers highlight the paper's strength in using VFMs to effectively combine geometric and semantic features for improved map quality. 
- Reviewer `Ywjl` praises the well-designed experiments with strong baselines and insightful analysis, particularly showcasing VFM benefits in Figure 4 and 5. 
- All reviewers agree the paper is well-written and organized, with Reviewer `Ywjl` emphasizing its easy-to-understand approach and clear presentation of the method. 

Weaknesses:

- Reviewers `XNej` and `Wz3t` both express concern about the limited evaluation, highlighting the need for more robust comparisons and analysis of the proposed method's effectiveness.

Overall, this seem a good paper with solid real world experiments. There is only one review leaning towards rejection comments on the novelty of this work, which the authors give a good response in rebuttal phase. The strength of this paper outweighs the limitations and I would recommend accepting.
Decision: Accept