Perceive With Confidence: Statistical Safety Assurances for Navigation with Learning-Based Perception
Authors: Anushri Dixit
Zhiting Mei
Meghan Booker
Mariko Storey-Matsutani
Allen Z. Ren
Anirudha Majumdar
Venue: CoRL 2024 (2024)
Keywords: Uncertainty quantification
occupancy prediction
robot navigation
TL;DR: An uncertainty quantification framework for perception-based navigation tasks that provides formal assurances for end-to-end safety.

Abstract: Rapid advances in perception have enabled large pre-trained models to be used out of the box for transforming high-dimensional, noisy, and partial observations of the world into rich occupancy representations. However, the reliability of these models and consequently their safe integration onto robots remains unknown when deployed in environments unseen during training. In this work, we address this challenge by rigorously quantifying the uncertainty of pre-trained perception systems for object detection via a novel calibration technique based on conformal prediction. Crucially, this procedure guarantees robustness to distribution shifts in states when perceptual outputs are used in conjunction with a planner. As a result, the calibrated perception system can be used in combination with any safe planner to provide an end-to-end statistical assurance on safety in unseen environments. We evaluate the resulting approach, Perceive with Confidence (PwC), with experiments in simulation and on hardware where a quadruped robot navigates through previously unseen indoor, static environments. These experiments validate the safety assurances for obstacle avoidance provided by PwC and demonstrate up to 40% improvements in empirical safety compared to baselines.

Perceive With Confidence: Statistical Safety Assurances for Navigation with Searning-Based Perception
Anushri Dixit, Zhiting Mei, Meghan Booker, Mariko Storey-Matsutani, Allen Z. Ren, Anirudha Majumdar
Department of Mechanical and Acrospace Engineering, Princeton University
Email: anushridixin wwt.iittccllleee.oo.o.
Abstract: Rapid advances in perception have enabled large pre-trained models to be used out of the box for transforming high-dimensional. noisy, and partial observations of the world into rich occupancy representations. However, the relia- bility of these models and consequently their safe integration onto robots remains unknown when deployed in environments unseen during training. In this work, we address this challenge by rigorously quantifying the uncertainty pre-trained perception systems for object detection via novel calibration technique based on conformal prediction. Crucially, this procedure guarantees robustness to distribu- tion shifts in states when perceptual outputs are used in conjunction with planner. As a result, the calibrated perception system can be used in combination with any safe planner to provide an end-to-end statistical assurance on safety in unseen en- vironments. We evaluate the resulting approach, Perceive with Confidence (PwC), in simulation and on hardware where a quadruped robot navigates through previ- ously unseen indoor, static environments. These experiments validate the safety assurances for obstacle avoidance provided by PwC and demonstrate up 40% improvements in empirical safety compared to baselines.
Keywords: Uncertainty quantification, occupancy prediction, robot navigation
Introduction
How can we decide if the outputs of a given perception system are sufficiently reliable for safety- critical robotic tasks such autonomous navigation? Significant strides in perception over the past few years have enabled large pre-trained models be used out of the box [1] for tasks such as oc- cupancy prediction, which serves fundamental building block for navigation. However, current pre-trained models are still not reliable enough for safe integration into many real-world robotic systems. Despite being trained on vast amounts of data, these systems can often fail to generalize to novel environments [2. 3, 4]. In this paper, we ask: how can we leverage the power of large pre-trained occupancy prediction models while providing safety assurances for robot navigation?
Consider legged robot tasked with navigating in cluttered environment such as a home, office, or warehouse (Figure 1). A typical navigation pipeline for such system consists of two modules: (i) perception module that detects obstacles, and (ii) planner that produces collision-fre trajectories assuming accurate perception. However, there are two challenges associated with obtaining reliable outputs from the perception module. First, the environments in which we deploy our robots will be unseen during training, and thus require generalization to new obstacle geometries, appearances, and other environmental factors. Second, closed-loop deployment of the perception system in con- junction with a planner causes a shift in the distribution of states (e.g., relative locations to obstacles) that are visited by the robot. Since the robot's planner influences future states, the robot may view obstacles from unfamiliar relative poses (Figure 1) and cause the perception system to fail.
In this paper, we address these challenges by performing rigorous uncertainty quantification for the outputs of a trained perception system in order to achieve reliably safe (i.e., collision-free navigation. We utilize techniques from conformal prediction [5] order to lightly process the outputs of pre-trained obstacle detection system in a way that provides formal assurance on correctness: with user-specified probability e, the processed perceptual outputs will correctly detect obstacles in a new enviromment. To enable this, we assume access to a modest sized (c.g.,/+
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

Pood and tolled)
Goal
Calibration w/o accounting for distribution shift
Crash
PwC (ours)
Pre trained perception
Calibration
Distribution
Deployment
states
Figure 1: PWC lightly processes the outputs of a pre-trained perception system (green bounding boxes) using conformal prediction in order to ensure bounded misdetection rate despite any distribution shift in states (gray dots). The calibrated perception system (blue boxes) paired with a -etermimmiiiitic filter and a safe planner provide an end-to-end statistical assurance on safety in new test environments.
400) dataset of environments that are representative of deployment environments with ground-truth obstacle annotations, and use these for calibrating the outputs of the perception system. Crucially, we propose a novel calibration technique that ensures robustness of the perception system to any closed-loop distribution shift in states. Hence, the calibrated outputs can be used in conjunction with any safe planner to provide an end-to-end statistical assurance on safety in new static environments with user-specified threshold e. To the best of our knowledge, this is the first work calibrate given black-box perception system in way that ensures robustness to closed-loop distribution shifts in order to provide end-to-end statistical assurances on safe navigation.
Our framework, Perceive with Confidence (PWC), is evaluated with experiments in simulation and hardware on the Unitree Gol quadruped navigating in indoor environments with objects that are unseen during calibration (Figure 1). We validate PwC's ability to provide end-to-end statistical assurances on collision avoidance, while also providing to 40% increase in safety with only modest reductions in task completion rates compared to baselines that use the pre-trained perception model directly, fine-tune on the calibration dataset, or utilize conformal prediction for uncertainty quantification but do not account for closed-loop distribution shift.
2 Problem Formulation and Overview
Dynamics and environments. Suppose that the dynamics of the robot are described by 5t41 fø(s;, az), where E S is the robot's state at time-step t, dg E A is the action, and E E E is the environment that the robot operates in during a given episode. We primarily focus on navigation with static obstacles; in this context, the environment E specifies the locations and geometries of objects. We assume that environments that the robot will be deployed in are drawn from an unknown distribution Dg, e.g., a distribution over possible rooms that the robot may be deployed in. We will make assumptions this distribution besides the ability sample finite dataset D {E...., EN3 of ii.d. environments from Dz.
Sensor and perception system. The robot is equipped with sensor o S x & O that provides observations 0r oss,, E)) (e.g., depth images) based on the robot's state and environment. We assume access to a pre-trained perception model Φ O + 2, which processes raw sensor observa- tions into an occupancy representation of the environment. In this paper, we work with perception models for obstacle detection that output 3D bounding boxes. The representations (zo.... 2,) up to the current time-step are aggregated into an overall representation m, E M (e.g., a map).
Policy. The representation m, used by a planning algorithm in order to produce actions. Denote the resulting end-to-end policy that utilizes a perception model by nP O + z + M A, which maps histories of sensor observations actions.
Safety and task performance. Let Craffe be a cost function that captures safety (c.g., obstacle avoidance). Specifically, let So,5 denote the allowable set of initial conditions in environment E. Then, ( E {0, 1} assigns cost of policy & maintains safety from any initial state so So,E when deployed over given time horizon in environment E, and cost of 1 otherwise. An additional cost function Clask can be used to capture task performance (e.g., time to reach goal).

---

Goal: statistical safety assurance. Our goal is to provide a statistical assurance safety for the end-to-end policy æ. We propose procedure that uses a finite dataset of environments order to produce a calibrated perception system Φ O S Z 4 Z. Our approach is modular: outputs of the calibrated perception system may be used with any safe planner (cf. Section 5) to ensure:
Csate (TT)===
E E-DE
safe (TT (m0)
S E,
(1)
for a user-specified safety tolerance while also post-processing outputs from & as lightly (i.e., non-conservatively) as possible in order to allow the robot to optimize task performance.
3 Background: Conformal Prediction
Conformal prediction (CP) [5, 6] will be our primary tool for performing rigorous uncertainty quan- tification for perception. Given N iid. (or exchangeable) samples U1.... , Un of scalar random variable U, we compute the threshold, -i--. such that the next sample, Uwst satisfies,
PlUwa S 21-0221- -, Ôn-c- UCICNDDD-ON if + 11 - SN x otherwise,
(2)
where Un1) S U(2) S ... S U(N) are the order statisties (sorted values) of the samples UI....,UN In the CP literature, U is known as the non-conformity score and a measure of the (in)correctness of a model. The above guarantee (2) is marginal, i.e., (2) holds over the sam- pling of both the calibration dataset U1..... UN and the test variable Ues Hence, we will need to .,,...,,
generate a fresh set of i.i.d. calibration data .,,..., UN for the guarantee to hold for new sample Uset However, in practice, one typically only has access to a single dataset of examples; inferences from this dataset must be used for all future predictions on test examples. this work, we use the following condittaon guarantee [7. 8] that doesn't require us to generate of N new samples for every test prediction and holds with probability over the sampling of the calibration dataset:
PlUrest -1-e UN| 2 Betay (6),
L(N + 1)ēJ,
where, Beta -- (0) is the 8- quantile of the Beta distribution with parameters N + +1 u and v, and we can choose é to achieve the desired 1 coverage.
Offline: Calibrating the Perception System
In this section, we describe our approach to the uncertainty quantification of pre-trained perception system. We focus on the challenges highlighted in Section 1: providing statistical assurances on safe generalization novel environments and ensuring that the offline calibration procedure is robust to shifts in the distribution of states induced by the online implementation of the planner.
4.1 Misdetection Rate
We focus on perception systems that output bounding boxes that predict the locations of objects in the environment. As an example, Figure (left) shows one such real-world environment wherein the union A of the black boxes denotes the ground-truth locations of the chairs. Let B, denote the union of the green bounding boxes predicted by the perception system & from robot state s E S. Since the environment in which the robot deployed may contain partially occluded objects that was not explicitly trained on, the perception system's 's outputs may be inaccurate.
Our key idea for ensuring generalization to new, unseen environments and tackling the distribution shift arising from the closed-loop deployment of the calibrated perception system with plannner to a melecy-indeenndent misdetection cost, Cg. which considers worst-case errors across all states in an environment! Cg(() == max.es AggB, We will present a calibration procedure that bounds this misdetection cost with high probability in a new environment, and thus guarantee the correctness of the calibrated perception system independent of the robot policy using CP.
It would be infeasible to consider all possible states in an environment. In practice, we use sampling- based motion planner and consider fixed set of samples for our calibration that could be used by any planner.
3

---

4.2 Calibration Procedure
{Ei,... FE) ~ De (cf. Section 2). In each environment, E, we have access to the union A, of the ground-truth bounding boxes of all the objects in the environment and the unions a, of the predicted bounding boxes gen- erated by the pre-trained perception system & from each state E S. Care is required to ensure that the calibration environments are representative of deployment environments. As such, we construct the calibration dataset either using real-world environments or create simulation environments using real-world data [9, 10, 11] to ensure sufficient variation in environmental factors (e.g., geometry and locations of obstacles, lighting, etc.).
Dataset. We assume access to dataset of i.i.d. environments
Calibration. each calibration environment Es, we find the inflation 4g of the bounding box predictions Bad so as to ensure that all the ground-truth boxes are fully enclosed by the inflated boxes, i.e, S s,,i+ Ag. As E S, where, S is assumed to finite, discrete set. Here, Bz, + Ag. the inflation of each bounding box in the union Bx. by 2g, along each dimension. We define the non-conformity score for environment Eg to be the minimum required inflation in that environment:
U: = min 9 S.t S BaGi + A Vs E.
(4)
Observe that U, S 0 = A, S Bass, Va E S and growing U, signals a worse performance of the pre-trained perception system. We can compute the nonconformity scores for the i.i.d. sampled environments (E1...... EN and the quantile -1-c = Quantile
[(N+1((-E))
Proposition 1 Consider the calibrated perception system o that modifies every bounding box oulput of the perception system & by scaling the predicted bounding boxes as B B + D With probability over the sampling of the dataset used for calibration, the calibrated perception system, o is guaranteed to have an e-bounded misdetection rate on new test environments:
CEno (o))(()) (o))UU,.....U Sc.
E Faves De
(5)
The above proposition (proof Appendix A) gives us formal assurance on the correctness of the perception system independent of the robot's policy. As we describe below, the calibrated perception thus be combined with any safe planner to bound the collision rate
Online: Perception and Planning
Space
Unkmnnn Space
We now focus on the online implementation of the method described in Section 4 to reduce conservatism when used conjunction with safe planner. In general, safe planner takes into account the dynamics of the robot and pro- duces plans the state space We call X' the configuration space of the robot (e.g., z-y lo- cation for point). For any given environment E, we partition X' into three sub-spaces: the known free space free known occupied space xr,, and unknown space Xuunknown
(a) A line-of-sight depth sensor along with bounding box esti- mater partition the configuration space three.
(b) The non-deterministie filter takes intersection over the occu- pied space and takes union over space.
Figure
Non-deterministic filter. We utilize the assurance obtained from Section 4 to implement non- deterministic filter [12, Ch. 1 11.2.2) which shrinks the occupied space and grows the known free space over time. Suppose the robot's perceived partition of the state space at time is denoted by the triplet {X, free X _.inknoww which represents the overall map m, of the environment. At new time step + 1, the robot's perception system returns new estimation for the occupied space, Poee. The filter intersects the occupied spaces: are = x n Preee - We compute the new estimation of free space &frce based on X considering occlusion and limited field of view.
28 is the calibration threshold such that the dataset conditional guarantee (3) achieves the desired (1 coverage with probability 8 = 0.99 over the sampling of the calibration dataset.

---

The new perceived free space is updated by taking the union: Re Pt deterministic filter pairs effectively with method in Section 4 for two key reasons: 1) it mitigates the conservatism of our bounding box expansion by intersecting tic rapidly reducing its size even if the initial prediction with CP bounds appears generous; and 2) Prop. 1 ensures that with high probability in new test environment, X free never intersects the true occupied space X We demonstrate the rapid expansion of known free space in Figure 3 for our simulated setup (Sec. 6).
2 The
Safe planning. With our formal assurance the estimated free space X, we utilize any safe planner [13, 14, 15] ensure end-to-end safety, long as the planner includes safety filter that takes into account the robot's s dynamics in order to reject potentially unsafe actions with the assump- tion of known state and static (but unknown) environment [16, Corollary 1.4). For our simulation and hardware experiments, we use the safe planner proposed in [17], which enforces an inevitable collision set (ICS) constraint [18]. We describe implementation details Appendix C.
Proposition 2 For any user-specified safety tolerance e, the calibrated perception system in Proposition combined with any safe planner that chooses actions based on the outputs of the non-deterministic filter ensures the end-to-end safety for the overall policy π.:
Cgale (T
E E-De
[ ( 1
(6)
where C (T*) is the cost function for safety from Section 2.
This result (proved in Appendix D) is a direct consequence of the formal assurance on the calibrated perception system that ensures correctness from any state in test environment (sampled i.i.d. from the same distribution as the calibration environments) with probability over environments.
Simulated Experiments: Vision-Based Navigation
(b)
(o)==8
(a) Simulation environment in Pybullet.
True occupied space goc
Predicted occupied space X
Predicted space xefr JUnknown space
(8)
(8)
Figure 3: Simulation and -en-ettrrmiiittic filter updates. (a) An example environment simulation. (b d)
The robot begins with large occupied space predictions due to the inflation obtained through offline calibration (Section 4). After few updates, the predicted occupied space _etce shrinks significantly.
We evaluate our approach for vision-based navigation in the PyBullet simulator [19] using diverse set of chairs from the 3D-Front dataset [1]]. We use the 3DETR end-to-end transformer model [20] as our pre-trained perception system.
Baselines. We compare our approach (Perceive with Confidence - PwC) C) to three baselines to illus- trate its effectiveness achieving a user-specified safety rate. First, we consider the most common approach of directly using the outputs the perception system [20] in our planning pipeline. We call this baseline 3DETR. Next, we consider the common practice of fine-tuning the outputs of the perception system using small dataset of task-representative environments Druns (cf. Section E.I). We call this perception system 3DETR-fine f-nne-inn-tueed. Lastly, perform calibration using confor- mal prediction; however, instead of accounting for the closed-loop distribution shift, bound the
5

---

Simulation
Hardware
0.03
Cullision
0.8
Nasdetection
Reached
8.0
Path Length (m))
Dalission
Moddetection
Goel Reathed
% Lenggthghh
0.6
6.8
7.5
0.6
75
0.02
0.6
7.0
0.3
0.6
7.0
0.01
0.2
0.2
0.2
0.00
0.0 0.0
5.0
0.0
0.0
0.0
5.0
3DETR
IDETR-Bre fund
CP-00g
PWC
PaC fine-tnned
Figure 4: (Left) Results for the simulated experiments described in Section 6. Simulations are across 100 new environments with 1-5 chairs. (Right) Results for the hardware trials described in Scction Experiments are across 30 different chair configurations with 4-8 chairs present in cach configuration. Here the path length is averaged only for successful trials for both PWC and CP-avg. duc to the varying goal locations.
misdetection rate averaged across environments and states (similar to [21], which does not utilize conformal prediction, but quantifies expected errors in perception system for pre-defined distri- bution of states). We refer to this baseline as CP- avg. We consider two variations of our approach for comparison to the above baselines. First, we refine 3DETR outputs using our calibration proce- dure described in Section 4. We call this approach PwC. Second, the 3DETR outputs are fine-tuned and calibrated using split conformal prediction as described in Appendix 1; we call this approach PwC-fine-tuned. Details regarding calibration and the planner setup are provided in Appendix F.
Results: Misdetection Rate. We first compare our method, PwC, to the baseline CP-avg that is also calibrated using con- formal prediction but without accounting for the closed-loop dis- tribution shift. We compare the misdetection rate, i.e., whether obstacles in the scene classified as free space at any point during trial. We vary the allowable misdetection bound for each method, and compute the rate of misdetections in 100 test environments. As seen in Figure 5, our method is guarantees a rate of misdetection lower than the threshold while CP-avg violates this threshold for every considered.
(Cang)
1.01
Bowel (EP say)
0.77
Acceptable (d]
0.50
0.25
0.60
0.06
0.10
0.28
0.20
0.5
Miderstion Roard
Figure 5: As relax the confi- dence threshold by increasing e, the misdetection rate increases but re- mains bounded for PwC. The base- line method has misdetection rate much higher than acceptable.
Collision Rate, We compare PWC to the baselines in 100 new environments drawn from the same distribution as cali- bration environments. Figure illustrates one such test environ- ment and the evolution of the free space in this environment using PwC. Figure 3 shows that though the initial calibrated perception system outputs are inflated, the non-deterministic filter is able to expand the predicted free space in few time steps and ensure that the robot can navigate without unnecessary conservatism. The results are summarized in Figure 4 and the metrics for success and failure are described in Appendix F. We observe that our proposed approaches, PWC and P-C-fine- tuned, have no collisions in any environments. While the robot reaches the goal in a slightly lower percentage of environments compared baselines, we emphasize that ours is the only approach that able to ensure a low, statistically guaranteed misdetection rate across test environments.
Results:
To further illustrate the effect of misdetections on safety, we con- sider different distribution of environments wherein we ran- domly place single chair in the straight line path between the initial position of the robot and the goal. For safety thresh- old 1 - = 0.85, we compare PwC, CP-avg, and 3DETR. The results are provided in Figure 6 for 100 new test environments, wherein the goal is reached the robot navigates to within 2 m of the goal. In these environments, the desired safety rate is not met by the baselines while our approach still statistically guaranteed to be safe.
Collision
Goal Rearhed
0.5
0.2
0.3
0.2
0.0
Ch-agg
Figure 6: A comparison between collision rates of different perception systems that use the same planner.
We provide additional simulation results to illustrate the effects of 1) closed-loop distribution shifts on safety in Appendix F.4 wherein PwC is robust to increase in the level of closed- loop distribu- tion shift while the baseline, CP-avg., is not which leads to higher collision rates for CP-avg., 2) the

---

tradeoff in different partition sizes for fine-tuning using Split-CP in Appendix E.1.2, 3) the effect of varying on the safety rate in Appendix F.2, 4) impact of using different number of sampled config- urations for calibration and online planning in Appendix F.3, and 5) comparison against additional uncertainty-aware perception systems that use a heuristic notion of uncertainty Appendix F.5.
Hardware Validation: Vision-Based Quadruped Navigation
We now validate the end-to-end statistical safety assurance of our approach on quadrupedal hard- ware platform. As in our simulation setup in Section 6, the robot is tasked with navigating to a goal location while avoiding different chairs placed in varying configurations across a 8m x 8m room. We utilize the perception system calibrated in simulation with guaranteed safety rate - e = 0.85, and compare our PwC method against CP-avg. (defined in Section 6) across 30 different physical environments (60 trials total). One challenge ensure minimal sim-to-real gap for perception. In order address this, utilize depth measurements the robot's sensory input. This choice facilitates small sim-to-real gap, observed in prior work [22, 23]. See Appendix G for more details about the hardware setup.
thars
Iiee est (CP avg.l
free est (Pwc)
goal
0
[m)
(a) Environment
(b) Environment 4
(c) Environment
Figure 7: (Top) The physical layouts of the example hardware trails. (Bottom) A bird see''seye view of the estimated spaces (shaded regions), and trajectories performed by the robot (solid lines) with our method (blue) and the baseline (orange). In all three trials, PwC is able to successfully navigate to the goal through narrow paths (in Environment 1) and occluded areas/goal (in Environment 3). Baseline approach, CP-avg.. misdetects free space in all environments lcading to collisions in Environments and 3.
Results. For PwC, used the qo. 85 0.73m threshold found in simulation inflate the pre- dicted bounding boxes returned from 3DETR in order to achieve 85% confidence that robot will remain safe in new environments. We summarize key statistics of PwC compared against CP- avg. (90.85 = 0.02) across 30 different environments in Figure (right). Importantly, our trials demonstrate that our confidence bound holds on hardware real environments and without being too conservative. PWC was safe through 90% of the trials and also had comparable path length to the baseline. Meanwhile, the baseline struggled the real environments by having misdetections in each trial and colliding with chair in half of the trials. See Figure for trajectories and free space estimations through several environments with narrow spaces, occluded chairs, and occluded goals. The supplementary video contains full example trials.
7

---

PWC's low misdetection rate and higher success rate in these trials emphasize the efficacy of the bounding box inflation provided by CP paired with the non-deterministic filter. This pairing, in a principled way, inflates the (potentially poor) bounding box detections to properly capture obstacles but quickly shrinks the occupied space with the filter such that the robot can still navigate effectively.
8 Related Work
Safe planning. Collision avoidance is a crucial goal in autonomous navigation. Safe planning methods typically rely on the assumption that the robot has perfect knowledge of its state and envi- ronment [16]. Recent approaches have allowed for occlusion [17, 24, 25, 26] or accounted for losing sight of a previously tracked object [27], but still require either perfect detection of seen objects or bounded sensor noise. Such assumptions impractical for learning- based perception modules that can fail catastrophically in new environments.
Formal assurances for perception- based control. Proposed methods include control barrier func- tions (CBFs) [28, 29]. verification methods on neural networks (NNs) [30, 31], and other learning- based methods [31, 32, 33, 21, 34, 35, 36, 37]. However, these works either do not guarantee generalization to novel environments [30, 31], or ignore closed-loop distribution shifts [34, 21], or require end-to-end training and a good prior [35, 36, 37], or demand usage/design of specific con- trollers [28, 29, 32, 38]. Some make strong assumptions on the perception system [39, 40] that are unrealistic for deployment. In contrast, our method doesn' 't need any of the above, and is lightweight and modular, allowing for the use of any downstream safe planners to ensure end-to-end safety.
Conformal prediction. Conformal prediction (CP) [5, 8] is an uncertainty quantification frame- work particularly suitable for roboties applications [41, 42, 43, 44] where learned modules are de- ployed in environments drawn form unknown distributions. In this work, focus on providing uncertainty quantification for the perception system, which usually involves high-dimensional in- puts and closed-loop distribution shifts. Prior works [43, 21, 45, 46] either provide guarantees for a single environment, assume known environments, or do not account for closed-loop distribution shifts. To the best of our knowledge, this is the first work to obtain end-to-end safety assurances for the perception and planning system in new environments while being robust to closed-loop distribu- tion shifts and amenable to changes in the planner parameters.
Discussion and Conclusions
We presented a modular framework for rigorously quantifying the uncertainty of a pre-trained per- ception model order to provide an end-to-end statistical safety assurance for perception-based navigation tasks. Notably, our statistical assurance holds for generalization new environmental factors (e-g, new obstacle geometries and configurations) and allows for the distribution shift of states that may occur during closed-looy deployment of the perception system with the planner. Our simulation and hardware experiments validated the theoretical safety assurances provided by PwC, while demonstrating significant empirical improvements in safety compared to baseline approaches that do not consider closed-loop distribution shift.
Limitations and future work. One limitation of our work is the assumption of static obstacles. As a future direction, we are interested in quantifying uncertainty both the state of agents moving in the environment and predictions of their semantic labels (e.g., "pedestrian" vs. "bicyclist"), and utilizing game-theoretic planning techniques that account for the uncertainty in the agents' current state and future motion. Additionally, the inflation of bounding boxes we acquire from CP intro- duces some conservatism. We outline extension to approach in Appendix E to address this challenge by utilizing more general occupancy representations beyond bounding boxes, e.g., scene completion networks [47], which produce voxel-wise occupancy confidences. Constructing differ- ent non-conformity score functions that incorporate confidences from a pre-trained model could also potentially reduce conservatism. Lastly, we are interested in uncertainty quantification for per- ception models that support tasks beyond point-to-poing navigation, e.g.. calibrating the outputs of multi-modal foundation models for ianguag--instructe navigation where we ensure accurate detec- tion. We expect that rigorous uncertainty quantification is necessary step towards fully leveraging the power of large foundation models [1] while safely integrating them into future robotic systems.
8

---

Acknowledgments
The authors were partially supported by the Toyota Research Institute (TRI), the NSF CAREER Award [#2044149] 149], the Office of Naval Research [N00014-23-1-2148,, and the Princeton SEAS Innovation Award from The Addy Fund for Excellence in Engineering. This article solely reflects the opinions and conclusions of its authors and not NSF, ONR, TRI or any other Toyota entity.
References
[1] R. Firoozi, J. Tucker, S. Tian, A. Majumdar, J. Sun, W. Liu, Y. Zhu, S. Song, A. Kapoor, K. Hausman, B. Ichter, D. Driess, J. Wu, C. Lu, and M. Schwager. Foundation models in robotics: Applications, challenges, and the future. arXiv preprint arXiv:2312.07883 2023.
[2] E. Arnold, O. Y. Al-Jarrah, M. Dianati, S. Fallah, D. Oxtoby, and A. Mouzakitis. A survey on 3d object detection methods for autonomous driving applications. IEEE Transactions on Intelligent Transportation Systems, 20(10)33782-3777 3795, 2019.
[3] A. Wang, M. Islam, M. Xu, Y. Zhang, and H. Ren. SAM meets robotic surgery: An empirical study in robustness perspective. arXiv preprint arXiv:2304. /4674, 2023.
[4] N. Sünderhauf, O. Brock, W. Scheirer, R. Hadsell, D. Fox, Leitner, B. Upcroft, Abbeel, W. Burgard, M. Milford, al. The limits and potentials of deep learning for robotics. The International journal of robotics research, 37(4-5):405-4200 2018.
[5] V. Vovk, A. Gammerman, and G. Shafer. Algorithmic learning in random world, volume 29. Springer, 2005.
[6] A. N. Angelopoulos, S. Bates, A. Fisch, L. Lei, and T. Schuster. Conformal Risk Control, Apr. org/ abs /200 02814. arXiv:2208.023 022214 [cs, math, stat].
2023. URL
[7] V. Vovk. Conditional validity of inductive conformal predictors. In Asian Conference on Machine Learning, pages 475-490. PMLR, 2012.
[8] A. N. Angelopoulos and S. Bates. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511 2022.
[9] M. Uy, Q.-H. Pham, B.-S. Hua, D. T. Nguyen, and S.-K. Yeung. Revisiting point cloud classification: new benchmark dataset and classification model on real-world data. In Inter- national Conference on Computer Vision (ICCV), 2019.
[10] B. Calli, J. Bruce, A. Walsman, K. Konolige, S. Srinivasa, P. Abbeel, and A. Dollar. Yale- cmu-berkeley dataset for robotic manipulation research. The International Journal of Robotics Research, 36:027836491777007, 04 2017. dd::1001177007778444110000
[11] H. Fu, B. Cai, L. Gao, L.-X. Zhang, J. Wang, C. Li, Q. Zeng, C. Sun, R. Jia, B. Zhao, et al. 3D>FFront: 3D furnished rooms with layouts and semantics. In Proceedings of the JEFF/CVF International Conference on Computer Vision, pages 10933-10942, 2021.
[12] S. M. LaValle. Planning algorithms. Cambridge University Press, 2:3671-366 2006.
[13] T. Schouwenaars, É. Féron, and J. How. Safe receding horizon path planning for autonomous vehicles. In Proceedings of the Annual Allerton Conference on Communication Control and Computing, volume 40, pages 295-304. The University; 1998, 2002.
[14] Bouraine, T. Fraichard, and O. Azouaoui. Real-time safe path planning for robot navigation in unknown dynamic environments. In Conference Computing Systems and Applications, 2016.

---

[15] E. Pairet, J. D. Hernández, M. Carreras, Y. Petillot, and M. Lahijanian. Online mapping and motion planning under uncertainty for safe navigation unknown environments. IEEE Trans- actions on Automation Science and Engineering,
19(4):3356-332 19(
2021.
[16] K.-C. Hsu, Hu, and J. Fisac. The safety filter: unified view of safety-critical con- trol in autonomous systems. Annual Review of Control, Robotics, and Autonomous Systems (ARCRAS), 2023.
[17] L. Janson, T. Hu, and M. Pavone. Safe motion planning in unknown environments: Optimality benchmarks and tractable policies. arXiv preprint 1804. 2018.
[18] T. Fraichard and H. Asama. Inevitable collision states. A step towards safer robots? In Proceedings of the JEEE/RSJ International Conference on Intelligent Robots and Systems, pages 388-393, 2003.
[19] E. Coumans and Y. Bai. Pybullet, Python module for physies simulation for games, robotics and machine learning. http://pybullae org, 2016-2022.
[20] I. Misra, R. Girdhar, and A. Joulin. An end-to-end transformer model for 3D object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2906- 2917, 2021.
[21] D. Sun, B. C. Yang, and S. Mitra. Learming-based perception contracts and applications. arXiv preprint arXiv:2309. 13515, 2023.
[22] A. Loquercio, E. Kaufmann, R. Ranftl, M. Müller, V. Koltun, and D. Scaramuzza, Learning high-speed flight in the wild. Science Roboties, 6(59):eabg5810, 2021.
[23] T. Gervet, S. Chintala, D. Batra, J. Malik, and D. S. Chaplot. Navigating to objects in the real world. Science Robotics, 8(79)eedf6691, 2023.
[24] Z. Zhang and J. F. Fisac. Safe ocelusion-aware autonomous driving via game-theoretic active perception. arXiv preprint arXiv:2105.0816 2021.
[25] C. Packer, N. Rhinehart, R. T. McAllister, M. A. Wright, Wang, He, S. Levine, and
J. E. Gonzalez. anyone there? Learning a planner contingent on perceptual uncertainty. In Proceedings of the Conference Robot Learning, pages 1607-1617. PMLR, 2023.
[26] M. Koschi and M. Althoff. Set-based prediction of traffic participants considering occlusions and traffic rules. IEEE Transactions on Intelligent Vehicles, 6(2):249-265, 2020.
[27] Laine, C.-Y. Chiu, and C. Tomlin. Eyes-closed safety kernels: Safety for autonomous sys- tems under loss of observability. arXiv preprint arXiv:2005.07144, 2020.
[28] Dean, Taylor, R. Cosner, Recht, and A. Ames. Guaranteeing safety learned percep- tion modules via measurement-robur control barrier functions. In Proceedings of the Learning for Dynamies and Control (L4DC) Conference, pages 654-670. PMLR, 2021.
[29] C. Dawson, B. Lowenkamp. D. Goff, and C. Fan. Learning safe, generalizable perception- based hybrid control with certificates. arXiv preprint arXiv:2201.00933 2022.
[30] C. Hsieh, Y. Li, D. Sun, K. Joshi, S. Misailovic, and S. Mitra. Verifying controllers with vision- based perception using safe approximate abstractions. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 41(11):4205 42166005566666222111111166666616 2022.
[31] S. M. Katz, A. L. Corso, C. A. Strong, and M. Kochenderfer. Verification of image-based neural network controllers using generative models. Journal of Aerospace Information Sys- tems, 199)):57--544 2022. 2022.
10

---

[32] S. Ghosh, Y. V. Pant, H. Ravanbakhsh, and S. A. Seshia. Counterexample-guiee guided synthesis of perception models and control. In Proceedings of the IEEE American Control Conference, pages 3447-3454. IEEE, 2021.
[33] S. Dean and B. Recht. Certainty equivalent perception- bassdd control. In Proceedings of the Learning for Dynamics and Control (L4DC) Conference, pages 399-411. PMLR, 2021.
[34] Y. Liu, N. Mishra, M. Sieb, Y. Shentu, P. Abbeel, and X. Chen. Autoregressive uncertainty modeling for 3d bounding box prediction. In European Conference on Computer Vision, pages 673-694, Springer, 2022.
[35] A. Majumdar, A. Farid, and A. Sonar. PAC-Bayes control: Learning policies that provably generalize to novel environments. The International Journal of Robotics Research, 40(2-3): 574-593, 2021.
[36] A. Farid, S. Veer, and A. Majumdar. Task-driven out-of-distribution detection with statistical guarantees for robot learning. In Conference on Robot Learning, pages 970-980. PMLR, 2022.
[37] A. Farid, D. Snyder, A. Ren, and A. Majumdar. Failure prediction with statistical guarantees for vision-based robot control. In Proceedings of Robotics: Science and Systems, 2022.
[38] R. Sinha, Schmerling. and M. Pavone. Closing the loop on runtime monitors with fallback- safe mpe. In 2023 62nd IEEE Conference on Decision and Control (CDC), pages 6533-6540. IEEE, 2023.
[39] S. Dean, N. Matni, B. Recht, and V. Ye. Robust guarantees for perception- based control. In Proceedings of the Learning for Dynamics and Control (LADC) Conference, pages 350-360. PMLR, 2020.
[40] G. Chou, Ozay, and D. Berenson. Safe output feedback motion planning from images via learned perception modules and contraction theory. In Algorithmic Foundations of Robotics XV, pages 349-367. Springer International Publishing, 2023.
[41] A. Z. Ren, A. Dixit, A. Bodrova, S. Singh, S. Tu, N. Brown, P. Xu, L. Takayama, F. Xia, J. Varley, Z. Xu, D. Sadigh, A. Zeng, and A. Majumdar. Robots that ask for help: Uncertainty alignment for large language model planners. arXiv preprint arXiv:2307.01928 2023.
[42] L. Lindemann, M. Cleaveland, G. Shim, and G. Pappas. Safe planning dynamic environ- ments using conformal prediction. IEEE Robotics and Automation Letters, 2023.
[43] A. Dixit, L. Lindemann, S. X. Wei, M. Cleaveland, G. J. Pappas, and J. W. Burdick. Adap- tive conformal prediction for motion planning among dynamic agents. In Proceedings of the Learning for Dynamics and Control (L4DC) Conference, pages 300-314 PMLR, 2023.
[44] R. Luo, Zhao, J. Kuck, B. Ivanovic, S. Savarese, E. Schmerling, and M. Pavone. Sample- efficient safety assurances using conformal prediction. In International Workshop on the Al- gorithmic Foundations of Robotics, pages 149-169. Springer, 2022.
[45] S. Yang, G. Pappas, R. Mangharam, and L. Lindemann. Safe perception- control under stochastic sensor uncertainty using conformal prediction. arXiv preprini arXiv:2304.00194, 2023.
[46] S. Park, O. Bastani, N. Matni, and I. Lee. PAC confidence sets for deep neural networks via calibrated prediction. arXiv preprint arXiv:2001.00106 2019.
[47] E. Chatzipantazis, S. Pertigkiozzglou, Dobriban, and K. Daniilidis. SE(3)-equivariant tention networks for shape reconstruction in function space. arXiv preprint 27X.::2204.02399, 2022.
11

---

[48] L. Janson, E. Schmerling, A. Clark, and M. Pavone. Fast marching tree: A fast marching sampling-based method for optimal motion planning in many dimensions. The International Journal of Robotics Research, 34(7):883 921, 2015. ISSN 0278-3649.
[49] E. Schmerling, L. Janson, and M. Pavone. Optimal sampling based motion planning under dif- ferential constraints: The driftless case. In Proceedings of the IEEE International Conference on Robotics and Aulomation, pages 2375. IEEE, 2015.
[50] E. Schmerling, L. Janson, and M. Pavone. Optimal sampling- based motion planning under differential constraints: The drift case with linear affine dynamies. In Proceedings of the IEEE Conference Decision and Control, pages 2574-2581 IEEE, 2015.
[51] H. Rezatofighi, Tsoi, J. Gwak, A. Sadeghian, I. Reid, and S. Savarese. Generalized inter- section over union: A metric and loss for bounding box regression. Proceedings of the Conference Computer Vision and Pattern Recognition (CVPR), pages 658-666 2019.
[52] K. Neklyudov, D. Molchanov, A. Ashukha, and D. Vetrov. Variance networks: When expec- tation does not meet your expectations. International Conference on Learning Representa- tions, 2019, URL https:/ http///pppeeeeeieeeeeeeeeeeeeeet net http://wwwee///oormaddddadaa...e Ldd
[53] J. Park, Florence, J. Straub, R. Newcombe, and S. Lovegrove. Deepsdf: Learning continu- ous signed distance functions for shape representation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[54] S. Levine, A. Kumar, G. Tucker, and J. Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643, 2020.
[55] S. Ross, G. Gordon, and D. Bagnell. A reduction of imitation learning and structured predic- tion no-regret online learning, In Proceedings of the International Conference Artificial Intelligence and Statisties, volume 15, pages 627-635 PMLR, 2011.
[56] M. Inc MATLAB version: 9.13.0 (r2022b), 2022. URL https: /tl:://wwwwww http://wwwwttt......./ coM.
12

---

A Proof of Proposition
As seen in Appendix 3, conformal prediction gives us the following Cataset-conditional guarantee on new sample of the nonconformity score Uws corresponding to test environment Evsx- With probability 1 å over the sampling of U1, ....... UN.
P(Uest I 9 UN| -e
We can rewrite the event U I 91- as:
(Ved S q/-ec
2 min Qeea Aus S Bs,test + Apn Vs E ES}
Asest & Bsstest + A. E s}
{ [Aest S Ba,tosts Vs E S}
{CEnn (ō) = =0}.
which gives us the desired result (5).
B Implementation with a limited field-of-view
natural question that arises after following the calibration procedure described above is: what happens if the robot is not able to observe objects in the environment from all states? This may happen due limited sensing capability or because some parts of the environment are occluded from view. We address this issue in calibration procedure implementation by only taking into account perception errors for objects that are within the held-of-view of the robot in a given state, and masking any ground-truth -trutttthhhhh bounding boxes that are not visible to the robot, i.e., A (which now depends on state 8) is the union of all the ground-truth bounding boxes of the visible objects. Hence, the perception system correctness assurance stated above holds for all objects within the field-of- view of the robot at any given state. The presence of possibly occluded obstacles are dealt with by safe planner, which we describe next.
C Planner implementation details
For our simulation and hardware experiments, we use the safe planner proposed in [17] due to its approximate optimality and case of implementation. The safety filter in this case is an inevitable collision set (ICS) constraint [18], where the robot forbidden to enter any state that will eventually result in collision no matter what control actions are taken. Within the known free space X, free the robot plans using the fast marching tree algorithm (FMT*) [48] with dynamics [49]. If the goal is not visible within Tfee the robot plans to an intermediate goal on the boundary of its free space. The intermediate goals are chosen based on the cost-to-come from current robot state to the intermediate goal, and the distance-to-go from the intermediate goal to the actual goal. The robot x replans whenever it receives sensor update and updated t+1 from its non-deterministic filter, and accounts for ICS constraints [50] in-between sensor updates.
D Proof of Proposition 2:
As shown in Proposition 1, the misdetection rate of the calibrated perception system is e-bounded on environments drawn from at each time step 1, where the robot at state 8t. other words, the predicted occupied space 2roce each time step contains the true obstacles A with high probability across environments. Conversely, the predicted free space &fte each time step does not contain the true obstacles A with high probability across environments, If consider selety-relevant
13

---

misdetection cost at time step t:
if A S &fte (unsafe), otherwise,
Cr (ō. 8,) =
then the misdetection rate over the set of states should be e-bounded across environments by Propo- sition 1:
max ENDD tE(0,T]
Caafe (ő, 8x) S.
(8)
Because the expectation in Equation (8) is over the set of environments, the following statement holds in any environment (with probability 1 - 8 over the calibration dataset of environments),
Pr max tE(0,21
Éz (ō. (ō. 8t) = =0} 2 c
Given m {x free x X Nunknown }. safe planner never drives the robot outside of the free space. Therefore, the safe planner guarantees Ctaff((=))) I Caa (Ø).
Pr Caaft (x²) = 0} 2 c.
(10)
E Extensions
In this section, we outline few extensions to the basic technical approach described in Sections 4 and 5: (i) fine-tuning a pre-trained perception model, (ii) incorporating sensor and dynamics uncer- tainty, and (iii) calibrating perception modules beyond bounding box prediction.
E.1 Fine- TininTuning& a Pre-Trained Perception Model
In Section 4, we assumed access to pre-trained perception model & that outputs bounding boxes. The conformal prediction- based uncertainty quantification procedure then uses the calibra- tion dataset = {F1,... Ex } of environments to produce a calibrated perception system Φ which lightly processes the outputs Φ by inflating the predicted bounding boxes. In practice, it may also be useful to fine-tune for our target deployment environments before performing uncertainty quantification.
This can be achieved using split conformal prediction [8], where one splits the overall dataset into D Drune U Deat If the perception model takes the form of neural network &w. parameterized by weights w.
use D to fine-tune (or the weights of residual network). We can then utilize Deal in order to perform the CP-based calibration described in Section 4. As demonstrate in Section 6, this additional fine-tuning step before calibration can reduce the conservatism of outputs and improve end-to-end success rates.
The typical choice of loss function for training bounding box predictor is the generalized intersection-over--niioo (gloU) loss [51]. This is a differentiable version of the IoU loss: given ground-truit bounding box and a predicted box B, one computes L(A, B) := |An BI/|AU A B]. However, while this loss is popular in computer vision, particular, the IoU loss symmetric: does not distinguish between the ground-truth and predicted bounding box and thus does not encourage the predicted box to contain the ground-truth box. We propose modification to the gloU loss in Appendix E.1.1, which encourages that the predicted bounding box encloses the ground-truth box while also ensuring that the predicted box is not too large. Similar to the gloU loss, this loss is (almost-everywherr) differentiable and scale invariant. We utilize this loss for fine-tuning in our experiments (Section 6). However, one could use any other method for finetuning not limited training a simple neural network with gloU loss [52].
not suitable for robot navigation. In
E.1.1 Loss Function for Fine- Tine-Tuning
14

---

C
B
A\B
B(A
C\(AUB)
Figure 8: Visualization of different terms in the loss function for a single object setting.
We use (almost-everywhere) differentiable loss function for training. The loss function seeks to ensure that the predicted shape (e.g.. bounding box) encloses the ground truth shape while also ensuring that the predicted shape not too large.
Let's consider the simplest setting wherein we have one object in the scene and we are making single prediction. In this case, denotes the (convex) ground-truth shape and B denotes the (convex) predicted shape. Let denote the convex hull of A and B. Our loss function a weighted combination of three terms,
JAB A
BAA B
(A
:: w1l 1 + w222 + wal3 = Wi
+ 02
+ W3
|C|
The first term is the most important; it tries ensure that B encloses A. The second term tries to make sure that B is not much larger than it needs to be, see Figure 8. The first and second terms are sufficient if A and B are overlapping. However, if they do not overlap. there is no gradient information provided by the first two terms. Following [51], we introduce a third loss term order to provide gradient information when the shapes do not intersect. The loss terms 4,, 12, La are each bounded within [0, 1]. Hence, if choose W1, W2, Wy such that Σ. w, 1, then the overall loss also bounded within [0, 1]. Now let's consider the setting wherein, denotes the union of multiple ground-truth bounding boxes (say have objects in the scene) and B is the union of all the predicted bounding boxes (we predict n boxes). We consider all the individual bounding box predictions B,, Vi E [I....n] n and associate the closest visible ground-trute bounding box A to each prediction. Now we can define C, as the convex hull of A, and B, and the resulting loss function, Lis
|Ar
[BA
CAA(A, U B+I
L = w1
W2
+ W3
[Aj
B
[C/
Hence, the overall loss is,
L
Σ L.
Please refer to (51, Appendix 4.3] for instructions on how to compute the loss analytically for axis-aligned bounding boxes.
E.1.2 Simulation Results Efffec of finetuning dataset size
Upon collecting a calibration dataset of ~ 400 environments, as described in the experiment setup in Section 6, we may choose to use smaller subset of the calibration dataset to further finetune the pre-trained perception model to perform better in the types of environments we are interested in deploying the robot in. We consider the effect of different dataset split sizes for finetuning and then calibration. Using larger of environments for finetuning Daueel may result in a better tuned model, but will leave fewer environments for calibration, Deails resulting in more conservative è and 91 - that satisfies the dataset-condittonall guarantee (3), and vice versa. This trade-off is seen
15

---

in Table 1, where we observe the best performance when we have an equal split between finetuning and calibration.
Split size ( [Diuee] + Deal))
40.85 (in m)
Collision
Misdetection
Goal
Reached
100 + 300
0.68
0%
1%
89%
200 + 200
0.64
0%
1%
94%
300 + 100
0.93
0%
2%
76%
Table A comparison of the effect of various partition sizes for finetuning and calibration for PwC.
E.2
Sensor Errors and Dynamics Uncertainty
In Section 2, we modeled the robot's sensor as a deterministic mapping a :S x provides observations from a particular state in a given environment. This formulation allows to also incorporate sensor errors. Specifically, any errors randomness in the sensor be formally included as part of the environment E E &. Thus, in addition to sampling environmental variables such as obstacle locations, geometries, etc., each environment E also samples random variables that prescribe sensor errors from each state E S in the environment. This way of modeling sensor errors allows a to be deterministic (since all sources of randomness are included in E), allows the sensor errors to be dependent on the relative pose of the robot relative to obstacles (e.g., modeling the fact that depth estimates are often further from ground-truth depth values as distance increases), and also allows to model correlations in sensor errors from different locations (e.g., capturing the fact that sensor errors from nearby robot locations can be highly correlated). Modeling time- varying sensor errors (i.e., different sensor errors from the robot state at different times) is not as immediate, but could potentially be incorporated by augmenting the state space S to include the time-step.
O, which
In addition to errors in sensing, one can also account for uncertainty in the dynamics of the robot by using robust planner (see [16] for an overview). In the experiments described in Section 7, we incorporate uncertainty by generating plans that prevent the robot from entering the inevitable collision set (ef. Section 5) even with bounded uncertainty in the dynamics.
E.3 Calibration with General Occupancy Prediction Models
Section 4 introduced the CP-based calibration procedure in the context of bounding box prediction. However, the theoretical formulation in Section 4 is applicable more general occupancy prediction models; the key requirement is the presence of a scalar quantity that monotonically grows the size of the predicted occupied space (e.g., g., the inflation parameter for bounding boxes in Section 4). This allows one to define the non-conformity score U, for an environment E; as in (4) to be the smallest scalar such that the inflated predicted occupied space contains the ground-truth obstacles (for all robot locations). Hence, we can calibrate the outputs of any perception system that predicts an occupied set or performs occupancy prediction more generally, i.c., assigns a (heuristic) occupancy confidence each point in the space. Possibilities for the latter include scene completion networks [47] deep signed-distand digned-distance function representations [53]. A threshold as the scalar parameter that monotonically controls the size of the predicted occupied space. The conformal prediction procedure from Section 4 can then be used to find confidence threshold such that predicted occupied space contains the true occupied space (with probability 1 - e in a new environment).
this confidence acts
F Calibration and planning
We collect a calibration dataset of 400 environments wherein we randomly place 1 5 chairs from the diverse 3D-Front dataset [1]] in 8 m x8 m room. In this 8 m x8 m space, we use a fixed motion planner and use the same set of
set 2000 sampled configurations for the sampling-base samples for the calibration procedure. We construct the calibration dataset simulation using CAD models of real fumitture pieces from the 3D-Front dataset [11]. which contains highly diverse array of industrial CAD models developed by professional designers to ensure that the performance of the
16

---

perception system remains the same in its simulation and hardware implementation. Similarly, we collect an additional fine-tuning dataset Dune consisting of 100 environments. These environments include ones with occlusions of the goal and objects in the scene.
F1 Metrics for experiments
We simulate the dynamics of the Unitree Gol quadruped robot and task the robot with navigating to a goal location that is ~ 7m away from the initial location of the robot. The robot camera has a field of view of 70 and a visibility range of [1, 5] m. With an allowable misdetection rate of = 0.15, we obtain 90.85 = 0.75 m for PWC, $0.85 0.65 m for PwC-fine-tuned, and 9o.s5 = 0.05 m for CP-avg. through calibration. The planner replans and obtains a new sensor observation to update the filter every 0.5 or less (if the previous plan is already completed).
We utilize the following metrics for our simulation experiments: a trial is counted as a collision the robot collides with an obstacle and we count a misdetection for a trial if the free space predicted by the planner has any intersection with the ground truth bounding boxes of the obstacles. We say that the goal has been reached in given trial if the robot is able navigate within 1 m around the goal in less than 140s We also record the average path length for trials in which the goal is reached.
F.2 Results: Effects of varying on safety rate
CP-avg PwC
0.20
95%
100%
We compare our method, PWC, to the baseline CP-avg. We vary the allow- able safety rate e for each method, and compute the rate of safety 100 test environments. As seen in Table 2 and Figure 5, our method guarantees not only that the rate of misdetections the safety rate. The safety rate of PwC is also consistently better than that of CP-avg.
0.10
98%
99%
0.15
99%
100%
0.10
98%
100%
0.05
98%
100%
guaranteed to be bounded, but also
Table 2: A compari- son of the safety rates of CP-avg. and PwC when we vary the con- fidence threshold e.
F33 Results: Effect of varying the number of sampled configurations
For our experiments, we used a fixed set of 2000 sampled configura- tions. However, depending on the plan- ner configuration requirements and de- sired speed of computation, the user may decide to have a different number of configuration samples for calibration and planning. We study the change in the CP inflation, 40.85. the resulting collision, misdetection, and task completion (reaching goal) rates. As we can see in Table in our case, we have far fewer misdetections with fewer samples, but also observe decrease in number of times the robot reaches the goal. We suspect that with fewer samples of configurations (consisting of z, y, Uay vg), it is harder for the sampling-based motion planner to find feasible paths. On the other hand, we also observe a less conservative qu 85 when we use fewer samples; this is presumably also result of us- ing fewer samples to compute the non-conformity score that comprises of the worst-case perception error across all configurations.
# samples Go 85
Collision Misdelection % Goal Reached
1050
0.7086
0%
1%
47%
1500
0.6910
0%
0%
57%
2000
0.75
0%
7%
90%
Table 3: A comparison of the CP inflation, 9o.65 when we vary the number of sampled configurations.
F44 Results: Effects of closed-loop distribution shift on misdetections
In addition to the challenge of generalization, we highlight another challenge that any uncertainty
Method
Collision Mis-detection KL-divergence
CP-avg. (w = 1)
14%
54%
2.09
CPavg (w 10)
2%
64%
2.72
quantification method for perception must tackle. Suppose we fix a policy T (that uses perception system Φ) and collect dataset of observations in different calibration environments from the states that result from applying . We can use ground- truth bounding boxes in these environments to produce calibrated perception system Ø with statistical assurance correctness for the distribution of observations induced by क However, if
PWC
0%
0%
1.48
PWC (w = 10)
0%
2%
2.04
Table 4: A comparison of the effect of changing the planner parameters on CP-avg. and PwC.
17

---

we now apply the policy Th using the calibrated perception system o, the resulting distribution of states will be different from the distribution that forms the calibration dataset, thus invalidating the statistical assurance. We refer to this challenge as closed-loop distribution shift, which is similar to challenges that arise offline reinforcement learning [54] and imitation learning [55].
To illustrate the effect of closed-loop distribution shifts on misdetections, we used exactly the same setup described above to obtain the simulation results in Figure 4. We changed the planner cost to have different weighting on the cost-to-go. For one setting, we chose a weight cost-to-go, which is the same as the weighting on the cost-to-come. In another setting, we chose weight 10 on the cost-to-go, and hence 10x more emphasis on the cost-to-go compared to the cost-to-come. Table 4 shows the KL-divergence between the states visited by the planner and the sampling distribution of states for calibration as a measure of the closed-loop distribution shift. Increasing closed-loop shifts lead to higher misdetections. One can see that a simple change in the planner parameters can lead to potentially large changes in the safety rates for CP-avg. The closed- loop shift we may see in practice is unknown apriori. Hence, is difficult to make any statements on the planner safety in closed-loop despite using CP for calibration of the perception system. PwC, on the other hand, is robust the closed-loop shifts and can still satisfy the misdetection and safety assurance regardless of the planner parameters used.
on the
E,5 Results: Comparison to heuristic inflation
We compare PwC to the baseline method of inflating the bounding box predictions based on some heuristic confidence level, i.e., we scale the bounding box with 1 confidence (so we scale the boxes where we are less confident by larger amount). While this baseline demonstrates higher completion rate, both the collision rate and the misdetection rate increase significantly, leading to unsafe situations. Further, while method provides statistical safety guarantee, the baseline method does not admit any formal assurance.
Method Collision % Misdelection % Goul Reached %
PWC 0%
7%
90%
Heuristic
3%
67%
Table 5: A comparison of the effects of using
heuristic inflation versus PwC.
G Hardware
G.1 Hardware and Environmental Setup
We represent the robot's state as st [r, y, Uns uy]" where æ and y are its position in the envi- ronment and Va and vy are the respective velocities. For each trial, the robot initialized around position [4, 0Jm (with the origin set to bottom left corner of the room) and has time horizon of 60 seconds to reach the goal within Im radius. The robot replans every Is in a receding horizon manner using the safe planner described in Section 5. The goals are varied every 10 environments and include positions [2, mm [7, 0Jm, and [6, 7mm.
Hardware. We use the Unitree Gol quadruped robot with fully onboard sensing and computation. The robot is equipped with a ZED 2i RGB-D camera and ZED Box computer attached to the base of the robot as shown the top row of Figure 7. The Zed 2i provides the Gol with point cloud observations with a 70" field of view and a visibility range of [1, 5]m. The Zed 2i also uses vision- inertial odometry to provide accurate positional state estimates in the environment. The Zed Box includes an 8-core ARM processor and 16GB Orin NX GPU. This allows to process the point cloud observations in order to produce bounding boxes using the pre-trained 3DETR model [20]. The bounding boxes are aggregated over time to update the estimated free, occupied, and unknown spaces as described in Section 5. The safe planner described in Section 5 is used to output Cartesian velocity commands bounded at a speed of 0.8m/s; these commands are sent from the Zed Box over UDP to the Gol's processor. Our method implemented real-time on the Zed Box hardware with replanning every 0.5 seconds of which the non-deterministic filter takes 0.00025 seconds to run. The dynamics of the Gol are estimated using MATLAB's System Identification Toolbox [56] and provided in Appendix G.2.
18

---

Environments. We test the robot in 30 different environments, consisting of various chair config- urations and geometries an m x8 m room. Configurations range from random, occluded goal, occluded chairs, clustered chairs, and narrow paths (approximately 1.8m in width leaving 0.4m of available freespace for PWC to find). Each environment has between and chairs present. See Appendix G.3 and G.4 for the unseen chairs used testing and the environment configurations re- spectively. We Vicon motion capture system log the ground-truth placement and bounding boxes of the chairs for each environment.
G.2 System Identification
To perform system identification of the Unitree Gol quadruped robot, we collected trajectories us- ing a Vicon motion capture system. We then used MATLAB's system identification toolbox [56]- Specifically, we provided an initial linear ODE grey box model guess and then used prediction error minimization (PEM) for refinement. The resulting system shown in (11) where z and y describe the positional state of the robot in the environment, V/2 and By describe the respective velocities, and uz and Uy describe the respective commanded velocities.
0
0
0
0
y
0
1
y
+
Up
(11)
0
0
-2.5170
0.1353
Ur
2.3350
0 4.6510
Mly
0y
-0.5197
-3.9680
G.3
Chair Test Dataset
Our dataset of chairs for the experiments conducted in Section 7 included chairs with diverse sizes and geometries unseen in training and calibration for the perception system. Test chairs are shown below in Figure 9.
Figure 9: New, unseen test chairs used in hardware experiments.
G.4 Environments
As described Section 7, the robot was tested 30 unique environments with varying furniture configurations and goals. The following 30 figures show an image of each configuration, accompa- nied by hird's-eye map the obstacle and goal locations.
19

---

goal
chairs
Chalrs goal
4
Em)
(m)
[m]
(1) Environment
Eavironment 2
(3) Environment
20

---

chairs
chairs gool
goel
0
(4) Environment
(5) Environment 5
(6) Enviromment
chairs
clakss goal
[ml
uml
2
2
4
0
0
(mi)
(mm)
[m))
Euvironmentt
Eavironment
Environment
21

---

chalts
clair gool
glair goel
(m)
(10) Environment 10
(11) Environment 11
(12) Environment 12
Chaits
chairs gool
chaiss goal
4
0
(m)
(m))
X (11)
(13) Euvironment 13
(14) Eavironment
Environment
22

---

goal
gool
goal
0
0
(16) Environment 16
(17) Environment 17
(18) Environment
Chirss
chaiis gool
goal
0
00
03
7
mm)
mmm
(mm)
(19) Environment 19
(20) Eavironment
Environment
23

---

chairs
goal
Chalis goal
0
(22) Environment 22
(23) Environment 23
(24) Environment 24
Chairs
chaiss
gual
(m)
X (m))
x (mm)
(25) Euvironment 25
(26) Eavironment
Environment

---

chairs
Chairs
Chairs
(ml
1m3 (30) Environment 30
(28) Environment 28
(29) Euvironment 29
25


Reviews:
[{"review_id": "OjzRCMQd32", "reviewer": "Reviewer_UaeJ", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper proposes the use of Conformal Prediction (CP) to calibrate the output of object detectors. The paper uses standard CP argument to show statistical guarantees of the classification model. The paper demonstrates the efficiency of the proposed method using multiple experiments.", "summary_of_recommendation": "While the paper is interesting, the technical argument behind Proposition 1 and 2 is not thorough and several hidden (and strong) assumptions are not thought out thoroughly.", "technical_quality": 4, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "The authors should address the weaknesses number (1), (2), and (3) in the review above.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tvoXE6NR7Z", "reviewer": "Reviewer_a8KQ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "To handle the reliability issue in using pre-trained learning-based perception models for planning in navigation, the authors proposed a simple yet effective uncertainty quantication framework based on conformal prediction. The proposed method can provide formal assurance of the correctness in its outputs and is robust against the state distribution shift during closed-loop deployment. Emprical evaluation of the proposed idea has been conducted in both simulation and real-world, showing strong benefits in decreasing collision and mis-detection rates with comparable success rates.", "summary_of_recommendation": "Due to the insufficient clarity in the presentation and lack of dicussion/experiments for the related work, I would vote for a weak reject at the current stage.", "technical_quality": 3, "clarity_of_presentation": 2, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "My suggestion is to address the points articulated in the list of weakness above.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "A7KPEyPNK6", "reviewer": "Reviewer_B6rP", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper presents a novel framework, Perceive With Confidence (PWC), that provides statistical safety assurances for navigation in robots using learning-based perception models. By leveraging conformal prediction for uncertainty quantification, the authors ensure robustness to distribution shifts in environments, leading to improved safety in both simulated and real-world scenarios.", "summary_of_recommendation": "The proposed framework, Perceive With Confidence (PWC), introduces a novel approach to uncertainty quantification through conformal prediction, providing robustness to distribution shifts. The thorough validation through both simulation and hardware experiments demonstrates the practicality and effectiveness of the approach. Despite some limitations, such as the assumption of static obstacles and the conservatism introduced by bounding box inflation, the paper offers substantial advancements in the field of robotics and presents a solid foundation for future research.", "technical_quality": 3, "clarity_of_presentation": 2, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "• The use of bounding box inflation to ensure safety can introduce conservatism, potentially affecting the robot’s efficiency. Can you discuss potential strategies to mitigate this conservatism without compromising safety?\n\n • The results indicate that varying the confidence threshold (ϵ) affects the misdetection rate. Can you provide more insights into how different safety thresholds impact overall system performance, and what guidelines would you suggest for selecting an appropriate threshold?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper is an interesting application of CP to object detection. The reviewers bring up several issues that should be addressed in the rebuttal:

- It is unclear which states can be used by the motion planner, is it the same set of states used for calibration?
- The explanation of the non-deterministic filter is unclear.
- The paper should be more self-contained (hard to find Equation 3 in [2])
- The paper would benefit from comparisons to other baselines and ablations
- There needs to be discussion of sim2real for this kind of approach
- Need to discuss the speed of the method

Please see reviewer comments for more details.

-----Post-rebuttal comments----

The authors' rebuttal clarified several issues raised by the reviewers, but the reviewers' ratings remain unchanged. Overall, the need for safety guarantees on learned systems is quite critical, so this paper is in an area that should be given more attention. While there are some limitations to the method (e.g. the fixed set of samples), there is still a contribution here.
Decision: Accept