REDOUBT: Duo Safety Validation for Autonomous Vehicle Motion Planning
Authors: Shuguang Wang
Qian Zhou
Kui Wu
Dapeng Wu
Wei-Bin Lee
Jianping Wang
Venue: NeurIPS 2025 poster ()
Keywords: autonomous driving
motion planning
Primary area: applications

Abstract: Safety validation, which assesses the safety of an autonomous system's motion planning decisions, is critical for the safe deployment of autonomous vehicles. Existing input validation techniques from other machine learning domains, such as image classification, face unique challenges in motion planning due to its contextual properties, including complex inputs and one-to-many mapping. Furthermore, current output validation methods in autonomous driving primarily focus on open-loop trajectory prediction, which is ill-suited for the closed-loop nature of motion planning. We introduce REDOUBT, the first systematic safety validation framework for autonomous vehicle motion planning that employs a duo mechanism, simultaneously inspecting input distributions and output uncertainty. REDOUBT identifies previously overlooked unsafe modes arising from the interplay of In-Distribution/Out-of-Distribution (OOD) scenarios and certain/uncertain planning decisions. We develop specialized solutions for both OOD detection via latent flow matching and decision uncertainty estimation via an energy-based approach. Our extensive experiments demonstrate that both modules outperform existing approaches, under both open-loop and closed-loop evaluation settings. Our codes are available at: https://github.com/sgNicola/Redoubt.

REDOUBT: Duo Safety Validation for Autonomous Vehicle Motion Planning
Shuguang Wang'
Qian Zhou' Kui Wu² Dapeng Wu Wei-Bin Lee Jianping Wang'
'City University of Hong Kong, Hong Kong, China
University of Victoria, B.C., Canada
Inforrmtion Security Center, Hon Hai Research Institute, Taipei, Taiwan
sgwang6-c0my cityu. edu. hk
fqiazhou, jianvang)@cityu.. edu. edu. h& wkui@uvic.cm
dpwu@leee org wei-bin wew..eeeeefoooon...mmcm com
Abstract
Safety validation, which assesses the safety of an autonomous system's 's motion planning decisions, is critical for the safe deployment of autonomous vehicles. Existing imput validation techniques from other machine learning domains, such as image classification, face unique challenges in motion planning due to its contex- tual properties, including complex inputs and one-to-many mapping. Furthermore, current output validation methods in autonomous driving primarily focus on open- loop trajectory prediction, which is ill-suited for the closed-loop nature of motion planning. We introduce REDOUBT, the first systematic safety validation frame- work for autonomous vehicle motion planning that employs duo mechanism, simultaneously inspecting input distributions and output uncertainty. REDOUBT identifies previously overlooked unsafe modes arising from the interplay of In- Distribution/Outtof-D- -f-istri-ittiinnonnnn (OOD) scenarios and certain/uncertain planning decisions. We develop specialized solutions for both OOD detection via latent flow matching and decision uncertainty estimation via an energy-based approach. Our extensive experiments demonstrate that both modules outperform existing approaches, under both open-loop and closed-loop evaluation settings. Our codes are available /httt:: btttps///githin com www.////Niolllaaammomoomoooaaaaa Reedoutt
Introduction
Motion planning the process of generating trajectory for the ego- vehicll based on sensed traffic scenario data-is is cornerstone of autonomous driving. Learning-baser motion planning methods [48] 54 47 45 77 which leverage datasets of human driving behavior to train autonomous systems, have gained increasing attention due to their potential for greater adaptability compared traditional rule-based approaches. Despite significant advances, autonomous vehicles continue to struggle with the full complexity of real-world driving scenarios [1] 28 42).
Safety validation for motion planning, which assesses whether a vehicle's decisions are safe in given scenario, is thus critical. When an unsafe decision is detected, the validation module can: (I) immediately override the decision and initiate fallback measures (e.g., manual takeover [I]) to ensure short-term safety, and (2) flag the scenario as "challenging" to prioritize improvement in subsequent continual learning iterations 37 41. Safety validation could reduce or prevent tragedies like the March 29, 2025, Xiaomi SU7 incident, which resulted in three fatalities
[12]. Nevertheless, such a systematic safety validation framework is currently lacking-a gap our this work seeks to address.
Safety validation can potentially be approached via two main angles: input inspection and output inspection. In the first category, input inspection, we draw inspiration from Out-of-Distribunne Distribution
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(OOD) detection in other machine learning fields, such as image classification 49 3 to establish a novel approach for assessing safety levels autonomous driving. This approach determines whether given input falls within the trained model's s familiar data distribution: it is termed In- o--Disrribution (InD) if it does, and otherwise Out-of-Distribution (OOD) [5.. our context, if a traffic scenario identified as OOD, the system could downgrade the safety score of the resulting decision. However, input inspection strategies based on OOD detection have not yet been used in motion planning, and adapting them poses unique challenges. Unlike classification tasks, motion planning requires predicting distributions over plausible future trajectories conditioned on complex, context-dependent inputs [52]. This one-to-many mapping obscures the underlying data distribution, making it difficult to define meaningful feature distances or confidence margins for OOD detection.
In the second category, output inspection, safety validation assesses the uncertainty of the autonomous system's 's output and deems those with high uncertainty as unsafe. However, these approaches are mainly designed for trajectory prediction rather than motion planning [65 In nutshell, trajectory prediction predicts the future paths of other road agents, e.g., vehicles, pedestrians, and cyclists, based on their current/past behavior and environmental context, while motion planning determines the ego vehicle's s own trajectory to navigate safely and efficiently toward a goal, considering predicted trajectories of others. Safety validation for trajectory prediction faces significant barriers when applied to motion planning. This is because trajectory prediction is typically validated in an open-loop setting (39 (6). where predicted trajectories of surrounding agents are compared against ground-trute truth logs from recorded datasets. In contrast, motion planning operates in a closed-loop manner: the ego-vehicle continuously uses feedback from its current state and the dynamic environment to update and adjust its plans in real time 17]. Consequently, applying existing output inspection approaches directly to motion planning could lead to critical failures, such falsely certifying the ego vehicle's trajectory as safe when it actually results in collisions or road departures [10].
We propose REDOUBT, the first framework to systematically integrate input inspection through OOD detection and output inspection via uncertainty estimation for the safety validation of autonomous vehicle motion planning.
First, we develop specialized solutions for OOD scenario detection and uncertain decision estimation within the context of autonomous vehicle motion planning. We identify distributional shifts by estimating distribution likelihoods in the latent space using flow matching. Specifically, we employ a velocity field, enhanced with Gaussian Fourier Projection to capture temporal dynamics, to match target distributions and enable a principled estimation of distributional shifts in trajectory evolution. For estimating planning decision uncertainty under closed-loop dynamics, we introduce energy- based approach that assigns risk-sensitive energy scores. These scores, computed based on future distances to obstacles or surrounding agents, provide a dense and continuous measure of safety risk, unlike sparse binary violation labels. During inference, we aggregate energy scores and trajectory probabilities to assess safety risks under closed-loop execution.
Moreover, REDOUBT employs a duo validation srrroaccc--imultannouuii inspecting input distri- butions (traffic scenarios) and output uncertainty (planning decisions). The key insight behind this duo strategy, supported by our experiments, that input and output validation are complementary, not interchangeable. By jointly validating both dimensions, it categorizes driving situations into four types: (a) InD scenario + certain decision - Safe, (b) InD scenario + uncertain decision + Unsafe, (c) OOD scenario certain decision - Unsafe, and (d) OOD scenario + uncertain decision + Unsafe. Notably, REDOUBT uniquely identifies situations (b) and (c), which have been overlooked by existing approaches that rely solely on input output inspection, thereby offering more comprehensive safety guarantees. In summary, our contributions are:
We propose novel duo safety validation approach for autonomous driving motion planning that concurrently inspects both input and output, revealing two previously overlooked unsafe situations.
We design a latent flow matching method to estimate distributional shifts and detect OOD traffic scenarios, effectively modeling the multi-modal and dynamic nature of planning contexts.
We present an energy-based risk prediction approach that integrates risk signals and trajectory probabilities to estimate driving decision uncertainty without requiring explicit violation labels.
We outperform previous approaches on the nuPlan dataset in both OOD detection and decision uncertainty estimation. Our method demonstrates superior performance in both open-loop and closed-loop settings.
2

---

2 Related Work
Out-of-Distriuution (OOD) Detection. OOD detection has been extensively studied in image classification [22 [53] 50] but remains underexplored in motion planning. Post-hoc methods such as Maximum Softmax Probability [9]. MaxLoggi [18]. and energy-basee scoring [34] derive OOD scores from the output layer of trained classifiers, offering simplicity and ease of implementation. Feature-space methods, including Mahalanobis distance [26] and k-Nearest Neighbor (KNN) ap- proaches 40), estimate distributional deviation based on distances in the learned representation space. However, applying these methods to motion planning is poaillnnnigg-os--teeee scores often yield overconfident predictions, while feature-space methods struggle with the one-to-many mapping and complex inputs, which obscure the data distribution.
Motion Planning. Motion planning aims to generate safe and efficient trajectories for autonomous agents. Hybrid methods, such as GC-PGP [10], integrate learning- saaningg-bsedd models with rule- based priors through centerline selection algorithms, achieving balance between adaptability and safety. Fully bearning-based models, including GameFormer [22 and PlanTTF (8. leverage scene encoding to learn decision-making policies, generate multi-modal trajectory candidates, and select optimal plans based on trajectory probabilities and constraint satisfaction. Methods such as PLUTO 7 address distribution shift via contrastive learning, using data augmentations to shape latent representations. PlanScope [46] incorporates auxiliary losses based on Euclidean Signed Distance Fields (ESDF) to model cost and enforce safety constraints during planning. While these approaches have significantly advanced the performance of learning-based motion planning, they primarily overlook OOD detection.
Uncertainty Estimation. Uncertainty estimation in motion prediction has emerged tool for safety validation of trajectory forecasts [36. For example, Joodu 44 uses imitation error proxy for uncertainty, while Filos et al. [13 leverage disagreement among predicted trajectories to detect potential distribution shifts. DECODE [27) employs normalizing flows for domain awareness and selects the model with the highest likelihood to generate trajectory predictions. SOTIF [38] measures perceptual uncertainty with entropy, providing signals that guide risk evaluation in downstream motion prediction. Other works incorporate uncertainty arising from map inaccuracies and occlusions into trajectory [rediction[6]], and utilize trajectory distribution entropy to quantify predictive uncertainty (35.. Despite their utility, these uncertainty estimation approaches remain largely restricted to open-loop settings, relying on surrogate metrics such as imitation loss. These metries fail to capture critical closed-loop loop risks including collisions and road departures.
valuable
3 Methodology
This section introduces the REDOUBT framework, which consists of two key modules (dashed boxes in Figure (1) an
(OOD) Detection module based on Flow Matching and (2) a Decision Uncertainty Estimation module. REDOUBT is orthogonal to and can be integrated with various existing motion planning solutions to validate the safety of their planned trajectories.
3.1 Learning-based Motion Planning
To begin, we outline the background of modern ML-based motion planning models. These models leverage scene encoders to transform multi-modal driving contexts into high-dimensional latent representations 23 77 which encapsulate semantic and contextual information essential for trajectory Formally, the driving context is encoded as a scene representation = (Eg, EA. Eo. EMs); ) where Eg denotes the ego vehicle state, EA the states of dynamic agents, Eo static object features, and EM map information. The model predicts future trajectories for dynamic agents, denoted as y-1:7/ {v1 1:T,
prediction 225
24
33
[viTT ..... UNa 1:T,) Furthermore, generates a set of Nm, multi-modal
ego trajectories, represented Y,I.T/ {1o. I:T, I:m where p; the probability associated with the i-th candidate trajectory. The final trajectory -" is selected by optimizing p; while adhering to traffic-related constraints. Building on this, we propose a latent-space- framework for OOD detection and decision uncertainty estimation.
3

---

Uncertainty
Scene
Uncortainty
CollisionOOf-rron
Estimation
Imitation
Scene Encoder
Scene
Trajectory Decoder
Encoding
Prior Distribution
Latent Distribution
Log-Likke Lig-Liiklihood
Up
Velocity
Field
Flow Matching
OOD Score
Figure 1: Overview of the proposed REDOUBT framework. The OOD Detection module utilizes Flow Matching to model the In in-Distribution density of the latent space, learning velocity field transform the prior known distribution into the target distribution. The Decision Uncertainty Estimation module predicts collision and off-road violations, as well as imitation errors. The final decision-making process considers both the OOD score and the decision uncertainty.
3.2 OOD Detection based on Flow Matching
The OOD Detection module is designed to identify "unfamiliar" input scenarios and trigger signal of potential danger. Similar to OOD detection approaches in other domains (e.g., [15] 53)..
can be described as employing a score function to determine whether given input belongs to In-Distribution (InD) or Out-of-Distribution (OOD). We derive this score function from advancements in the generative AI field, applying it for safety validation. The core intuition is that generative models create score to evaluate the similarity between the current distribution and the target distribution, guiding their path toward the target. Leveraging this concept in novel way, we use the score to measure an input traffic scenario's proximity to the InD density.
Preliminary on Flow Matching. Flow Matching (FM) is a recently proposed paradigm for training generative models by transforming a simple source distribution into target data distribution 30 31 9. The aim of FM is to learn time-dependent flow o, that continuously maps samples from known source distribution, typically standard Gaussian, the target distribution [2]. This flow d, [0, 1] x IRd + R is defined by an ordinary differential equation (ODE) parameterized by velocity field uy:
dx) (x)))xx)).
di
The time-dependent probability path (Pt)osesr which smoothly interpolates between the source distribution po and the target distribution P1 A velocity field u, is suggested to generate this probability path. To learn the velocity field, a neural network uf used to approximate the target velocity field u, The parameters the neural network are typically estimated by minimizing the following objective function:
C ( = Es- ||4, (x) u, 2x))
(2)
where (x) represents the target velocity field derived from the continuity equation.
Velocity Field. Let z E Rd represent the latent encoding of a motion planning trajectory obtained from the scene encoder. We define time-dependent flow o (z). governed by neural velocity field u.. which evolves along interpolation path from known prior distribution Po to the target data distribution Pt. To incorporate the temporal dimension, we employ a Gaussian Fourier Projection to embed the scalar time t into high-dimensional feature vector. The time embedding is given by:
(t) = [sin(2 cos(,tt))) W gd//2
(3)
where W a projection matrix that encodes multi-frequency information. This time embedding then processed by residual Multi-Layer Perceptron (MLP) architecture.

---

The velocity field is trained using a velocity matching loss, which minimizes the discrepancy between the predicted flow velocity ug (z,) and the velocity di given by:
da along the interpolation path. The loss function
(
dzt dt
Lnow = Eo, (..0.. \m)00
(4)
where 2, is sampled along an interpolation path between 20 and 21, with velocity das along the path. Here, T defines the joint distribution of Zo and Z1, and the interpolation is guided by the Affine Probability Path under a Conditional Optimal Transport schedule [14].
di
Computing Log-Likelihood. Once the velocity field is trained, the log-likelihood of sample 21 ~ P1 can be computed by solving the flow ODE backward in time using an Euler solver. For the prior distribution Po. adopt Gaussian Mixture Model (GMM) with Nm, components to represent the multi-modal latent space. Using the Instantaneous Change of Variables formula [6]. the log-likelihood of the target distribution is expressed
log pi (zi) = log
N
Ni ne o,11)
-
S/" v. dl,
(5)
N
1=1
where (izo M4; I) represents the i-th Gaussian component of the prior distribution, characterized by mean μe and isotropic covariance matrix I. The GMM contains N,. components, each with an equal mixture weight 1/N.. The term, Ju' ug (z:) dt, is the divergence integral, which quantifies the density evolution as the sample flows from Z1 to Zu under the velocity field ull (2.).
3.3 Decision Uncertainty Estimation
The Decision Uncertainty Estimation module is a multi-output network designed to predict various uncertainty metrics for planned trajectories, encompassing both open-loop and closed-loop measures. First, in open-loop settings, uncertainty is represented by imitation error, often quantified using Average Displacement Error (ADE): ADE ,, ΣI' llve de ll2. where 1/2 is the ground truth trajectory, and ût the predicted trajectory. Second, in closed-loop dynamics, uncertainty is indicated by two metrics: 1) No-Fault Collisions (NFC), which refer to collisions that could have been avoided with proper planning, and 2) Drivable Area Compliance (DAC), which assesses whether the ego vehicle operates within the mapped drivable area. Safety violations are represented as Boolean values (0: no violation, 1: violation). In the absence of violation labels during closed-loop no-feedback training, we propose geometry-driver Violation Energy Score pseudo-labels 43] Leveraging signed distance as a physically intuitive metric, it transforms trajectories into continuous risk signals, capturing finer-grained risk features and addressing data scarcity in collision and driving area violation scenarios. The Violation Energy Score is further combined with trajectory probability to estimate decision uncertainty.
Violation Energy Score, At first, we construct the cost map using the Euclidean Signed Distance Field(ESDF) according to prior work [7,54] Non-drivable regions, such as obstacles and off-road areas, are rasterized into binary mask. distance transform is applied to compute signed distance field, encoding the shortest distance of each pixel to the nearest obstacle or boundary. The vehicle approximated as set of N covering circles, each with a predefined radius R., Each point in the trajectory {y' [1= 1,...,,7 Try } is projected onto the ESDF, where bilinear interpolation is used to calculate the signed distance de between the i-th covering circle at time and the nearest semantic region. A penalty is applied when df < Re. enforcing adherence drivable area constraints.
The violation energy for given trajectory is defined as:
Tj N ΣI M sp(d))
E T H=
(6)
where Ty is the trajectory length, N is the number of covering circles, M is a binary mask indicating whether the i-th circle at time intersects a region of interest (e-g., obstacles or boundaries), and (p(d) is penalty function quantifying the risk associated with the signed distance d;.
5

---

To address different safety constraints, we define two penalty functions: (ate(d)) for No at-fault Collisions (NFC) and ( for Drivable Area Compliance (DAC). The NFC penalty is defined as:
Pate (d:) max(0, exp -d:) 1),
which grows exponentially as d; approaches or falls below Re. This emphasizes the critical im- portance of avoiding overlaps with obstacles. When dī Re, the penalty is zero, indicating safe distance. This exponential design ensures heightened sensitivity near obstacles, particularly in dynamic or high-risk scenarios.
The DAC penalty is defined as:
Pduc (d.) max(0, Rs -d;),
(8)
which increases linearly d approaches the boundary (d² < Re). This formulation encourages smooth corrective actions and avoids ver-pennlizzing minor deviations. For dž 2 Re, the penalty is zero, indicating compliance.
Training Objective. The (T) is transformed into a soft binary indicator by comparing to a threshold The module predicts this signal using likelihood prediction head, which outputs the logits (T). These logits are optimized using the Binary Cross-Entrops Loss with Logits, defined as:
C
Σ BCEWithLogiis
e)).
(afe
where (T) > E] is the threshold indicator for each safety objective (k E {nfe, dac}). and Eg (T) is the Violation Energy Score associated with the respective constraint. The total loss function combines the constraint loss L. with an imitation loss Limiattions which predicts the imitation error of the driven trajectory. The total loss is given by:
Liorll Limitation + C. = lle él2 Lc:
(10)
where e is the ground-truth imitation error, and é is the predicted imitation emor. During inference, the module predicts Enfe o (T). the estimated probability of collision violation, and Edac = (f (7)), the estimated probability of drivable area compliance violation. Additionally, the predicted imitation error è is output to estimate trajectory imitation performance.
3.4 Ensemble with Trajectory Probability
As mentioned in Section 3.1 motion planning generates Nm candidate trajectories y-TTT
{1o. .p,)) where p: denotes the probability of the i-th candidate trajectory. The final tra- jectory 7" selected by maximizing Pt, subject traffic and safety constraints. To align with the decision-making, planner's 's decision- making, we combine the modulc-predicee
score with trajectory probabilities. Instead of directly using softmax probabilities, according (20, we adopt an energy-base score Ep to reflect the planner's 's confidence in its planned trajectory r" The InD score quantifies a scenario's similarity with the training distribution and is defined as:
E,p log exp(p:), EIAD Alogpi (1 A)
(11)
where log (2,) represents the normalized log- log-likelihood of the scene context from Flow Matching. high E, indicates the planner is confident in its distribution of candidate trajectories.
For decision uncertainty estimation,
Enfe XEnfe + A), Edoc XE + (1 X)
(12)
where Enfe and Edac are the predicted collision and compliance risks. Normalization ensures that all risk terms and E. are on comparable scale. The hyperparameter À E [0, 1] controls the relative weighting of the predicted violation risks versus the trajectory's confidence.

---

4
Experiments
We first outline our experimental settings. Next, we present the quantitative results for the OOD Detection module and the Decision Uncertainty Estimation module, respectively, followed by ablation studies. These results demonstrate that each module significantly outperforms existing methods, offering more reliable safety validation. Finally, we provide qualitative demonstrations showing that the duo validation approach REDOUBT, based on these two modules, effectively categorizes driving situations into four types. Notably, uncovers two unsafe situations overlooked by existing works, achieving more comprehensive safety guarantees for autonomous vehicles.
4.1 Experimental Setups
Dataset. We conduct our evaluations using nuPlan 4) which is currently the most widely used and the only large-scale dataset for motion planning evaluation. This dataset has 73 unique and well-labeled scenario types, defined by low-level driving attributes. We partition the dataset based on scenario frequency, following a strategy similar to that used in Shifts [36]. Specifically, the top 50% most frequent scenario types- types-which account for 95% of the total data-are categorized as the In- Distribution (InD) set, while the remaining, less common scenarios constitute the (OOD) set. For more details, please refer to Appendix I
Our evaluations are carried out across three planning evaluation modes defined by the nuPlan [4] (I) Open-loop (OP): The evaluation is conducted using log replay. Notably, open-loop tests have limited value for motion planning, which continuously interacts with environments. However, they remain relevant in scenarios with mainly static obstacles, so we include them for evaluation completeness. (2) Closed-loop Non-Reactive (CNR): Both the ego vehicle and other agents can deviate from their original trajectories, but the agents are non-reactive. (3) Closed-loop Reactive (CR): Both the ego vehicle and other agents can deviate, and the agents are reactive to the ego vehicle's 's behavior.
Planners. Our REDOUBT framework can be universally applied to various motion planning solutions to validate the safety of their planning results. To evaluate its effectiveness and model- agnostic nature, we integrate REDOUBT into the latent spaces of four diverse learning-based motion planners introduced in Related Work 2 PlanTTF, PLUTO, GameFormer. and PlanScope. For implementation details, please refer Appendix A.2
Baselines. REDOUBT is evaluated on: (1) OOD Detection and (2) Decision Uncertainty Estimation.
For OOD Detection, we compare against post hoc methods (MSP [19], MaxLogit [18), Entropy [3}
Energy S and feature-space methods (Mahalanobis Distance (MDS) (26. KNN 40 IGMM
For decision uncertainty estimation, we evaluate in Closed-loop (CNR and CR) and Open-loop (OP)
settings. Closed-loop metrics include (1) No At-Fault Collisions (NFC), assessing whether the ego vehicle avoids fault-based collisions, and (2) Drivable Area Compliance (DAC), checking trajectory adherence to drivable areas. the OP setting, we compute Average Displacement Error (ADE) within a bounded region. NFC and DAC are compared with the Energy score method, while ADE is evaluated against Shifts 36 baselines: Model Averaging (MA) and VAR [3
Evaluation Metrics. We evaluate OOD detection using: (1) the false positive rate (FPR95) of OOD samples when the true positive rate of InD samples 95%, and (2) the area under the receiver operating characteristic curve (AUROC). For decision uncertainty estimation, we consider: (1) the false positive rate (FPR95) of (No At-Fault Dollision)((rivving Area Compliance) samples when the true positive rate of violation samples is 95%, and (2) the AUROC. For imitation error, we use the area under the error-retention curve (R-AUC), which tracks trajectory imitation error as ground-truth labels progressively replace predictions in descending order of predicted error.
4.2 Evaluation of OOD Detection
Our method consistently outperforms existing OOD detection approaches across all three evaluation modes. As shown in Table which compares OOD detection methods trained with the same four motion planning models, our approach achieves notable improvements both FPR95 reduction and AUROC increase in OP, CNR, and CR modes. These advancements stem from the effective density estimation of InD features. Moreover, our method demonstrates strong performance across various motion planners, highlighting its versatility and reliability under diverse evaluation conditions.
7

---

Table 1: Comparison with OOD detection methods trained with the four motion planning models across three evaluation modes on nuPlan dataset. indicates larger values are better and vice versa. The best result each column shown in bolded. OP: Open-loop evaluation mode. CNR: Closed- loop non-reactive evaluation mode. CR: Closed-loop reactive evaluation mode.
Mode
Method
PlanTF
PLUTO
GameFormer
PlanScope
Average
FPR95
AUROC
AUROC1
FPR95
AUROCT
FPR95
AUROC t
MSP
88.92
64.29
79.70
74.15
91.16
62.13
83.85
71.37
85.91
67.99
MasLogit
69.04
54.08
Entropy
88.96
58.77
71.56
62.64
96.77
50.89
84.24
66.87
85.38
59.79
OP
Energy
MDS
67.77
75.93
65.17
76.50
$7.43
55,56
73.74
74.17
73.53
70.54
KNN
89.31
66.87
71.05
79.03
62.66
80.24
84.44
76.70
76.87
75.71
IGMM
94.54
65.81
66.30
79.51
87.28
70.72
72.38
79.87
80.13
73.98
REDOUBT(OOrs)
$1.02
84.48
53.54
83,50
29,59
84.82
56.18
85.83
$5.08
85.16
MSP
91.22
59.50
80.94
73.56
91.60
$9.65 65
84.23
70.68
85
MaxLogit
77.79
69.68
66.84
78.24
92.81
$2.71
76.53
74.61
78.49
68.81
Entropy
92.21
55.83
75.17
62.79
95.90 96
$1.39
82.07
65.65 65
86, 35
58.91
CNR
Energy
76.75
MDS
92.39
65.45
63.49
$1.50
68.74
71.68
69.39
80.09
73.50
74.68
KNN
90.56
66.05
74.99
78.44
65.80
77.58
75.85
78.
76.80
05
JGMM
88.77
67.10
76.51
77,83
68.88
77.26
67.89
79:93
75.51
75.53
REDOUBT(OurS)
65.11
81.41
57.11
84,83
66.35
S0.22
50,10
85.54
59.67
83,00
MSP
88.03
58.26
84.19
69.20
94, 94
$1.94
85.99
65 52
88: 29
61.23
MaxLogit
81.79
64.98
71.93
72.83
91.77
58.47
74,96
70.67
80.11
66.74
Entropy
93.29
51.17
75.17
53.35
CR
Energy
81.25
65
71.38
70.36
93.84
$4.08
73.25
71.25
79.93
65.26
MDS
90.84
71.47
70.81
74.65
73.63
KNN
90.01
66.14
73.23
78.66
67.17
77.01
67.00
80.82
74.35
75.66
IGMM
86.31
71.74
74.41
77.37
69.33
76.63
73.34
76.96
75.85
75.68
REDOUBT(OUr))
Figure 2: Retention curves of uncertainty estimation in open-loop evaluation across different planners. The ADE is plotted over the retention fraction. Lower R-AUC values indicate better performance. The oracle retention curve (green) is obtained by sorting samples in descending order of true ADE.
(a) PIanTF
(b) PLUTO
(c) GameFormer
(d) PlanScope
4.3
Evaluation of Decision Uncertainty Estimation
Closed-loop Evaluation. Table 2 presents the evaluation results uncertainty estimation, including No At-fault Collision (NFC) and Driving Area Compliance (DAC) under the closed- loop non-reactive and closed-loop reactive modes. Our method demonstrates superior performance across all four motion planners. For example, for predicting NFC violations, method improves the average AUROC by 12.81% % and 10.03% in the CNR mode and the CR mode, respectively.
Table Comparison of NFC and DAC estimation results, trained with the four motion planning models and evaluated under the CNR and CR modes nuPlan dataset.
Mode
Uncertainty
Method
GameFormer
PlantScape
Average
FPRO5
AUROCC
FPR95
AUIRCC
FPROS J AUROC
FPRO5
AUROCC
FPRUS ] AUROCT
Energy
79.33
63.10
89.52
56.36
91.63
52.93
73.40
57.30
83.47
NPC
70.93
70.45
71.62
56.85
74.44
$7.30
74.41
63.28
72.73
CNR
Energy
76.88
59.01
76.70
95.38
51.19
69.36
66.21
79,58
59.93
DAC
REDOUBTOOurs)
$5.13
$2.33
$2.97
77.33
52.36
75.38
55.94
75.00
$4.10
77.51
Energy
$2.70 74
64.39
82:27
71.58
87.88
59.62
80.84
67.24
83.43
65.71
NPC
OOuS)
67.04
71.97
$8.29
78.11
$7.34
73.37
49.22
79.51
$7.97
75.74
Energy
73.70
DAC
REDOUBTIOurS)
52.24
$4.03
58.59
76.90
51.63
76.90
48.67
80.00
$2.78
79.46
Opon Evaluation. The retention curves of uncertainty estimation under the open-loop evaluation mode, as shown in Figure 2 demonstrate that our method consistently performs closer to the Oracle retention compared to the best basclines across multiple planners. For example, in the PlanTF planner,

---

our method achieves an ADE R-AUC of 0.263, which is closer to the Oracle value of 0.239, compared the best baseline (0.282). This indicates that our method exhibits better uncertainty calibration, as it effectively aligns uncertainty estimates with the true ADE.
4.4 Evaluation of Duo Safety Validation
We evaluate the full REDOUBT system under closed-loop reactive setting, jointly using OOD detection and decision uncertainty estimation. We report recall as the primary metric, measur- ing coverage of potential unsafe scenarios. Here, potential unsafe scenarios comprise three cate- gories: Inl) + Uncertain (InD inputs with driving violations), OOD + Uncertain (OOD inputs with driving violations), and OOD + Certain (OOD inputs without driving violations). Further qual- itative analysis of unsafe scenarios is provided
Table 3: Duo safety validation under closed-loop reactive mode: recall across OOD detection and decision uncertainty estimation
Method
PlunTF
PLUTO
GameFormer
PlanScope
Avg.
KIMM
49.95
Fnergy
59.52
72.61
REDOUBT
90.24
90.52
91.34
90.02
90.53
Category
Decomposition of Recall by Sceaario Category
0.70
1.76
1.79
0.87
1.16
OOD+Certin
10.04
15.61
10.50
6,78
10.73
GOD+Uncettin
79,50
73.65
79.05
82.37
78.64
Section 4.6
REDOUBT attains 90.53% recall, substantially outperforming strong baselines- 55.29% for driving uncertainty estimation and 65.73% for OOD detection alone. This indicates that the unified design captures a broader set of unsafe cases.
We further decompose recall by category. OOD Uncertain constitutes the majority (78.64%). InD + Uncertain accounts for 1.16%; despite its small share, these cases are safety critical. OOD + Certain contributes additional 10.73%, reflecting OOD situations that may not immediately cause violations but remain hazardous under distribution shift. These results underscore the value of unifying OOD detection with decision uncertainty estimation for safety validation.
4.5 Ablation Studies
We conduct ablation experiments to evaluate the importance of the component design in method.
Impact of Trajectory Probability Energy. We analyze the impact of incorporating the trajec- tory probability energy (Ep) on OOD detection and NFC and DAC estimation. Table 4 sum-
Table 4: Effects of Trajectory Probability Energy Ep on OOD detection and Uncertainty Estimation.
Metric
Method
marizes the average evaluation results without (w/)) and with E, (REDOUBT). Notably, the inclusion of E,, significantly enhances OOD de- tection under all modes and NFC and DAC es- timation under the CNR and CR modes. These findings demonstrate Ep 's effectivvness in im- proving OOD detection and uncertainty estima- tion, as it is enhanced by encoding trajectory probability information. Additionally, we ana- lyze the sensitivity of the ensemble weight X, which balances the contributions of flow match- ing and E, in Appendix B.4
63.47
56.15
w/S,
57.97
AUROCT
w/e.
80
77.51
75.02
78.58
Table 5: Effects of GFP module and GMM Prior.
Mode
Module
FPR95
AUROC 1
who GFP
61.67
82.88
OP
w/o GMM
61.11
84.42
REDOUBT
55.08
85,16
w/o GFP
62.67
82.15
CNR
w/o GMM
61.68
82.79
REDOUBT
59.67
83,00
w/o GFP
63.68
80.87
Impact of Gaussian Fourier Projection. Ta- ble 5 demonstrates the impact of the Gaussian
CR
w/o GMM
70.33
75.46
REDOUBT
61.48
82.05
Fourier Projection (GFP) on OOD detection performance across three modes (OP, CNR, and CR). GFP, which embeds the scalar time into high-dimeessonnl feature vector, consistently improves AUROC scores. These results prove the effectiveness of GFP in capturing temporal features for better OOD detection. Further details provided in Appendix B.2
The ablation study JableeJexaminee the impact of using a Gaussian Mixture Model (GMM) prior in flow matching, compared to Standard Normal distribution. The results consistently demonstrate that the GMM prior enhances Out-of-Distributioon (OOD) detection performance across all modes. Further details provided in Appendix B.3
Impact of Gaussian Mixture Model Prior.

---

4.6 Demonstration of Duo Safety Validation
REDOUBT simultaneously inspects input distributions and output uncertainty, categorizing driving situations into four combinations based on the interplay InD/OOD scenarios and certain/uncertain planning decisions. See Figure for examples of each category.
00:0
(a) InD + Certain (Safc)
(b) InD + Uncertain (Unsafe)
(c) OOD + Certain (Unsafe)
(d) OOD Uncertain (Unsafe)
Figure 3: Safety categorization in REDOUBT: (a) InD + Certain (Safe, e.g., straight intersection traversal); (b) InD + Uncertain (Unsafe, e.g., high-density left turns with trained-but-falled scenarios); (c) OOD + Certain (Unsafe, e-g., overconfident unprotected cross-turns); (d) OOD Uncertain (Unsafe, e.g., exiting drivable areas at drop-off locations). The scenarios illustrated include: Orange lines for the planned trajectory of the motion planner, Blue lines for the predicted trajectories of agents, and Gray lines for the candidate trajectories by the planner.
REDOUBT identifies two previously overlooked unsafe modes and illustrates them with represen- tative failures. First, the InD + Uncertain scenario is overlooked by an input-only inspection that treats all InD as safe. Even in familiar scenarios, inherent complexity can induce uncertain decisions and unsafe outcomes. In Case (b), the High-Density Traffic during Left Turns scenario involves left-turn maneuver in high-density traffic environment, which common challenge motion planners. Although this category of scenario was encountered during training, observed that the planned trajectory by PlanScope planner and the candidate trajectories still resulted collision with surrounding vehicles. This failure highlights the complexity of high-density interactions where subtle variations can lead to collisions despite prior training exposure.
Second, the OOD + Certain scenario is overlooked by output-only inspection that treats all "certain" decisions as safe. Rare scenarios can induce overeonfident decisions that remain hazardous under shift. In Case (c), the Unprotected Cross- Turns scenario involves unprotected cross-turns, a category of scene that was not observed during training. Despite the absence of such scenarios the training data, the planner still needs to output decisions. In Case (d), the Traversing Pickup/Drop-Ont Areas scenario involves the ego vehicle traversing a drop-off area while not stopping. Such scenarios were also not present in the training data, and observed that the planned trajectory drifted outside the drivable area. This failure illustrates the planner's 's limitations in handling complex, context-specific scenarios that require precise spatial reasoning and awareness.
5 Conclusion
In this work, we propose REDOUBT, a duo safety validation framework for autonomous vehicle motion planning that addresses two critical aspects: OOD detection and decision uncertainty esti- mation. Our approach introduces a latent flow matching method to model distributional shifts and detect OOD traffic scenarios, effectively capturing the planning contexts' multi-modal and dynamic nature. Additionally, we design an energy- risk prediction method that integrates trajectory probabilities and risk signals to estimate decision uncertainty without relying explicit violation la- bels. Experiments and ablation studies on the nuPlan dataset validate the effectiveness and versatility of REDOUBT, demonstrating its ability to identify previously overlooked unsafe situations.
Acknowledgement
The work supported in part by Hong Kong Research Grant Council under CRF C1042-23GF, GRF 1 11216323, and GRF 1 11214925.
10

---

References
[1] Pasquale Antonante, Sushant Veer, Karen Leung, Xinshuo Weng, Luca Carlone, and Marco Pavone. Task-aware risk estimation of perception failures for autonomous vehicles. arXiv preprint arXiv:2305.01870 2023.
[2] Heli Ben-Hamu, Samuel Cohen, Joey Bose, Brandon Amos, Aditya Grover, Maximilian Nickel, Ricky TQ Chen, and Yaron Lipman. Matching normalizing flows and probability paths on manifolds. arXiv preprint arXiv:2207.04711 2022.
[3] Julian Bitterwolf, Maximilian Mueller, and Matthias Hein. In or out? fixing imagenet out-of- distribution detection evaluation. In ICML, 2023.
[4] Holger Caesar, Juraj Kabzan, Kok Seang Tan, Whye Kit Fong, Eric Wolff, Alex Lang, Luke Fletcher, Oscar Beijbom, and Sammy Omari. nuplan: closed-loop ml-based planning benchmark for autonomous vehicles. arXiv preprint arXiv:2106. 11810, 2021.
[5] Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, and Hongyang Li. End-to-end autonomous driving: Challenges and frontiers. IEEE Transactions on Putlern Analysis and Machine Intelligence, 2024.
[6] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
[7] Jie Cheng, Yingbing Chen, and Qifeng Chen. Pluto: Pushing the limit of imitation learning- based planning for autonomous driving. arXiv preprint arXiv:2404. 200.22444.4344 14327, 2024.
[8] Jie Cheng, Yingbing Chen, Xiaodong Mei, Bowen Yang, Bo Li, and Ming Liu. Rethinking imitation-based planner for autonomous driving, 2023.
[9] Quan Dao, Hao Phung, Binh Nguyen, and Anh Tran. Flow matching in latent space. arXiv preprint 2.t.:333000869 2023.
[10] Daniel Dauner, Marcel Hallgarten, Andreas Geiger, and Kashyap Chitta. Parting with mis- conceptions about learning-based vehicle motion planning. In Conference Robot Learning (CoRL), 2023.
[11] Christopher Dichl, Peter Karkus, Sushant Veer, Marco Pavone, and Torsten Bertram. Lord: Adapting differentiable driving policies to distribution shifts. In International Conference on Robotics and Automation, 2025.
[12] Coco Feng. Xiaomi founder has 'the hardest time' since fatal crash involving one of its cars. South China Morning Post, 2025. Published: May 10, 2025.
[13] Angelos Filos, Panagiotis Tigas, Rowan McAllister, Nicholas Rhinehart, Sergey Levine, and Yarin Gal. Can autonomous vehicles identify, recover from, and adapt to distribution shifts? In International Conference on Machine Learning (ICML), 2020.
[14] Tor Fjelde, Emile Mathieu, and Vincent Dutordoir. An introduction to flow matching, January 2024.
[15] Soumya Suvra Ghosal, Yiyou Sun, and Yixuan Li. How to overcome curse-of- -imee-f--iimensionnalty for out-of-distribution detection? In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 19849-19857, 2024.
[16] Xunjiang Gu, Guanyu Song, Igor Gilitschenski, Marco Pavone, and Boris Ivanovic. Producing and leveraging online map uncertainty in trajectory prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14521-14530, June 2024.
[17] K. Tan et al. H. Caesar, Kabzan. Nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles. In CVPR ADP3 workshop. 2021.

---

[18] Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Joe Kwon, Mobammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distribution detection for real- world settings. arXiv preprint arXiv: arXiv: 2019.
[19] Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and -uttof-distribution examples in neural networks. Proceedings of International Conference on Learning Represen- lations, 2017.
[20] Claus Hofmann, Simon Schmid, Bernhard Lehner, Daniel Klotz, and Sepp Hochreiter. Energy- based hopfield boosting for out-of-distribution detection. arXiv preprint arXh::2405.08770 2024.
[21] Hong Huang and Dapeng Wu. Quaff: Quantized parameter-efficien fine-tuning under outlier spatial stability hypothesis, 2025.
[22] Zhiyu Huang, Haochen Liu, and Chen Lv. Gameformer: Game-theoretic modeling and learning of transformer- bbssed interactive prediction and planning for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 3903-3913, October 2023.
[23] Xiaosong Jia, Penghao Wu, Chen, Yu Liu, Hongyang Li, and Junchi Yan. Hdgt: Heteroge- neous driving graph transformer for multi-agent trajectory prediction via scene encoding. IEEE transactions on pattern analysis and machine intelligence, 45((1))
2023.
[24] Bo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao, Jiajie Chen, Helong Zhou, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Vad: Vectorized scene representation for efficient autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8340-8350, 2023.
[25] Zhiqian Lan, Yuxuan Jiang, Yao Mu, Chen Chen, and Shengbo Eben Li. Sept: Towards efficient scene representation learning for motion prediction. arXiv preprint arXiv:2309. 15289, 2023.
[26] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. simple unified framework for detecting dusto--distr samples and adversarial attacks. Advances in neural information processing systems, 31, 2018.
[27] Boqi Li, Haojie Zhu, and Henry X. Liu. Decode: Domain-aware continual domain expansion for motion prediction, 2024.
[28] Quanyi Li, Zhenghao Peng, Lan Feng, Zhizheng Liu, Chenda Duan, Wenjie Mo, and Bolei Zhou. Scenarionet: Open-source platform for large-scale traffic scenario simulation and modeling. Advances in Neural Information Processing Systems, 2023.
[29] Yiming Li, Zehong Wang, Yue Wang, Zhiding Yu, Zan Gojeic, Marco Pavone, Chen Feng, and Jose M. Alvarez. Memorize what matters: Emergent scene decomposition from multitraverse. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[30] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.
[31] Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky TQ Chen, David Lopez-Paz, Paz, Heli Ben-Hamu, and Itai Gat. Flow matching guide and code. arXiv preprint arXiv:2412.00
2024.
[32] Litian Liu and Yao Qin. Fast decision boundary based outoo--distributinn detector. ICML, 2024.
[33] Mengmeng Liu, Hao Cheng, Lin Chen, Hellward Broszio, Jiangtao Li, Runjiang Zhao, Monika Sester, and Michael Ying Yang. Laformer: Trajectory prediction for autonomous driving with lane-aware scene constraints. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2039-2049, 2024.
[34] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based dut-o---istoibuttio detection. Advances in Neural Information Processing Systems, 2020.
12

---

[35] Juanwu Lu. Can Cui, Yunsheng Ma, Aniket Bera, and Ziran Wang. Quantifying uncertainty in motion prediction with variational bayesian mixture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15427, June 2024.
[36] Andrey Malinin, Neil Band, Alexander Ganshin, German Chesnokov, Yarin Gal, Mark F. Gales, Alexey Noskov, Andrey Ploskonosov, Liudmila Prokhorenkova, Ivan Provilkov, Vat- sal Raina, Vyas Raina, Denis Roginskiy, Mariya Shmatova, Panos Tigar, and Boris Yangel. Shifts: A dataset of real distributional shift across multiple large-scale tasks. arXiv preprint arXiv:2107.07455. 2021.
[37] Jianbiao Mei, Yukai Ma, Xuemeng Yang, Licheng Wen, Xinyu Cai, Xin Li, Daocheng Fu, Bo Zhang, Pinlong Cai, Min Dou, et al. Continuously learning, adapting, and improving: A dracess approach to autonomous driving. arXiv preprint arXiv:2405.15324 2024.
[38] Liang Peng, Boqi Li, Wenhao Yu, Kai Yang, Wenbo Shao, and Hong Wang. Sotif entropy: Online sotif risk quantification and mitigation for autonomous driving. IEEE Transactions on Intelligent Transportation Systems, 25(2):1530-15555 1546000-56,6000 2024.
[39] Shaoshuai Shi, Li Jiang, Dengxin Dai, and Bernt Schiele. Motion transformer with global inten- tion localization and local movement refinement. Advances in Neural Information Processing Systems, 2022.
[40] Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. ICML, 2022.
[41] Eli Verwimp, Kuo Yang, Sarah Parisot, Lanqing Hong, Steven McDonagh, Eduardo Pérez- Pellitero, Matthias De Lange, and Tinne Tuytelaars. Clad: A realistic continual learning benchmark for autonomous driving. Neural Networks, 001::599669, 2023.
[42] Yun Wang, Junjie Hu, Junhui Hou, Chenghao Zhang, Renwei Yang, and Dapeng Oliver Wu. Rose: Robust self-supervised stereo matching under adverse weather conditions. arXiv preprint arXiv:2509. 19165, 2025.
[43] Yun Wang, Jiahao Zheng, Chenghao Zhang, Zhanjie Zhang, Kunhong Li, Yongjian Zhang, and Junjie Hu. Dualnet: Robust self-supervised stereo matching with pseudo-label supervision. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 8178-8186. 2025.
[44] Julian Wiederer, Julian Schmidt, Ulrich Kressel, Klaus Dietmayer, and Vasileios Belagiannis. Joint out-of-distribution detection and uncertainty estimation for trajectory prediction. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5487-5494. IEEE, 2023.
[45] Wei Wu, Xiaoxin Feng, Ziyan Gao, and Yuheng Kan. Smart: Scalable multi-agent real-time simulation via next-token prediction. arXiv preprint 200:22405.5677., 2024.
[46] Ren Xin, Jie Cheng, and Jun Ma. PlanScope: learning to plan within decision scope does matter, 2024.
[47] Ren Xin, Hongji Liu, Yingbing Chen, Jie Cheng. Sheng Wang, Jun Ma, and Ming Liu. A generic trajectory planning method for constrained -tleetig-stterring robots. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2024.
[48] Zebin Xing, Xingyu Zhang, Yang Hu, Bo Jiang, Tong He, Qian Zhang, Xiaoxiao Long, and Wei Yin. Goalflow: Goal-driven flow matching for multimodal trajectories generation in end-to-end autonomous driving. arXiv preprint arXiv:2503.05663 2025.
[49] Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey. arXiv preprint arXiv:21 10. 2021.
[50] Boxuan Zhang, Jianing Zhu, Zengmao Wang, Tongliang Liu, Bo Du, and Bo Han. What if the input is expanded in ood detection? Advances in Neural Information Processing Systems, 01221229-21333 2024.
13

---

[51] Jingyang Zhang, Jingkang Yang, Pengyun Wang, Haoqi Wang, Yueqian Lin, Haoran Zhang, Yiyou Sun, Xuefeng Du, Yixuan Li, Ziwei Liu, Yiran Chen, and Hai Li. Openood v1.5: Enhanced benchmark for out-of-distribution detection. arXiv preprint arXiv:2306.09301, 20233.
[52] Junrui Zhang, Chenjje Wang, Jie Peng, Haoyu Li, Jianmin Ji, Yu Zhang, and Yanyong Zhang. Cafe-ad: Cross-scenario adaptive feature enhancement for trajectory planning in autonomous driving. arXiv preprint arXiv:2504 065544 2025.
[53] Yonggang Zhang, Jie Lu, Bo Peng, Zhen Fang, and Yiu-ming Cheung. Learning shape in-distribution feature space for out-of-distribution detection. In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024.
[54] Yinan Zheng, Ruiming Liang, Kexin ZHENG, Jinliang Zheng, Liyuan Mao. Jianxiong Li, Weihao Gu, Rui Ai, Shengbo Eben Li, Xianyuan Zhan, and Jingjing Liu. Diffusion-base -ased planning for autonomous driving with flexible guidance. The Thirteenth International Conference on Learning Representations, 2025.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have clearly stated the main contributions made in the paper and important assumptions and limitations in both the Abstract and the Introduction (Section T
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
In Appendix Section D we have discussed the limitations of this work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
The paper does not include new theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced
statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Yes, the paper discloses all the necessary details about the implemented architectures used, hyperparameters for each setting and other experimental details to reproduce all the experiments in the paper. These details can be found in Section A in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
16

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
We will release the code upon acceptance of the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https /nips
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: In Section 4 we have briefly introduced the experimental setups about datasets, model setups, and evaluation metrics. Furthermore, we have provided full details of experi- mental settings in Appendix(A]
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: In the ablation study to evaluate the significance of each component design in our method, and the factors of variability are clearly stated in Section 4.5 In Appendix B we have provided the full results of this experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have provided sufficient information on the computer resources, such the type, amount, and memory of compute workers GPU required for the experiments in the reproducibility statement at the Appendix A.2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every respect, with the NeurlPS Code of Ethics, and the authors remain anonymous.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed the potential broader impact of this paper in Appendix
Guidelines:
18

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All used assets in the paper properly credited, and the datasets used in this paper are properly licensed.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
19

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.com//aaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
20

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: This research does not involve LLMs as any important, original, or non- standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurl)) (ttt::ee//earrrgg.ccc////feeeeer / for what should or should not be described.
21

---

A More Experimental Details
A.1 Training Settings
The training dataset contains total of 300,000 scenarios, including specific InD scenario types. The evaluation dataset consists of 19,685 scenarios, with fixed scenario tokens. For each scenario type (except for the "Unknown" scenario, which cannot be verified based on the scenario type), either 500 scenarios are included, or, fewer than 500 scenarios are available, all the available scenarios are used. All planners are trained exclusively on the InD dataset and evaluated on the full dataset, which includes both InD and OOD scenarios.
A.2 Implementation Details
Motion Planning Model. Our experiments are conducted on a server with 8 NVIDIA RTX 5880 Ada GPUs and the Pytorch platform. The training experiments consist of two phases. In the first phase, each planner is trained according to its original settings. Specifically, PlanTF is trained with a batch size of 128, a weight decay of 10-4, and for 25 epochs [8) The learning rate starts at 1 x 10-3 and decays to zero following cosine schedule. PLUTO is also trained with a batch size of 128 and a weight decay of 10 for 25 epochs [7]. Its learning rate is linearly increased to 10 over
the first 3 epochs and then follows a cosine decay. GameFormer is trained with batch size of 32, weight decay of 0.01, and for 20 epochs 22 The learning rate starts at 10-1 and halves every epochs after the 10th epoch. PlanScope is trained with a batch size of 32 per GPU for 25 epochs, with warm-up epochs, using the Adam optimizer and an initial learning rate of x 10 [46].
REDOUBT Model. In the second phase, the flow matching and uncertainty prediction modules are optimized separately. The flow matching module is trained for 2000 epochs using the Adam optimizer, a batch size of 1024, and a cosine learning rate schedule with an initial learning rate of 1 10 The uncertainty prediction module trained for 25 epochs, also using the Adam optimizer, a batch size of 1024, and a fixed learning rate of 1 x 10-3 30-3 During this phase, the weights of the scene encoder and trajectory prediction decoder are frozen to preserve motion planning performance.
B Additional Ablation Studies
Table 6: Detailed results of the ablation study on Trajectory Probability Energy Ep.
Evaluation
Mode
Method
PlanTF
PLUTO
GameFormer
PlanScope
Average
FPR95 1
AUROCT
FPR95 J AUROCT
FPR95 J AUROC
FPR95
AUROCT 1
FPR95 1 AUROC 1
wlo E, 62.20
82.19
62.60
$2.30
62.93
83.67
60.60
84.74
62.08
83.23
w/ Ep
51.02
OOD
CNR
W/o E, Ep
90.48
69.10
65.35
81.52
66.83
79.92
61.58
83.32
71.06
78.47
50.10
CR
wlo Ep w/ Ep
63.77
61.81
81.81
80.43
63.84
39,14
82,58
80.10
66.89
66.83
80.32
79.64
58.12
55.38
82.82
83.47
63:47
61.48
82,05
80.82
NYC
CNR
wlo E, 77477%
w/ Ep
70.93
69.01
70.45
71.68
68.04
71.62
68.11
56.85
76.62
74.44
62.96
$7.30
64.30
72.62
74,41
71.85
63.28
72.73
6% 18
CR
who E w/ Ep
67.04
73.31
71.97
69.55
72.90
58.29
78.11
76.77
57.34
59.12
71.08
73.37
72.34
49.22
71.09
79.51
$7.97
69.42
72.12
75.74
DAC
CNR
wio F.
w/ Ep
55.72
55.13
82.33
82,28
71.86
52.97
75.53
77.33
52.36
52.35
75.38
75.33
71.49
55.94
74.94
75.00
62.86
54.10
77.02
77.51
wlo 53.55
83.55
02
52.12
56.15
w/ Eg 52.24
84.03
$8.59
76.90
51.63
76.90
48.67
80.00
$2.78
79.46
B.1 Impact of Trajectory Probability Energy
Table compares performance with and without the inclusion of trajectory probability energy (Ep) across various models and evaluation scenarios. The results demonstrate that incorporating Ep consistently enhances performance across all metries (FPR95 and AUROC), as evidenced by lower FPR95 values and higher AUROC scores. The improvements are particularly significant in the OOD and NFC evaluations, where Fp significantly reduces FPR95 and increases AUROC, especially for models such PF and PlanScope.
22

---

Table 7: Detailed results the ablation study on Gaussian Fourier Projection (GFP).
PlanIF
PLUTO
GameFormer
PlanScope
Average
Mode
Module
FPR95 4
AUROCT
FPR95 І
AUROCT
FPR95
AUROC1
FPR95
AUROC
FPR95
AUROC
w/o GFP
69.34
82.23
56.42
83,20
65.32
80,30
58.58
85,77
61.67
82.88
OP
w/ GFP
51.02
84.48
53.54
85.50
59.59
84.82
56.18
85.83
55.08
85.16
CNR
w/o GFP
62.87
82.15
66.69
80.23
57.25
85,49
62.67
82.15
w/ GFP
65.11
81.41
57.11
84.83
66.35
80.22
50.10
85,54
59.67
83,00
w/o GFP
65.74
79.10
65.20
81.54
$6.85
80.72
66.94
82,13
63.68
80.87
CR
80.32
58.12
83.47
61.48
82.05
B.2
Impact of Gaussian Fourier Projection
Table 7 presents a comparison of OOD detection performance before and after removing the Gaussian Fourier Projection (GFP) in the velocity field module of flow matching. The results indicate that incorporating the GFP improves the overall OOD detection performance across all four planners.
Table 8: Detailed results of the ablation study Gaussian Mixture Model (GMM).
Mode
Module
PlanTF
PLUTO
GameFormer
PlanScope
Average
FPR95 L
AUROC
HPRY5 1 AUROCT
FPR95
AUROC
HPR95 | AUROCT
HPR95
AUROC
OP
w/o GMM
w/ GMM
51.02
62.05
84,48
82.55
53.54
63.21
85.73
85.50
59.59
62.66
84,82
83.64
56.18
56.53
85.83
85.76
$5.08
61.11
85,16
84.42
W/o GMM
67.56
81.08
55.72
85,31
62.76
80.17
60.66
84.59
61.68
82,79
w/ GMM
65.11
81.41
57.11
84.83
66.35
80.22
50.10
89.67
83.00
CR
71.38
73.97
62.80
80.36
71.70
w/ GMM
61.81
81.81
59.14
82.58
66.83
80.32
58.12
83.47
61.48
82.05
B.3
Impact of Gaussian Mixture Model Prior
Table S presents comparison of OOD detection performance before and after replacing the Gaussian Mixture Model (GMM) with a standard distribution. The results indicate that GMM prior distribution more effective.
Table 9: Inference time of REDOUBT on different motion planners.
Planner
PlanTF
PLUTO
GameFormer
PlanScope
Inf. Time
3.12 ms
3.30 ms
10.79 ms
3.25 ms
Figure 4: Ablation study on A: (a) OOD Detection (OP Mode), (b) NFC Estimation (CNR Mode),
and (c) DAC Estimation (CNR Mode).
OOD Detection (OP Mode)
(b) NFC Estimation (CNR Mode) (c) DAC Estimation (CNR Mode)
B.4 Hyperparameter Sensitivity
Figure contains three subfigures illustrating performance trends for OOD detection under the
OP mode, NFC estimation, and DAC estimation under the CNR mode. Each subfigure shows the
23

---

performance of various planners as À varies from 0 to 1, where À = 0 represents the configuration without Flow Matching, and À represents the configuration without E The results indicate that the optimal range for À lies between 0.5 and 0.8. For example, OOD detection with À = 0.8 achieves the optimal performance for GameFormer. The hyperparameter averages the strengths of both methods, avoiding the shortcomings of relying exclusively on one.
C Inference Time
Table illustrates the inference time REDOUBT on different motion planners. REDOUBT introduces additional inference time of only 3.12 to 10.79 ms. Considering that the original planner operates at 20 Hz (resulting a latency of 50 ms per frame), the extra latency from REDOUBT is minimal. Importantly, the total latency remains well below the maximum threshold of 1000 per inference mandated by the nuPlan benchmark [17]. These results demonstrate that REDOUBT is computationally efficient and capable of meeting the real-time performance requirements of modern motion planners.
D Limitations
We use nuPlan for validation because it is currently the only large-scale dataset specifically designed for motion planning, offering both open-loop and closed-loop evaluation settings. Most existing ML--based motion planners [2 46 are exclusively trained and tested on nuPlan due the lack of other comparable motion planning datasets. We recognize that further validation on additional datasets would enhance the generalizability of our framework, and we aim to include them in the future when new datasets become publicly available.
E Broader Impacts
This paper presents REDOUBT, a novel duo safety validation framework for motion planning in autonomous driving. Autonomous driving is safety-critical domain where rigorous safety validation is crucial to prevent potentially hazardous decisions. REDOUBT is the first systematic safety validation framework that employs a duo mechanism simultaneously inspect both input distributions and output uncertainty. This dual inspection approach reveals two previously overlooked unsafe scenarios arising from the interplay between InD/OOD inputs and certain/uncertain planning decisions.
To address these scenarios, propose two specialized solutions: (1) OOD detection using latent flow matching, and (2) decision uncertainty estimation via an energy-based approach. Extensive experiments demonstrate that both modules significantly outperform existing methods under both open-loop and closed-loop evaluation settings. By improving the safety and reliability of autonomous driving systems, REDOUBT offers broad applicability across industries, contributing safer and more dependable autonomous vehicles.
Notably, several leading autonomous driving companies, such as Momenta [5], have already adopted learning-basee planners in their systems. Our method well-suited for such deployments, as integrates seamlessly into the pipeline. Specifically, the input representation from the autonomous driving stack, and the output would include whether it represents an OOD scenario and risk probabilities of potential driving violations.
method would be the latent space
Several practical challenges need to be addressed for real-world deployment. Our approach could identify more unsafe scenarios than traditional methods. This raises the question of how to balance sensitivity and specificity in decision making (21). To address these challenges, we propose that the output of our method serves helpful, supplementary information to existing decision-making mechanisms within the stack. For instance, could integrate with existing systems to trigger takeover requests needed. Additionally, system could be optionally enabled to prioritize safety in specific scenarios, such as high-risk environments or areas prone to OOD events.
24


Reviews:
[{"review_id": "YQ0nTPAMS9", "reviewer": "Reviewer_GyUM", "summary": "This paper presents REDOUBT, a safety validation framework for autonomous vehicle motion planning that addresses gaps in current safety assessment approaches. The key innovation lies in its dual validation mechanism that simultaneously evaluates both input distributions (traffic scenarios) and output uncertainty (planning decisions). The framework tackles two challenges: Out-of-Distribution (OOD) detection for identifying unusual driving scenarios, and decision uncertainty estimation for assessing the reliability of planning outputs. REDOUBT employs latent flow matching for OOD detection and an energy-based method for uncertainty quantification based on safety risks from obstacle proximity. The authors demonstrate through comprehensive experiments on the nuPlan dataset that their approach significantly outperforms existing methods across multiple evaluation metrics, providing enhanced safety guarantees for autonomous driving systems.", "questions": "See the weaknesses.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The paper tries to address a critical and practically important problem in autonomous driving.\n- The proposed approach is technically sound and well-motivated.\n- Comprehensive experimental evaluation on the nuPlan dataset with comparison against existing methods shows clear performance improvements across multiple scenarios.\n\nWeaknesses\n- Limited discussion of computational overhead and real-time performance implications.\n- While the approach shows promise, the practical deployment challenges and integration with existing autonomous driving stacks are not thoroughly discussed.\n- More detailed analysis of failure cases would be valuable.\n- Only InD scenarios with certain decisions can be considered safe; such a condition may be too strict and could cause excessive takeovers in practical driving scenarios.", "was_revised": true, "final_justification": "Just keep my original rating.", "rebuttal": "Thank you for reviewing our paper. We appreciate your valuable feedback and will try to address your concerns below:\n\n**Q1: Discussion of computational overhead and real-time performance implications.**\n\n**A1:** Due to the page limit, we report the inference times of our method with different motion planners in Appendix C, Table 8. REDOUBT introduces an additional inference time of only 3.12 to 10.79 ms. Considering that the original planner operates at 20 Hz (resulting in a latency of 50 ms per frame), the extra latency from REDOUBT is minimal. Importantly, the total latency remains well below the maximum threshold of 1000 ms per inference mandated by the nuPlan benchmark. These results demonstrate that REDOUBT is computationally efficient and capable of meeting the real-time performance requirements of modern motion planners. Based on your feedback, we will expand the discussion in Appendix C to provide a more comprehensive analysis of inference times and their real-time applicability.\n\n**Q2: Discussion on practical deployment challenges and integration with autonomous driving stacks.**\n\n**A2:** Based on your feedback, we will expand our discussion to address our approach's practical deployment and integration with existing autonomous driving stacks as follows: Notably, several leading autonomous driving companies, such as Momenta [1], have already adopted learning-based planners in their systems. Our method is well-suited for such deployments, as it integrates seamlessly into the pipeline. Specifically, the input to our method would be the latent space representation from the autonomous driving stack, and the output would include whether it represents an out-of-distribution (OOD) scenario and risk probabilities of potential driving violations. Several practical challenges need to be addressed for real-world deployment. Determining takeover frequency: Our approach could identify more unsafe scenarios than traditional methods. This raises the question of how to balance sensitivity and specificity in decision-making. To address these challenges, we propose that the output of our method serves as helpful, supplementary information to existing decision-making mechanisms within the stack. For instance, it could integrate with existing systems to trigger takeover requests as needed. Additionally, our system could be optionally enabled to prioritize safety in specific scenarios, such as high-risk environments or areas prone to OOD events. \n\n[1] Lan, Gongjin, and Qi Hao. \"End-to-end planning of autonomous driving in industry and academia: 2022-2023.\" arXiv preprint 2023.\n\n**Q3: More detailed analysis of failure cases.**\n\n**A3:** To address this, we will add more detailed analysis in **Section 4.5** to include a comprehensive discussion of failure cases observed during our experiments as follows: We identified and analyzed several representative failure cases encountered in our experiments: **Case (b) High-Density Traffic during Left Turns:** This scenario involves a left-turn maneuver in a high-density traffic environment, which is a common challenge for motion planners. Although this category of scenario was encountered during training, we observed that the planned trajectory by PlanScope planner and the candidate trajectories still resulted in a collision with surrounding vehicles. This failure highlights the complexity of high-density interactions where subtle variations can lead to collisions despite prior training exposure. **Case (c) Unprotected Cross-Turns:** This scenario involves unprotected cross-turns, a category of scene that was not observed during training. Despite the absence of such scenarios in the training data, the planner still needs to output decisions. **Case (d) Traversing Pickup/Drop-Off Areas:** This scenario involves the ego vehicle traversing a drop-off area while not stopping. Such scenarios were also not present in the training data, and we observed that the planned trajectory drifted outside the drivable area. This failure illustrates the planner’s limitations in handling complex, context-specific scenarios that require precise spatial reasoning and awareness.\n\n**Q4: Strict InD conditions may lead to excessive takeovers in practical driving.**\n\n**A4:** We acknowledge that our approach MIGHT identify more scenarios as unsafe. However, it is worth noting that our false positive rate is slightly lower than that of existing methods, demonstrating stronger performance in avoiding the misidentification of safe scenarios as unsafe. Furthermore, the benefit of our research is justified as follows. First, our approach can benefit offline continual learning by providing a more comprehensive set of unsafe scenarios for model improvement. Second, in online driving, we do not advocate for automatic manual takeovers in every reported unsafe situation. Instead, the output of our method serves as supplementary information that supports other decision-making mechanisms in initiating takeover requests. It can also be optionally activated in scenarios that empirically warrant heightened caution—such as detecting a road construction flag or a school bus with children ahead. Compared to existing approaches that may overlook potentially unsafe situations, our caution-oriented method offers a safer alternative. After all, when human life is at stake, it is better to err on the side of caution."}, {"review_id": "c8SNDFevr9", "reviewer": "Reviewer_Kf2E", "summary": "This paper introduces REDOUBT, a novel duo safety validation framework for autonomous driving motion planning that jointly considers scenario distribution (via OOD detection) and output uncertainty (via planning confidence). The framework is validated across multiple learning-based motion planners on the nuPlan dataset under both open-loop and closed-loop settings and demonstrates consistent improvements over existing baselines.", "questions": "* The paper evaluates OOD detection and uncertainty estimation separately, but lacks evaluation for overall validation effectiveness. Can the authors discuss how to evaluate the performance of the whole system?\n* How can REDOUBT be applied in end-to-end autonomy systems that operate directly on raw sensor inputs", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- The duo validation perspective is insightful. This paper provides a compelling argument for jointly validating both input distribution and output uncertainty, highlighting overlooked unsafe cases that each dimension alone would miss (InD + Uncertain and OOD + Certain). This insight is practically meaningful for AV safety.\n- The use of flow matching for OOD likelihood estimation is interesting and shows consistent improvements over a wide range of post hoc and feature-space baselines.\n- The decision uncertainty estimation module works for both open-loop and closed-loop settings, and is evaluated across multiple motion planners and under three settings (OP/CNR/CR), with consistent performance gains.\n \n**Weaknesses**\n- The paper evaluates OOD detection and uncertainty estimation independently but does not quantify REDOUBT’s overall safety validation performance as a unified system. It would be valuable to define and report metrics that directly reflect end-to-end validation effectiveness.\n- The method assumes access to structured scenario inputs (e.g., object trajectories, maps), which may not be available in end-to-end autonomous systems that go directly from raw sensor input to planned trajectory. Can REDOUBT be adapted to operate in such sensor-to-plan pipelines?", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for reviewing our paper. We appreciate your valuable feedback and will try to address your concerns below:\n\n**Q1: Discussion on metrics and evaluation for REDOUBT’s unified end-to-end safety validation.**\n\n**A1:** Thank you for your feedback. Based on your comment, we present the validation performance of the entire REDOUBT system when OOD detection and uncertainty estimation are used together. Due to space constraints, we focus on the ‘recall’ metric, which measures how effectively the method covers all potential unsafe scenarios.\n\n| Recall           | PlanTF | PLUTO  | GameFormer | PlanScope | Average |\n| ---------------- | ------ | ------ | ---------- | --------- | ------- |\n| Uncertain (lGMM) | 0.5350 | 0.6588 | 0.5181     | 0.4995    | 0.5529  |\n| OOD (Energy)     | 0.5952 | 0.6582 | 0.6497     | 0.7261    | 0.6573  |\n| Ours             | 0.9024 | 0.9052 | 0.9134     | 0.9002    | 0.9053  |\n\nOur results show a significant increase in recall, rising from 0.5529 for SOTA uncertainty estimation in our baselines and 0.6573 for OOD detection, to 0.9053 with our approach. This demonstrates our method's effectiveness in identifying a broader range of unsafe scenarios.\n\nAdditionally, it is worth noting that our false positive rate is slightly lower than that of existing methods, demonstrating stronger performance in avoiding the misidentification of safe scenarios as unsafe. We will incorporate these results and discussions in the paper.\n\n**Q2: How can REDOUBT be applied in end-to-end autonomy systems that operate directly on raw sensor inputs?**\n\n**A2:** Thank you for your comment. Yes, REDOUBT applies to end-to-end sensor-to-plan pipelines. Inside most modern end-to-end architectures [1], raw sensor inputs (e.g., maps, cameras, lidars) are first encoded into a latent scene representation using neural networks, and then trajectory planning is conducted in this latent space. \n\n**Scenario inputs:** As illustrated in Figure 1, our system operates on the outputs of the scene encoder (i.e., the latent representation) rather than the raw inputs. This alignment makes it well-suited for end-to-end autonomous systems. Based on your feedback, we will refine Figure 1 and the accompanying text to further clarify this point.\n\n[1] Lan, Gongjin, and Qi Hao. \"End-to-end planning of autonomous driving in industry and academia: 2022-2023.\" arXiv preprint 2023."}, {"review_id": "oY5OV3HtEL", "reviewer": "Reviewer_g3mR", "summary": "The paper presents REDOUBT, a novel approach for self-driving ML planner’s OOD detection. In addition to the trajectory decoding head, it adds two head that predicts 1) uncertainty, supervised by heuristics on collision and offroads and 2) a flow-matching based OOD detector.", "questions": "- Given the OOD detection enabled by REDOUBT, how much better does the overall planning stack improve? Can the authors evaluate this with standard planning metrics? \n- It’s unclear to me why it has to be flow-matching which requires solving ODE during inference if the sole purpose is to score the trajectory prediction. For example, why not a conditional normalizing flow or VAE?\n- I’d like to see some case or qualitative analysis on the failure modes, e.g. false positives and negatives.\n- What are the additional computes/latency of REDOUBT to a planner stack?", "limitations": "The main limitations at the moment is the lack of closed-loop evaluation with planning metrics on the entire system. Also, no explicit discussion on the limitations.", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "### Strengths\n+ The paper is well written and tackles an important problem that is relatively under-explored.\n+ The proposed method is intuitive and easily integratable to arbitrary (ML)-planner stack.\n\n\n### Weaknesses\n- The statement in L57 “We propose REDOUBT, the first systematic safety validation framework for autonomous vehicle motion planning” felt overclaiming. There has been thorough prior work on self-driving safety validation [1,2]. I’d make sure the claim is more specific, e.g. clarify that the approach focuses on OOD detection.\n- My main concern is that there is no closed-loop evaluation on the entire stack – given the OOD detection enabled by REDOUBT, how much does the overall planning stack improve?\n\n[1] Determining Absence of Unreasonable Risk: Approval Guidelines for an Automated Driving System Deployment, Favaro et al.\n[2] Comparison of Waymo rider-only crash data to human benchmarks at 7.1 million miles, Kusano et al.", "was_revised": true, "final_justification": "After reading the authors' rebuttal, I think two major concerns remain:\n\n1) The design justification for using flow matching (A3) is a bit hand-wavy. A quantitative comparison would have been nice here.\n2) Motion planning from structured inputs is already a self-contained task, and if the method is evaluated without its full closed-loop metrics, I am afraid the scope of the paper becomes way too niche. In addition, without the full metrics the claim that \"the proposed method can be plugged in any ML planner\" is not sufficiently supported, because it's unclear how a ML planner work when it plugs in in the proposed method.\n\nBased on the above, I am unable to recommend acceptance at this point.", "rebuttal": "Thank you for reviewing our paper. We appreciate your valuable feedback and will try to address your concerns below:\n\n**Q1: Clarify the statement in L57 and make it more specific.**\n\n**A1:** Thank you for pointing this out. We agree that the statement in L57 could be more specific. In the revised version, we will reframe the claim as follows: *\"We propose REDOUBT, the first framework to systematically integrate input inspection through out-of-distribution (OOD) detection and output inspection via uncertainty estimation for the safety validation of autonomous vehicle motion planning.\"*\n\n**Q2: The closed-loop evaluation of the planning stack and how the OOD detection enabled by REDOUBT improves overall planning performance?** \n\n**A2:** We appreciate the reviewer’s insightful question regarding the closed-loop evaluation of the entire planning stack and how the OOD detection enabled by **REDOUBT** improves overall planning performance. We would like to **clarify the scope of our work while acknowledging its limitations and potential future directions**.\n\n**1) Scope of REDOUBT:** The primary focus of REDOUBT is safety validation, which assesses whether the vehicle's driving decisions are safe in specific scenarios. Once an unsafe condition is detected, subsequent response strategies (e.g., manual takeover to ensure safety or continual learning for model improvement) will be applied; however, they are beyond the scope of this work. Addressing response strategies involves tackling substantial and complex problems that merit their own paper and cannot be integrated into this safety validation work within a single 9-page paper. We intend to explore these strategies in future work.\n\n**2) Why Closed-Loop Evaluation of Overall Planning Stack Is Ill-Suited for Our Work:** As mentioned, this work concentrates solely on safety validation. The degree to which it can enhance overall planning performance also depends on the employed response strategies, such as manual takeover and continual learning. This involves tackling complex challenges like distribution shifts, designing robust learning strategies, and closed-loop training, which are outside the scope of this paper. Therefore, there is currently no \"entire stack\" available, making it infeasible to measure overall improvements.\n\nWithin the scope of safety validation, we believe our evaluation is comprehensive and sufficient. It is worth noting that related works, such as those focused on OOD detection [1] or corner case detection [2], also confine their evaluation scope at a similar level.\n\n[1] Yang, Jingkang, et al. \"Generalized out-of-distribution detection: A survey.\" IJCV 2024\n\n[2] Zhang, Xinhai, et al. \"Finding critical scenarios for automated driving systems: A systematic mapping study.\" IEEE TSE 2022\n\n**Q3: Why choose flow-matching for inference instead of a conditional normalizing flow or VAE?**\n\n**A3: 1) Continuous-Time Trajectory Modeling:** Flow Matching is specifically designed for modeling continuous-time stochastic processes. By formulating the transformation as an ODE, Flow Matching can naturally represent the fine-grained, continuous dynamics of real-world trajectories.\n\n**2) Comparison with Conditional Normalizing Flows (CNF):** While CNFs are powerful, they require computation of the Jacobian determinant during training and inference, which becomes costly as data dimensionality grows—a common scenario in trajectory prediction. In contrast, Flow Matching sidesteps this bottleneck by not requiring Jacobian computations, making it more scalable and efficient for high-dimensional, complex trajectories.\n\n**3) Limitations of VAEs:** VAEs model the latent space by assuming a simple prior distribution (usually a standard Gaussian). This assumption can limit their ability to accurately capture the full diversity and complexity present in real-world trajectory data, often resulting in less accurate or less diverse predictions.\n\n**Q4: I’d like to see some case or qualitative analysis on the failure modes, e.g. false positives and negatives.**\n\n**A4:** Section 4.5 presents trajectory planning scenarios of PlanScope recorded during closed-loop reactive simulation experiments. We will provide a more detailed analysis, particularly for the failure cases shown in Fig. 3b and 3d as follows: \n\nWe identified and analyzed several representative failure cases encountered in our experiments: **Case (b) High-Density Traffic during Left Turns.** This scenario involves a left-turn maneuver in a high-density traffic environment, which is a common challenge for motion planners. Although this category of scenario was encountered during training, we observed that the planned trajectory by PlanScope planner and the candidate trajectories still resulted in a collision with surrounding vehicles. This failure highlights the complexity of high-density interactions where subtle variations can lead to collisions despite prior training exposure. **Case (c) Unprotected Cross-Turns.** This scenario involves unprotected cross-turns, a category of scene that was not observed during training. Despite the absence of such scenarios in the training data, the planner still needs to output decisions. **Case (d) Traversing Pickup/Drop-Off Areas.** This scenario involves the ego vehicle traversing a drop-off area while not stopping. Such scenarios were also not present in the training data, and we observed that the planned trajectory drifted outside the drivable area. This failure illustrates the planner’s limitations in handling complex, context-specific scenarios that require precise spatial reasoning and awareness.\n\n**Q5: What are the additional computes/latency of REDOUBT to a planner stack?**\n\n**A5:** Due to page limitations, we present the inference times of our method as applied to various motion planners in Appendix C, Table 8. REDOUBT introduces an additional inference time of only 3.12 to 10.79 ms. Considering that the original planner operates at 20 Hz (resulting in a latency of 50 ms per frame), the extra latency from REDOUBT is minimal. Importantly, the total latency remains well below the maximum threshold of 1000 ms per inference mandated by the nuPlan benchmark. These results demonstrate that REDOUBT is computationally efficient and capable of meeting the real-time performance requirements of modern motion planners. Based on your feedback, we will expand the discussion in Appendix C to provide a more comprehensive analysis of inference times and their real-time applicability."}, {"review_id": "Y4Ja4cF7St", "reviewer": "Reviewer_pm9A", "summary": "This paper introduces REDOUBT, a duo safety validation framework for motion planning task in autonomous driving. REDOUBT features more robust safety validation, through concurrent scenario OOD detection via latent flow matching, and decision uncertainty estimation quantified by ESDF and modal probability. REDOUBT is validated through multiple planners in both open and closed loop settings, proving effectiveness across other OOD detection methods.", "questions": "1.  How is the difficulty of a scenario quantitatively defined or evaluated? Can the method distinguish between hard-but-frequent vs. rare-but-trivial cases?\n\n2. How does the proposed identification mechanism concretely benefit downstream planning performance, especially in closed-loop execution?\n\n3. Since the separation of training and evaluation sets is manually based on frequency, is there a risk of implicit leakage or circular reasoning when evaluating OOD detection? How sensitive are the results to the chosen threshold of frequency?\n\n4. To what extent do the detected OOD cases actually correlate with safety-critical planner failures? Could some OOD regions be benign or irrelevant to planning outcomes?\n\n5. Is there an ablation showing the importance of each verification module (e.g., flow-based, energy-based)? Could a simpler or more modular combination of components achieve similar results?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n1. The motivation for safety validation in autonomous driving is well-articulated and compelling.\n\n2. The paper presents extensive comparisons with OOD detection baselines in both open-loop and closed-loop settings. REDOUBT demonstrates effectiveness across multiple widely adopted planning algorithms.\n\n**Weaknesses**\n\n1. The originality of each verification module is somewhat limited. Prior works such as [1] and [2] have also explored latent flow matching and energy-based methods for domain identification and uncertainty quantification.\n\n2. Despite a thorough introduction and comprehensive experimental comparisons with OOD detection methods, the paper lacks deeper insights into how the proposed identification mechanism concretely improves or mitigates failures in open-/closed-loop planning.\n\n3. Since the train/evaluation split is manually separated based on frequency, it is unclear whether the method can truly identify hard-but-frequent edge cases.\n\n4. The qualiative results are not intuitative enough. More supplementary like closed-loop identification are expected. No code are provided for verification\n\n5. How is each planner trained? Is there poential risk for data leakage?\n\n[1] DECODE: Domain-aware Continual Domain Expansion for Motion Prediction\n\n[2] SOTIF Entropy: Online SOTIF Risk Quantification and Mitigation for Autonomous Driving", "was_revised": true, "final_justification": "After a detailed analysis through the rebuttal, the reviewer is convinced by the authors regarding the novelty in designing the safety validation framework. Still, its current evaluation is limited (as also noted by other reviewers) without close-loop planners. Hence, the reviewer is inclined to a borderline score.", "rebuttal": "**Q1:The originality of the verification modules compared to prior works.**\n\n**A1:** Thanks for pointing out DECODE and SOTIF Entropy. We will incorporate a discussion into the **Related Work** section. Below, we outline the key differences:\n\n**1) Differences from DECODE：A. Flow Matching vs. Normalizing Flow:** While DECODE employs **Normalizing Flow (NF)**, we utilize **Flow Matching (FM)**. NF relies on maximum likelihood training of invertible transformations and requires computing the Jacobian determinant, a computationally expensive step. In contrast, FM learns a vector field that defines the flow between distributions and avoids the computation of the Jacobian determinant, making it more efficient and scalable for our task. **B. Domain Identification Objective:** DECODE uses NF for domain-awareness, aiming to select the model with the highest likelihood score for making predictions. Our work, however, focuses on **OOD validation** to verify the **inputs** to motion planners.\n\n**2) Differences from SOTIF Entropy:** The entropy and energy formulas approach probability distributions in different ways. The entropy formula used in SOTIF quantifies uncertainty by evaluating the uniformity of a probability distribution. Entropy increases when probabilities are evenly distributed, indicating higher uncertainty. This makes it an effective metric for evaluating perceptual uncertainty, as it measures the uniformity of the probabilities of detected objects across categories.\n\nIn our work, the energy formula (Eq.(11) in the paper) focuses on the concentration of probabilities by amplifying the influence of high-probability candidates through the use of exponential (exp) and logarithmic (log) operations. The energy formula is particularly sensitive to concentrated probability distributions where a few probabilities dominate, with higher energy values reflecting stronger confidence in the most probable outcomes. This property makes it well-suited for tasks like trajectory planning, where identifying and emphasizing the most likely trajectory is critical.\n\n**3) Our Contributions:** Our framework uniquely integrates **input validation** and **output validation** for motion planners, a design absent in both DECODE and SOTIF Entropy.\n\n**Q2: How does the identification mechanism benefit downstream planning performance, especially in closed-loop execution?**\n\n**A2:** We clarify the scope of our work while acknowledging its limitations and potential future directions.\n\n**1) Scope of REDOUBT:** REDOUBT focuses on assessing whether the vehicle's driving decisions are safe in specific scenarios. Once an unsafe condition is detected, subsequent response strategies (e.g., manual takeover or continual learning) will be applied; however, they are beyond the scope of this work. Addressing response strategies involves tackling complex problems that merit their own paper and cannot be integrated into this safety validation work within a single 9-page paper.\n\n**2) Downstream Planning Performance Improvement:** Our work concentrates on safety validation. The degree to which it can enhance downstream or overall planning performance also depends on the response strategies, such as manual takeover and continual learning. This involves tackling complex challenges like distribution shifts, designing robust learning strategies, and closed-loop training, which are outside the scope of this paper. Thus, there is currently no \"entire stack\" available, making it infeasible to measure downstream or overall improvements.\n\nWithin the scope of safety validation, our evaluation is comprehensive and sufficient. It is worth noting that related works, such as those focused on OOD detection (Ref.[43] in the paper) or corner case detection [1], also confine the evaluation scope at a similar level.\n\n[1] Zhang, Xinhai, et al. \"Finding critical scenarios for automated driving systems: A systematic mapping study.\" IEEE TSE 2022\n\n**Q3: How to define scenario difficulty? Distinguish hard-frequent vs.rare-trivial? Correlation between OOD cases and safety-critical failures?** \n\n**A3:** First, we follow the widely accepted NuPlan benchmark, defining a scenario as difficult if the planner's decisions: 1) result in at-fault collisions that could have been avoided, or 2) violate drivable area compliance.\n\nSecondly, our system identifies both hard-but-frequent and rare-but-'trivial' cases. Below is a table illustrating our overall performance and the breakdown of different cases. Due to space limit, we focus on the ‘recall’ metric, which measures how effectively the method covers all potential unsafe scenarios.\n\n||PlanTF|PLUTO| GameFormer|PlanScope|Average|\n|-------|-----|------|----|---------|-----|\n|InD-Uncertain (frequent-but-hard)|0.0070|0.0126|0.0179|0.0087|0.0116|\n|OOD-Certain (rare-but-'trivial')|0.1004|0.1561|0.1050|0.0678|0.1073|\n|OOD-Uncertain (rare-and-hard)|0.7950|0.7365|0.7905|0.8237|0.7864|\n|Total|0.9024|0.9052|0.9134|0.9002|0.9053|\n\nOur overall recall of unsafe scenarios averages 0.9053. We consider three cases as unsafe: 1) OOD-Uncertain (rare-and-hard), commonly deemed unsafe by other works, covers 78.64% unsafe scenarios; 2) InD-Uncertain (frequent-but-hard) covers 1.16% unsafe scenarios. Although this may seem small, identifying such scenarios is crucial for human and property safety in real life; 3) our solution includes OOD-Certain (rare-but-'trivial') as unsafe, covers 10.73% additional unsafe scenarios. While uncertainty estimation approaches may consider this case trivial due to its high certainty, we argue that it is significant, hence our designation of rare-but-'trivial' (not really trivial).\n\nAdditionally, our false positive rate is slightly lower than that of existing methods, demonstrating stronger performance in avoiding the misidentification of safe scenarios as unsafe.\n\nIn summary, we consider all three cases as unsafe, and our evaluation demonstrates that each contributes to the detection of unsafe scenarios; none are unnecessary. We will include these results and discussions in the paper.\n\n**Q4: Make qualitative results more intuitive and provide code.**\n\n**A4: 1) Qualitative Results:** We will enhance the illustrations by adding a legend to make the scenarios more intuitive. We will add a legend to differentiate the elements in each scene: Orange lines for the planned trajectory of the motion planner, Blue-green lines for the trajectories of surrounding agents, and Gray lines for the candidate trajectories by the planner.\n\n**2) Detailed Analysis:** We will add a detailed analysis in **Section 4.5** to include a comprehensive discussion of failure cases observed during our experiments. **Case (b) High-Density Traffic during Left Turns:** This scenario involves a **left-turn maneuver** in a **high-density traffic environment**, which is a common challenge for motion planners. Although this category of scenario was encountered during training, we observed that the planned trajectory and the candidate trajectories still resulted in a collision with surrounding vehicles. This failure highlights the complexity of high-density interactions where subtle variations can lead to collisions despite prior training exposure. **Case (c) Unprotected Cross-Turns:** This scenario involves **unprotected cross-turns**, a category of scene that was **not observed during training**. Despite the absence of such scenarios in the training data, the planner still needs to output decisions. **Case (d) Traversing Pickup/Drop-Off Areas:** This scenario involves the **ego vehicle traversing a drop-off area** while not stopping. Such scenarios were also **not present in the training data**, and we observed that the planned trajectory drifted outside the drivable area. This failure illustrates the planner’s limitations in handling complex, context-specific scenarios that require precise spatial reasoning and awareness.\n\n**3) Code:** In the NeurIPS Submission Guidelines, open access to data and code is encouraged but not mandatory. We will release the code upon acceptance of the paper.\n\n**Q5: How is each planner trained? Are there risks of data leakage? How sensitive are results to frequency threshold?**\n\n**A5: 1) Training Details:** We provide detailed information in Appendix A.1 and A.2. To ensure consistency, all planners are trained using the original parameter settings prescribed by their respective papers or implementations.\n\n**2) Data Leakage:** There is NO data leakage in our setup. Our training and evaluation sets are derived from the nuPlan dataset's official splits, selected based on scenario frequency and type. Each scenario is assigned a unique token ID, ensuring no overlap between splits. Furthermore, the nuPlan dataset is designed to prevent leakage by not sharing data from the same day or city across splits [1].\n\n**3) Threshold of Frequency:** We chose the threshold for scenario frequency to balance the coverage of scenario types, employing a strategy similar to that used in Shifts (Ref.[33] in the paper). Specifically, we included the top 50% most frequent scenario types, which represent 95% of the total data. This threshold ensures broad coverage of diverse scenarios while maintaining a meaningful representation of the dataset. \n\n[1] Karnchanachari, Napat, et al. \"Towards learning-based planning: The nuplan benchmark for real-world autonomous driving.\" 2024 IEEE ICRA\n\n**Q6: Ablation of verification module.**\n\n**A6:** We present this ablation in Appendix B.4. We conducted an ablation study by adjusting the hyperparameter λ (ranging from 0 to 1): When λ = 0, it represents the complete removal of the Flow Matching module. When λ = 1, it represents the complete removal of the energy-based module. The experimental results demonstrate that utilizing either module alone (λ = 0 or λ = 1) does not achieve optimal performance. This demonstrates that both are critical for performance improvement."}]

Author rebuttal:

Meta-review: This paper received ratings of 4442. The authors propose a new safety validation framework for autonomous vehicle motion planning that integrates out-of-distribution detection and uncertainty estimation. The reviewers agree that the authors are addressing a critical issue in autonomous driving and have conducted thorough experimental validation across various planners. The rebuttal has addressed computational overhead concerns and practical deployment challenges by expanding discussions and providing additional analyses in the rebuttal. 

During the discussion period, there were still some unresolved issues regarding the use of flow matching and the interpretation of closed-loop evaluation from Reviewer g3mR. The AC has gone through the details and believes that it is reasonable to use flow matching based on the author's argument. The authors are still suggested to investigate using VAE or CNF as the baselines and provide some comparisons, but the AC does not think this is a critical matter. Another issue is the closed-loop evaluation. The AC understands the concerns of the Reviewer g3mR because the experiments are not conducted in a full-stack closed-loop manner, but the closed-loop metrics used in the submission are commonly used in the benchmark nuPlan, which is a closed-loop ML-based planning benchmark for autonomous vehicles. The AC suggests that the authors provide more information about these metrics.

Overall, the paper proposes a practical and systematic safety validation framework for autonomous driving, and the AC believes the work can be accepted for NeurIPS 2025.
Decision: Accept (poster)