Interpretable Self-Aware Neural Networks for Robust Trajectory Prediction
Authors: Masha Itkina
Mykel Kochenderfer
Venue: CoRL 2022 Poster (2022)
Keywords: autonomous vehicles
trajectory prediction
out-of-distribution detection
distribution shift
epistemic uncertainty estimation

Abstract: Although neural networks have seen tremendous success as predictive models in a variety of domains, they can be overly confident in their predictions on out-of-distribution (OOD) data. To be viable for safety-critical applications, like autonomous vehicles, neural networks must accurately estimate their epistemic or model uncertainty, achieving a level of system self-awareness. Techniques for epistemic uncertainty quantification often require OOD data during training or multiple neural network forward passes during inference. These approaches may not be suitable for real-time performance on high-dimensional inputs. Furthermore, existing methods lack interpretability of the estimated uncertainty, which limits their usefulness both to engineers for further system development and to downstream modules in the autonomy stack. We propose the use of evidential deep learning to estimate the epistemic uncertainty over a low-dimensional, interpretable latent space in a trajectory prediction setting. We introduce an interpretable paradigm for trajectory prediction that distributes the uncertainty among the semantic concepts: past agent behavior, road structure, and social context. We validate our approach on real-world autonomous driving data, demonstrating superior performance over state-of-the-art baselines.

Interpretable Self-Aware Neural Networks for Robust Trajectory Prediction
Masha Itkina and Mykel J. Kochenderfer
Department of Aeronautics and A Astronautics, Stanford University {mi (mitkina, myke1 mniikina,mykel] @stanford.
Abstract: Although neural networks have seen tremendous success as predictive models in variety of domains, they can be overly confident in their predictions on out-of-distribution (OOD) data. To be viable for safety-critical applications, like autonomous vehicles, neural networks must accurately estimate their epistemic or model uncertainty, achieving a level of system self-awareness. Techniques for epistemic uncertainty quantification often require OOD data during training multiple neural network forward passes during inference. These approaches may not be suitable for real-time performance on high-dimensional inputs. Further- more, existing methods lack interpretability of the estimated uncertainty, which limits their usefulness both to engineers for further system development and downstream modules in the autonomy stack. We propose the use of evidential deep learning to estimate the epistemic uncertainty over a low-dimensional, in- terpretable latent space in trajectory prediction setting. We introduce an inter- pretable paradigm for trajectory prediction that distributes the uncertainty among the semantic concepts: past agent behavior, road structure, and social context. We validate our approach on real-world autonomous driving data, demonstrat- ing superior performance over state-of-the-art baselines. Our code is available at: bttpp://githhe cwm/cco///////cc cammm/sss//mnni http://aaaaatepeeeeaaaaassaaaaaa
Keywords: Autonomous vehicles, trajectory prediction, distribution shift
1
Introduction
Deep learning techniques have had success across multitude of domains, including human trajec- tory prediction in the context of autonomous vehicles (AVs) [1, 2]. However, a key challenge in deploying these systems is their lack of self-awareness about the quality of their predictions. Deep learning models often overestimate their confidence in unfamiliar situations [3-9]. For robots de- ployed in human environments, this over-confidence could result in dangerous maneuvers and safety concerns. Flagging unfamiliar situations encountered in the real world could be helpful to down- stream autonomy stack components, such as path planners to execute a fail-safe mancuver, and to engineers for further system development.
Real-world systems are subject to aleatoric and epistemic uncertainty. The former is irreducible data uncertainty, which, for trajectory prediction, could be represented as a distribution over future trajectories given past observations (e.g.. turning or continuing straight given a straight past trajec- tory). Aleatoric uncertainty can be modelled explicitly during training of learning- based systems, for example, using variational approaches [10-16]. Epistemic uncertainty reflects what the model does not know, which can arise due to the limited ability of model to represent data and data distribution shift. Neural networks often fail to correctly calibrate this uncertainty, resulting in un- reliable predictions for out-of-distributis (OOD) inputs [3-9]. In this paper, focus epistemic uncertainty quantification for trajectory prediction.
There has been growing interest in estimating epistemic uncertainty for deep learning models [17]. Most existing methods consider small benchmark datasets or require OOD data for training [18- 23]. A few recent papers (e.g., [24-31]) have began exploring epistemic uncertainty estimation for learning-base based robot perception and prediction. Identifying OOD inputs for tasks like trajectory prediction is difficult due to the high dimensionality of the data. Since it is impossible to foresee
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

Encader Agent
Anger
Normalizing Flows Agent
Orges
Dir(oraant
Drooder
Agent
Prodiction
Hackbone
Encoder Map
Arap
Normalizing Flows Map
Maaay
Dir(sd)
Prodictioo
Input Represetation
Draoderr IED Map
Frender
Normalizing Flows Social Contest
Dir(sre))
Prediction
Social
Dosoder Social Contest
Figure 1: Interpretable self-aware prediction (ISAP) system overview. ISAP learns the uncertainty distributed over semantically interpretable latent concepts: the agent's past behavior, map, and social context. The network outputs parameters Olagent, Cmup, and Ose for the corresponding Dirichlet distributions, learned using normalizing flows, which can then be combined to form the output Dirichlet distribution, Dir(a)). The aleatoric uncertainty for the trajectory prediction task is modeled by the expected value of the Dirichlet parameters and can be used make trajectory predictions (threc most likely predictions are shown), while their sum ou indicator of epistemic uncertainty. high on indicates evidence for input, and corresponds low epistemic uncertainty.
all possible scenarios encountered on the road, an OOD dataset cannot be manually curated for training. Moreover, successful epistemic uncertainty estimation techniques, such ensembles [32] and Monte Carlo (MC) dropout [18], require multiple forward passes during inference, potentially hindering real-time performance. Finally, most methods output single value for epistemic uncer- tainty. cluttered, dynamic setting (e.g., urban driving), where there are multiple possible sources of uncertainty, this value could be challenging to interpret. For example, it may be unclear whether high epistemic uncertainty stems from an unfamiliar trajectory maneuver, new intersection type, or strange behavior of surrounding agents due to external disturbance (e.g.. road construction).
Modeling epistemic uncertainty for trajectory prediction has three key challenges: (1) lack of ex- plicit OOD data during training. (2) efficiency requirements during inference, and (3) interpretability of the computed uncertainty. This paper addresses these challenges for trajectory prediction through an evidential deep learning approach that estimates the epistemic uncertainty in one-shot with only a single forward pass through the model and requires no OOD data for training. We allocate this un- certainty among interpretable, now-dimensional latent variables. Evidential deep learning estimates parameters for second-order distribution (e.g., a Dirichlet distribution) to capture the epistemic un- certainty [19, 20, 33]. To avoid OOD data for training, normalizing flows may be used to constrain the learned Dirichlet distribution parameters by enforcing the densities for each class to integrate to the number of training samples in that class, as in the Posterior Network (PostNet) architecture [34].
In trajectory prediction, human behavior is often modeled using discrete modes that can represent high-level maneuvers like accelerating, braking, and turning [12, 35-39J. We apply ideas from PostNet [34] for modeling epistemic uncertainty to trajectory prediction architectures with discrete modes. To address interpretability in the learned epistemic uncertainty, we propose the following semantic insight. High epistemic uncertainty for trajectory prediction may originate from unfamil- iar input behaviors, road structures, or social contexts. We distribute the learned epistemic uncer- tainty over these categories encoded a low-dimensional latent space, thus encouraging tractability. Hence, we introduce trajectory prediction paradigm that is self-aware of its prediction confidence and able to provide rich, interpretable information to downstream autonomy stack components and to engineers for development. We call this paradigm Interpretable Self-Aware Prediction (ISAP).
Our key contributions are: 1) We propose novel application of evidential deep learning to the task of epistemic uncertainty quantification for trajectory prediction. 2) We introduce interpretability into the uncertainty estimate by distributing it over interpretable, low-dimensional latent variables. The epistemic uncertainty source is decomposed as coming from unfamiliar agent behavior, road con- figuration, or social context. 3) We demonstrate superior uncertainty estimation performance of our ISAP framework over state-of-tth-aar (SOTA) approaches on the real-world NuScenes dataset [40].

---

2 Related Work
Interpretable Trajectory Prediction. One way to encourage interpretability in trajectory predic- tion architectures is through discrete modes [12, 36-39, 41]. For example, Chai et al. [37], Man- galam et al. [42], and Hu al. [43] induce interpretability by learning distributions over discrete in- tention goals. Kothari et al. [41] learn probability distribution over possibilities in an interpretable discrete choice model. Due to the ubiquity of discrete modes in trajectory prediction literature, we develop an epistemic uncertainty quantification approach assuming the presence of discrete modes in the architecture. Another common method for facilitating interpretability
latent space is through an encoder-decoder structure. Neumeier et al. [44] use decoder with expert knowledge to produce an interpretable latent space in a trajectory prediction model. Inspired by this idea, we enforce the epistemic uncertainty to be learned over three separate latent encodings that correspond to observed agent behavior, road configuration, and social context. This interpretable structure is achieved by using decoder components that are learned through self-supervised signals.
Epistemic Uncertainty for Bearning-Based Perception and Prediction. Gawlikowski et al. [17] survey modern uncertainty quantification methods. A common approach to compute epistemic un- certainty in learning-
autonomous systems to use MC dropout [18] due to its simple imple- mentation and because it does not require OOD data for training. This approach is used in tasks like trajectory prediction [45, 46], pedestrian bounding box prediction [47]. semantic segmentation and depth regression [48], and inverse sensor model learning [49]. However, MC dropout requires multiple forward passes during inference and is less robust to distribution shift than ensembles [50]. Moreover, MC dropout requires dropout architecture element, which may not always be desir- able. Ensembles [32] are regarded as a robust epistemic uncertainty estimation technique, but can be expensive to train and perform inference, limiting their use in robotics. One-shot evidential un- certainty estimation methods, such as deep evidential regression, have recently shown promising results in perception tasks, like depth estimation [24, 25). In this work, we explore the of the PostNet evidential deep learning technique [34] within a trajectory prediction task. PostNet does not require OOD data during training and only needs one forward pass during inference obtain both the model output and the epistemic uncertainty estimate. We are interested in how this technique scales to nigh-dimensional data necessary for trajectory prediction.
3 Methods
Problem Definition. We consider the following trajectory prediction setting. We assume the input representation consists of combination of the road structure (e.g., high-definition (HD) map), the past trajectory and current state for the agent of interest, and the past trajectory information for the surrounding agents. The agent's s state consists of speed u, acceleration a, and heading change rate h. The goal of the trajectory prediction task is to predict a 2D position vector E IR2x7 for time horizon 7 time steps into the future. We assume that the trajectory prediction architecture has set of discrete anchors with ground truth labels d E (1....C), For example, the MultiPath [37] architecture uses k-means to cluster trajectories into a discrete latent space and the CoverNet [38] model constructs a set of possible trajectories with a specified level of coverage.
Posterior Network (PostNet). To estimate epistemic uncertainty in the trajectory prediction set- ting, we use ideas from PostNet [34]. PostNet is an evidential deep learning approach [19, 20) that uses normalizing flows [51] to learn closed-form posterior distribution over predicted probabili- ties. We analyze its performance as applied to the discrete anchor space of the trajectory prediction task. PostNet's posterior distribution encompasses both aleatoric and epistemic uncertainties with- out requiring OOD data during training. The uncertainty is represented by a Dirichlet distribution, the conjugate prior of the categorical distribution. This approach is one-shot as it takes one network pass to compute the epistemic distribution ()) and the aleatoric distribution p(0) for an input (1)
q i = Dir(a (9) and g( = C)) with
E(a) Se = E
(1)
where i is the dataset index, e is the anchor class, ( E RC are the Dirichlet parameters, and a (6) = De= a! is the total amount of evidence allocated to the input (higher og indicates lower
epistemic uncertainty). The parameters ((t) {10, E. e!0) 113 of a categorical distribution
3

---

p(0) = Cat(5()) can be sampled from the epistemic distribution: 5(1) ~ q()) An anchor predic- tion d is made according to: d()) arg max, < The Dirichlet parameters (+) are constructed as a greio + 5(), where gprioe RC is fixed prior and B(0) E H represents learned pseudo- counts as evidence for an input (ú), Following Charpenticr et al. [34], we use an uninformative prior with gprir = 1. With lower confidence (smaller parameters), want the learned distribution to be close to the uninformative prior parameterized by Sprice. The pseudo-counts 3 are defined as,
(2) where 2 is a low-dimensional continuous latent space, ó contains the network parameters, and No reflects the ground truth count for anchor class serving per class certainty budget. The probability density r(2(1) o) is learned over the low-dimensional latent space to encourage tractability and scaling of the algorithm to high-dimensional inputs. First, neural network encodes the input (+1) into the latent space, 20() f)())))) Then, due to its representational capacity, normalizing flow is used to learn the distribution over this latent space [51, 52]. It is important that
BC N.. -Nrrr(())) (;
r(2() G; 6) be normalized density to encourage the di parameters be high for high-density, in-distribution (ID) regions (low epistemic uncertainty) and low for low-density, OOD regions (high epistemic uncertainty). As r(((() c; Φ) goes to zero, the ( parameters reduce to the uninformative prior gprior 1. To optimize PostNet,
(5)
the evidence lower bound (ELBO) loss [53].
N
LELBO
Σ N
1
E [To (0g
(LL(g [)
|| Dir(1)).
Interpretable Self-Aware Prediction (ISAP). The PostNet approach uncertainty quantification naturally transfers to trajectory prediction models with supervised, discrete latent anchors. However, make the uncertainty estimates more informative in complex settings (e.g., urban scenes), we fuse interpretability into the latent space z, forming the proposed ISAP approach. We encode the input x into three separate latent variables: Zagents Zmup- and 2 representing the agent's past be- havior, the road structure map, and the social context surrounding the agent of interest, respectively. Such a decomposition has been shown to be effective at the input level for trajectory prediction, supporting our choice of interpretable structure [54]. These semantic concepts are encoded into the latent space using accompanying decoder components. The decoders are self-supervised with the in- put æ split into the agent's past trajectory, the road structure representation, and the past trajectories of other agents. The decoder weights are learned through associated reconstruction losses.
The ISAP network then outputs parameters for three Dirichlet distributions corresponding to the three semantic concepts. These Dirichlet distributions are combined through an equally weighted average of their parameters, signifying uniform prior over the three categories,
(Oagont + m'aa + 0ss)/33.
(4)
These a parameters are used to construct the distributions in Eq. (1). The predicted trajectory the most likely anchor trajectory according to the aleatoric categorical distribution p(o) The Dirichlet distribution (8)) defines the epistemic uncertainty of the prediction. The full ISAP loss then,
L LEBB + Xagent Crec. Amap frep + Are
(5)
where Aagents Amaps and ASe are scaling coefficients for each reconstruction loss term. Additional details on the reconstruction loss terms can be found in Appendix A. Postels et al. [27, 55] demon- strate that regularization of the latent space in terms of reconstruction capability improves epistemic uncertainty estimation. These findings further support our choice of interpretable architecture for the epistemic uncertainty quantification task. The full ISAP architecture is illustrated in Fig. 1.
4 Experiments
We empirically validate the epistemic uncertainty estimation and OOD detection capabilities of our ISAP paradigm. All models are trained on a single NVIDIA GeForce RTX 2080 Ti GPU. Further details are provided in Appendices A and B.
Data. We test ISAP on the NuScenes [40] autonomous driving dataset. The predictions are made for 6 in the future based on 1 of past data collected at 2 Hz, following Phan-Minhl et al. [38]. The

---

input representation E [0, 1|5000600xx3 combines the agent's ' past trajectory, HD map, and past trajectories of other agents into a bird's-eye view rendering of the scene (see Fig. 1). The agent's state [v. a, hi] serves as input to each network branch. We consider two OOD data splits. First, we split the data according to the agent's past trajectory. ID input trajectories are chosen to be slower than OOD ones. We use the l distance between the oldest and most recent waypoints as heuristic for trajectory 'speed'. We threshold the ID data to have an &g distance of less than 10 m, leaving faster trajectories for OOD data. We also consider an OOD data split according to the map structure. ID data is taken from Singapore (left-side driving) and does not contain 'roundabout" or "big street' in the description. OOD data is from Boston (right-hand driving) with 'roundabout' in the description. Since the metadata refers to the scene and not the current local map, some straight roads exist in the OOD data as well. We verify that our chosen OOD splits are difficult to generalize to for a trajectory prediction model and, thus, important for OOD detection in Appendix C.
Architecture Details. For our trajectory prediction architecture, we employ the CoverNet [38] model, which is the baseline technique for the NuScenes prediction task. This model is convenient because frames trajectory prediction as classification over a predefined set of trajectories. Thus, we can directly integrate our ISAP approach with this architecture. For our experiments, we use trajectory anchor set of size 64 for classification [38]. The latent variables Zugents Amap and 2 are set to be four-dimensional, as this low dimensionality results in good uncertainty estimation and computational efficiency for the normalizing flows. The probability density r(2(1) | c; Φ) for each of the three latent variables is modeled with radial normalizing flows consisting of eight layers as done by Charpentier et al. [34]. The map and social context decoders take as input features one layer upstream of zmap and Zsc of dimension 4, 096 to enable higher representational capacity. We modify the ELBO loss to use CoverNet's constant lattice loss in the reconstruction term. The classification labels are the trajectory anchor classes with the smallest & distance to the ground truth trajectories.
Baselines. We consider three baselines to our approach: CoverNet [38], Post-CoverNet, and en- sembles [32]. We benchmark against CovenNet for trajectory prediction and calibration performance without modifications. Post-CoverNet is ablation of our ISAP approach without the inter- pretability element, instead having a single, non-interpretable latent variable z. Finally, ensembles are SOTA for estimating epistemic uncertainty for neural network models. Gustafsson al. [26] show that ensembles [32] consistently outperform MC dropout. Thus, we baseline against the more performant of the two approaches with N 5 and N 10 models in the ensemble.
Metrics. We employ a variety of metrics to investigate how well ISAP (I) estimates epistemic uncertainty and (2) maintains trajectory prediction performance. To measure trajectory prediction performance, use standard trajectory prediction metrics [38]: minimum average displacement error over the most likely & modes (minADEx) and final displacement error (FDE). Lower is better.
We then evaluate the uncertainty estimation performance. Following Charpentier et al. [34], we use the area under the receiver operating characteristic (AUROC) and average precision (APR) to com- pute the confidence calibration in the predictions (higher is better). We want the network to output high confidence for correct predictions (labeled 1) and low confidence for incorrect ones (labeled 0).
The scores to compute the alcatoric confidence are max - For epistemic confidence, the scores are max, ae for Post-CoverNet and ISAP and 1 1/V ar, for ensembles where Var, is the empirical variance of the predicted class probability across the ensemble. The expected calibration error (ECF) compares the output distribution to model accuracy. The Brier score is another calibration metric:
N EN [15(0) - || where d(() are one-hot labels. Lower is better for these metrics.
To evaluate OOD detection performance, we use AUROC and APR with labels 0 for OOD and for ID data (higher is better). For OOD detection based on aleatoric uncertainty, the scores are
max, ((1) When based on epistemic uncertainty, the scores are (i) for Post-CoverNet and ISAP
and 1/Var, for ensembles. To provide further intuition for Post- Post-CoverNNt and ISAP, we also report the ratio of the average sums of the Dirichlet parameters for ID and OOD data ão OOD/ / (lower better). Finally, we consider entropy as an OOD detection indicator in Appendix D.
Results
Quantitative Results The trajectory prediction performance is reported in Table on ID test data. As could be expected, the best performing approaches are the ensemble baselines. Ensembles tend to be more robust than their single network counterparts, in this case represented by CoverNet [38].
5

---

Table 1: Trajectory prediction metrics (lower is better) on ID test data. The best performance is highlighted in bold. Our methods perform comparably ensembles and the original CoverNet model.
CoverNet (38)
Ensemble [32]
Ensemble [32] (N 10)
Post-CoverNet (Ours)
ISAP
(N
(Ours)
Input Past Trajectory Experiment
minADE;
4.327
4.241
4.246
4.529
4.711
minADEs
1,885
1.867
1.859
1.951
2,004
minADE
1,545
1,529
1.539
1.581
1.599
m5n
1.413
1.421
1.423
1,440
1.474
FDE
9.474
9,270
9.293
10.009
10.177
Map- Experiment
minADE;
4.732
4.227
4.227
4.726
4,822
minADEs
2.115
2,053
2,019
2.069
2.149
minADE
1.731
1.737
1,578
1,556
1.555
1.583
1,600
FDE
10.590
9.344
9.118
10.531
10.503
Interestingly, the smaller ensemble (NV 5) slightly outperforms the bigger ensemble (N 10) for the input past trajectory experiment, indicating that the higher variability among models may cause small drop in performance. As Post-CoverNet and ISAP add competing terms the tra- jectory prediction objective, it is not surprising that the trajectory prediction performance is mildly compromised as a result. However, is encouraging that the performance drop is only slight.
The focus of our work is on accurately estimating the epistemic uncertainty of the trajectory predic- tion model. The uncertainty quantification results are presented in Table 2. The ISAP model out- performs the baseline approaches across almost all metrics for the input past trajectory experiment. We make two interesting observations. The first is that ISAP outperforms Post- CoverCoo in epis- temic uncertainty estimation. It appears that the interpretability encoded into the latent space within ISAP helps its performance, particularly in OOD data detection. We hypothesize that distributing the uncertainty over simpler, interpretable latent variables makes the uncertainty estimation task eas- ier. The second observation is that ISAP outperforms ensembles in OOD detection. Ensembles are often the canonical method for OOD detection; however, ISAP and Post-CoverNet outperform the smaller ensemble by large margin. The bigger ensemble approaches Post- Cost-CoverNet performance, but ISAP still outperforms. This result supports similar findings by Charpentier et al. [34] for the PostNet architecture on smaller classification tasks.
The results for the map-based experiment largely follow the trends observed in the input past trajec- tory experiment. ISAP outperforms the ensembles in OOD detection. Generally, we did not find the confidence metrics or the Brier and ECE scores to be reflective of the OOD detection performance. The map-based experiment is overall more challenging than the input past trajectory experiment since the filters used to differentiate map characteristics describe scenes rather than local maps, re- sulting in straight roads appearing in OOD data. As such, the ID and OOD split not as clear-cut as in the input past trajectory experiment. Nevertheless, we observe similar trends in performance across the methods for both experiments. ISAP consistently outperforms the baselines in OOD detection, while remaining performant in trajectory prediction.
We investigate how well the learned pseudo-counts og reflect the true data distribution for the input past trajectory experiment in Fig. 2. We plot the true data distribution as a histogram over the & dis- tance between the oldest and most recent waypoints in the agent's past trajectory. We distinguish between ID (green) and OOD (orange) examples. The data peaks at 0 m (stopped) and at around 5 m (18 km/ /h),, and then slopes off for OOD data. The learned agem parameters reflect these trends. As the ID and OOD difference is less obvious for the map and social context latent variables, the map and at trends are more flat across both data types, although Cipmmup still shows a clear dis- tinction. We hypothesize that the road configuration (e.g., multi-lane highway versus roundabout) is correlated with agent speed, while the social context may not always reflect the agent's speed.
Qualitative Results. Fig. presents qualitative results for ISAP on ID and OOD examples. The figure shows the input to the network, the decoded latent variables, and their associated og values. In the input past trajectory experiment, the example has slower (shorter) input trajectory than the OOD example. The ISAP network produces clear distinction between the ID and OOD examples in the Dirichlet parameter reflecting the agent's past trajectory, as expected. The ID Org.agem value is much higher (lower uncertainty) than that for OOD. The OOD OrO.agent value reaches almost total

---

Table 2: Uncertainty estimation metrics. If there two numbers, they are for ID (OOD) test data. Otherwise, the data is detailed in Section 4. The best performance in bold. Our methods outperform across most metrics.
CoverNet [38]
Ensemble [32]
Ensemble (32)
Post-CoverNet
ISAP (Ours)
(N 5)
(N 10
(Ouns)
Input Past Trajectory Experiment
Alea. Conf. (AUROC) 1
0.638 (0.430)
0.638 (0.399)
0.636 (0.119)
0.630 (0.721)
0.652 (0.733)
Epi. Conf. (AUROC)
0.455 (0.648)
(0,991)
(0.789)
0.621 (0.745)
Alea. Conf. (
0.525 (0.171)
0.551 (0. 180)
0.542 (0.179)
0,465 (0.262)
0.486 (0.281)
Epi. Conf. (APR)
0.089 (0.001)
0.086 (0.034)
0.108 (0.293)
0,451 (0.316)
ECEE
0.021 (0:339)
0.045 (0.317)
0.056 (0.2800
0.017 (0.198)
0.048 (0.053)
Brier Score
0.837 (L.011)
0.835 (1.007)
0.840 (1.000)
0.857 (0.963)
0,850 (0.960)
Alen. OOD (APR)
0.542
0.530
0.538
0.833
0,930
Fpi. OOD (APR)
0.961
0,960
Alea, OOD (AUROC)
0
0.218
0.240
0.652
0.871
Epi. OOD (AUROC)
0.685
0.913
0.919
0.955
1
0.171
0.145
Map-Based Experiment
Alea. Conf. (AUROC) f
0.594 (0:415)
0.629 (0.636)
0.631 (0.616)
0.610 (0.593)
0.582 (0.630)
Epi. Conf. (AUROC) f
0.582 (0.618)
0.635 (0.681)
0.610 (0.647)
0.575 (0.707)
Alea. Cont. (APR)
0.399 (0.126)
0.531 2252
(0.292)
(0.222)
Epi. Conf. (APR) 1
0.103 (0.043)
0.129 (0.164)
0.463 (0.284)
0.425 (0.28.1)
ECCE J.
0,056 (0.200)
0.080 (0.118)
0.092 (0.087)
0.046 (0.132)
0.113 (0. 102)
Brier Score
0.873 (0.985)
0.845 (0.964)
(0.973)
0.868 (0.968)
Alea. OOD (APR)
0,906
0.946
0,941
0,913
0.956
Epi. OOD (APR)
0.876
0.875
0.941
0,968
Aleu. (AUROC)
0.690
0.777
0.724
Epi. OOD (AUROC)
0.552
0.553
0.756
(L838
Go,000/ /Go.D
0,502
0.245
uncertainty as the uniform prior over 64 possible anchors would produce oxp = 64. The map and 00,se values are also lower for the OOD input than the IDD one. We hypothesize that intersections are more likely in the ID data, where the agent of interest is traveling at slow speed, than large four- lane roads that resemble highways as in the OOD example. A surprising observation in Fig. 3 is that the OOD so value is quite high, suggesting low epistemic uncertainty. In both the ID and OOD examples, the agent of interest is traveling in traffic with many cars on the road. Such scenarios are likely common for the slow-speed training data, indicating ID characteristics. Thus, our ISAP paradigm provides insight into the interpretable sources of epistemic uncertainty.
In the map-based experiment, the ID agent of interest is traveling at moderate speed along a straight, two-lane road with car ahead. This scenario is pretty likely according to our expectations and the learned og pseudocounts. In the OOD data input, the map shows a roundabout, which should not be present in the ID data. As expected, the OOD do map value is significantly lower than that of the ID example the network has not seen this intersection type during training.
6 Conclusions
In this paper, we propose a new trajectory prediction paradigm called Interpretable Self-Aware Pre- diction (ISAP). ISAP learns the aleatoric and epistemic uncertainty over discrete set of supervised anchors in trajectory prediction architecture. These uncertainties are estimated in one-shot by us- ing ideas from evidential deep learning. We introduce interpretability into the epistemic uncertainty
True Data Crent
Distance (m)
Distance (m)
Distance (m)
Distance (tn)
Figure 2: Learned 000 00 maps and 00.s pseudocounts (3 right plots) in comparison to a histogram of the true data counts (left plot). All data is from the ID and OOD test sets, aside from the histogram ID data, which reflects the training distribution. Outliers with large og values omitted for visualization. The fg distance (in meters) between the oldest and most recent 2D waypoints is proxy for input trajectory speed. The an. parameters accurately reflect the trends the true data distribution with increasing input trajectory speed.
7

---

Input Past Trajectory Experiment
ID
-10
-20
Input Representation
(0.ageni
8137
C0 1060
00,ec
OOD
20
-5
Input Representation
(kQuapeni
69
00,mu 117
0p.c
Map- Map-Based Based Experiment
ID
-10
-20
Input Representation
(t0.agen
1983
(rg,nop 13929
0o,
OOD
20
Input Representation
Ogg.ggnn 2197
000mm 259
00 908
Qualitative results our ISAP framework input trajectories are in meters. In the input past trajectory experiment, the OOD example has faster input trajectory than the ID onc. Thus, the learned epistemic uncertainty in the agent behavior latent variable is higher agen is lower) for the OOD case than the ID In the map-based experiment, the OOD example portrays a roundabout, while data. Thus, the learned epistemic uncertainty the map latent variable is higher (ako is lower) OOD case than the ID case.
Figure 3:
and OOD examples. Plot axes for the decoded
one shows straight, two-lane road. Roundabouts should
present in the ID
estimate by subdividing the uncertainty into the semantic concepts: past agent behavior, road struc- ture map, and social context. Our approach maintains comparable trajectory prediction performance an unmodified trajectory prediction architecture and outperforms established techniques, like en- sembles, in uncertainty estimation while requiring only single network pass during inference.
Limitations. Although show that normalizing flows in the ISAP framework are performant for uncertainty estimation, they can be brittle and slow to train. They also struggle to scale to large latent spaces, restricting the representational capacity of the latent space. Moreover, by design, impose an inductive bias on our ISAP framework for trajectory prediction in the context of AVs. For other applications, like assistive robotics, the interpretable structure may need be adapted (e.g.. to include task-level concepts, such as cooking or cleaning). Lastly, although the drop in trajectory prediction metrics is small with the addition of epistemic uncertainty estimation, it is prudent to consider how this gap could be closed in future work.
Future Work. Another promising avenue for future work is extending our ISAP paradigm to map-centric environment prediction [56-58]. map-centric prediction, the inputs and outputs are sequences of occupancy grids, which are higher dimensional than inputs and outputs in traditional, object-centric trajectory prediction. Map-centric representations are robust to partial occlusions. handle arbitrary number of agents in the scene, and do not require significant preprocessing. Thus, scaling epistemic uncertainty estimation to these settings is an exciting open research problem.

---

Acknowledgments
This work was supported by funding from Waymo. We thank Ben Sapp and Dragomir Anguelov for insightful discussions throughout the project. We thank Spencer M. Richards and Ransalu Senanayake for their invaluable feedback.
References
[1] A. Rudenko, L. Palmieri, M. Herman, K. M. Kitani, D. M. Gavrila, and K. O. Arras. Human motion trajectory prediction: A survey. International Journal of Robotics Research, 39(8): 895-935, 2020.
[2] Leon and M. Gavrilescu. A review of tracking and trajectory prediction methods for au- tonomous driving. Mathematics, 9(6):660, 2021.
[3] N. Sünderhauf, O. Brock, W. Scheirer, R. Hadsell, D. Fox. J. Leitner, B. Uperoft, P. Abbeel, W. Burgard, M. Milford, et al. The limits and potentials of deep learning for robotics. Interna- fional Journal of Robotics Research, 37(4-))440--22) 2018.
[4] D. Hendrycks and K. Gimpel. A baseline for detecting misclassified and examples in neural networks. International Conference on Learning Representations (ICLR), 2017.
[5] A. Nguyen, J. Yosinski, and J. Clune. Deep neural networks easily fooled: High confidence predictions for unrecognizable images. Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pages 427-436. IEEE, 2015.
[6] K. Nguyen and B. O' O'Connor. Posterior calibration and exploratory analysis for natural lan- guage processing models. In Conference on Empirical Methods in Natural Language Process- ing (EMNLP), 2015.
[7] J. Provost Foster, F. Tom, and K. Ron. The case against accuracy estimation for comparing induction algorithms. In International Conference on Machine Learning (ICML), pages 445- 453, 1998.
[8] D. Yu, J. Li, and L. Deng. Calibration of confidence measures in speech recognition. IEEE Transactions on Audio, Speech, and Language Processing, 19(8):2461-244 2011.
[9] J. Nitsch, M. Itkina, R. Senanayake, J. Nieto, M. Schmidt, R. Siegwart, M. J. Kochenderfer, and C. Cadena. Out of distribution detection for automotive perception. In International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2021.
[10] J. Walker, C. Doersch, A. Gupta, and M. Hebert. An uncertain future: Forecasting from static images using variational autoencoders. In European Conference Computer Vision (ECCV), pages 835-851. Springer, 2016.
[11] M. Babaeizadeh, C. Finn, D. Erhan, R. H. Camphell, and S. Levine. Stochastic variational video prediction. In International Conference on Learning Representations (ICLR), 2018.
[12] T. Salzmann, B. Ivanovic, P. Chakravarty, and M. Pavone. Trajectron++:: Dynamically-feasible trajectory forecasting with heterogeneous data. In European Conference on Computer Vision (ECCV), 2020.
[13] M. Itkina, B. Ivanovic, R. Senanayake, M. J. Kochenderfer, and M. Pavone. Evidential spar- sification multimodal latent spaces in conditional variational autoencoders. Advances in Neural Information Processing Systems (NeurIPS), volume 33, 2020.
[14) P. Chen, M. Itkina, R. Senanayake, and M. J. Kochenderfer. Evidential softmax for sparse mul- timodal distributions in deep generative models. Advances in Neural Information Processing Systems (NeurIPS), volume 34, 2021.
[15] H. Cheng, W. Liao, M. Y. Yang, B. Rosenhahn, and M. Sester. AMENet: Attentive maps encoder network for trajectory prediction. ISPRS Journal of Photogranmetry and Remote Sensing, 172:253-266, 2021.

---

[16] M. Itkina, Y.J. Mun, K. Driggs-Campbell, and M. J. Kochenderfer. Multi-agett variational occlusion inference using people as sensors. In International Conference on Roboties and Automation (ICRA). IEEF, 2022.
[17] J. Gawlikowski, C. R. N. Tassi, M. Ali, J. Lee, M. Humt, J. Feng, A. Kruspe, R. Triebel, Jung, R. Roscher, et al. survey of uncertainty in deep neural networks. arXiv, 2021.
[18] Y. Gal and Z. Ghahramani. Dropout as Bayesian approximation: Representing model un- certainty in deep learning, In International Conference on Machine Learning (ICML), pages 1050-1059, 2016.
[19] M. Sensoy, L. Kaplan, and M. Kandemir. Evidential deep learning quantify classification uncertainty. In Advances in Neural Information Processing Systems (NeurlPS), pages 3179-
3 3189, 2018.
[20] A. Malinin and M. Gales. Predictive uncertainty estimation via prior networks. In Advances in Neural Information Processing Systems (NeurIPS), pages 7047-7058, 2018.
[21] Q Yu and K. Aizawa. Unsupervised out-of-distribution detection by maximum classifier dis- crepancy. In International Conference Computer Vision (ICCV), pages 9518 8-262.. IEEE, 2019.
[22] J. Ren, P. J. Liu, E. Fertig, Snock, R. Poplin, M. Depristo, J. Dillon, and B. Lakshmi- narayanan. Likelihood ratios for out-of-distribution detection. In Advances in Neural Informa- tion Processing Systems (NeurIPS), volume 32, pages 14707-14718, 2019.
[23] A. Vyas, N. Jammalamadaka, X. Zhu, D. Das, B. Kaul, and T. L. Willke, Out-of-distribution detection using ensemble of self supervised leave-out classifiers. In European Conference on Computer Vision (ECCV), pages 550-564, 2018.
[24] A. Amini, W. Schwarting, A. Soleimany, and D. Rus. Deep evidential regression. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[25] A. Malinin, S. Chervontsev, I. Provilkov, and M. Gales. Regression prior networks. arXiv, 2020.
[26] K. Gustafsson, M. Danelljan, and T. B. Schon. Evaluating scalable Bayesian deep learning methods for robust computer vision. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 318-319, 2020.
[27] Postels, M. Segu, T. Sun, L. Van Gool, F. Yu, and F, Tombari. On the practicality of deter- ministic epistemic uncertainty. arXiv, 2021.
[28] R. McAllister, G. Kahn, J. Clune, and S. Levine. Robustness to out-of-distribution inputs via task-aware generative uncertainty. In International Conference on Robotics and Automation (ICRA), pages 2083-2089. IEEE, 2019.
[29] A. Filos, P. Tigkas, R. McAllister, N. Rhinehart, S. Levine, and Y. Gal. Can autonomous vehicles identify, recover from, and adapt to distribution shifts? In International Conference on Machine Learning (ICML), pages 3145-3153. PMLR, 2020.
[30] A. Farid, S. Veer, and A. Majumdar. Task-driven out-of-distrrbure detection with statistical guarantees for robot learning. In Conference on Robot Learning (CoRL), pages 970-980. PMLR, 2021.
[31] J. Lee, J. Feng, M. Humt, M. G. Müller, and R. Triebel. Trust your robots! Predictive uncer- tainty estimation of neural networks with sparse Gaussian processes. In Conference on Robot Learning (CoRL), pages 1168-1179 PMLR, 2021.
[32] B. Lakshminarayanan, A. Pritzel, and C. Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems (NeurIPS), volume 30, 2017.
[33] M. Sensoy, I.. Kaplan, F. Cerutti, and M. Saleki. Uncertainty-aware deep classifiers using generative models. In Conference on Artificial Intelligence. AAAI, 2020.
10

---

[34] B. Charpentier, D. Zügner, and S. Günnemann. Posterior network: Uncertainty estimation without OOD samples via density-based pseudo-counts. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pages 1356-1367,2000
[35] E. Schmerling, K. Leung. W. Vollprecht, and M. Pavone. Multimodal probabilistic model- based planning for human-robot interaction. In International Conference on Robolics and Automation (ICRA), pages 1-9. IEEE, 2018.
[36] B. Ivanovic and M. Pavone. The Trajectron: Probabilistic multi-agent trajectory modeling with dynamic spatiotemporal graphs. In International Conference on Computer Vision (ICCV). IEEE, 2019.
[37] Y. Chai, B. Sapp. M. Bansal, and D. Anguelov. MultiPath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction. In Conference on Robot Learning (CoRL), pages 86-99. PMLR, 2019.
[38] T. Phan-Minh, E. C. Grigore, F. A. Boulton, O. Beijbom, and E. M. Wolff. CoverNet: Multi- modal behavior prediction using trajectory sets. In Computer Society Conference on Computer Vision and Pultern Recognition (CVPR), pages 14074-14083, IEEE, 2020.
[39] H. Zhao, J. Gao, T. Lan, C. Sun, B. Sapp. B. Varadarajan, Y. Shen, Y. Shen, Y. Chai, C. Schmid, et al. TNT: Target-driven trajectory prediction. In Conference on Robot Learning (CoRL). PMLR, 2020.
[40] H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom. NuScenes: A multimodal dataset for autonomous driving. In Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pages 11621-11631. IEEE, 2020.
[41] Kothari, B. Sifringer, and A. Alahi. Interpretable social anchors for human trajectory fore- casting in crowds. In Computer Society Conference on Computer Vision and Pattern Recogni- tion (CVPR), pages 15556-15566. IEEE, 2021.
[42] K. Mangalam, An, H. Girase, and J. Malik. From goals, waypoints & paths to long term human trajectory forecasting. In International Conference on Computer Vision (ICCV), pages IEEE, 2021.
[43] Y. Hu, W. Zhan, L. Sun, and M. Tomizuka. Multi-modal probabilistic prediction of interactive behavior via an interpretable model. Intelligent Vehicles Symposium (IV), pages 557-563. IEEE, 2019.
[44] M. Neumeier, M. Betsch, A. Tollkühn, and T. Berberich. Variational autoenccoerr-aasd ve- hicle trajectory prediction with an interpretable latent space. In International Conference on Intelligent Transportation Systems (ITSC), pages 820 827. IEEE, 2021.
[45] S. Capobianco, N. Forti, L. M. Millefiori, Braca, and P. Willett. Uncertainty-aware recurrent encoder-decoder networks for vessel trajectory prediction. In International Conference on Information Fusion (FUSION), pages 1-5. IEEE, 2021.
[46] P. Dijt and P. Mettes. Trajectory prediction network for future anticipation of ships. In Inter- national Conference on Multimedia Retrieval (ICMR), pages 73-81. ACM, 2020.
[47] A. Bhattacharyya, M. Fritz, and Schiele. Long-term on-board prediction of people in traffic scenes under uncertainty. In Computer Society Conference Computer Vision and Pattern Recognition (CVPR), pages 1194-4202 IEEE, 2018.
[48] A. Kendall and Y. Gal. What uncertainties do we need in Bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems (NeurIPS), volume 30, pages 5574-5584, 2017.
[49] D. Bauer, L. Kuhnert, and L. Eckstein. Deep, spatially coherent inverse sensor models with uncertainty incorporation using the evidential framework, In Intelligent Vehicles Symposium (IV), pages 2490-2495. IEFF, 2019.

---

[50] Y. Ovadia, E. Fertig, J. Ren, Z. Nado, D. Sculley, S. Nowozin, J. Dillon, B. Lakshminarayanan, and Snoek. Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, pages 13991-14002, 2019.
[51] D. Rezende and S. Mohamed. Variational inference with normalizing flows. In International Conference Machine Learning (ICML), pages 1530-1538 PMLR, 2015.
[52] D. P. Kingma, T. Salimans, R. Jozefowicz, X. Chen, I. Sutskever, and M. Welling. Improved variational inference with inverse autoregressive flow. Advances in Neural Information Pro- cessing Systems (NeurIPS), 29, 2016.
[53] D. Kingma and M. Welling, Auto-encoding variational Bayes. In International Conference on Learning Representations (ICLR), 2014.
[54] B. F. de Brito, H. Zhu, W. Pan, and J. Alonso-Mora. Social-VRNN: One-shot multi-modal trajectory prediction for interacting pedestrians. Conference on Robot Learning (CoRL), pages 862-872. PMLR, 2020.
[55] J. Postels, H. Blum, Y. Strümpler, C. Cadena. R. Siegwart, L. Van Gool, and F, Tombari. The hidden uncertainty in a neural networks activations. arXiv, 2020.
[56] M. Itkina, K. Driggs-Campbel and M. J. Kochenderfer. Dynamic environment prediction in urban scenes using recurrent representation learning. In Intelligent Transportation Systems Conference (ITSC), pages 2052-2059. IEEE, 2019.
[57] M. Toyungyernsub. M. Itkina, R. Senanayake, and M. J. Kochenderfer. Double-prood occu- pancy ConvLSTM: Spatiotemporal prediction in urban environments. In International Con- ference on Robolics and Automation (ICRA). IEEE, 2021.
[58] B. Lange, M. Itkina, and M. J. Kochenderfer. Attention augmented ConvLSTM for environ- ment prediction. International Conference on Intelligent Robots and Systems (IROS). IEEE, 2021.
[59] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.
[60] A. van den Oord, O. Vinyals, and K. Kavukcuoglu. Neural discrete representation learning. In Advances in Neural Information Processing Systems (NeurIPS), pages 6306-6315, 2017,
12


Reviews:
[{"review_id": "k5vWMk0OO1a", "reviewer": "Reviewer_we25", "summary": "This paper proposes a framework, named ISAP, for trajectory prediction with interpretable uncertainty estimates. The key idea enabling the interpretability is to split the network architecture into three low-dimensional but general components in the latent space: agents past behaviour, map, and social context. By using normalizing flows, the technique the paper proposes does not require OOD data for training. ISAP is benchmarked against nuScenes where in-distribution data is defined by a measure of the speed of the trajectory. The results show comparable trajectory prediction with baselines with improved uncertainty estimation.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n1. Algorithm (ISAP) is evaluated on a subset of the nuScenes dataset, which is relatively standard and contains a wide variety of conditions to test uncertainty over the latent space parameters in the method. \n2. Approach is described precisely in a way that seems possible to reproduce. Although I did not run the code, it looks well documented and provides clear instructions for running the experiments in the paper. The description of PostNet in particular was succinct but informative. \n3. Good selection of competitive baselines that the algorithm is compared against across a wide variety of metrics for uncertainty estimation.\n3. Experiments show competitive trajectory prediction and superior uncertainty estimation compared to baselines.  \n4. Qualitative experiments that show relative magnitudes of the learned uncertainty parameters along with thoughtful discussions interpreting those qualitative results in the context of the data. \n5. The paper is well-written and presented. The figures use space well to illustrate concepts and ground them in the data used. \n6. Very thorough discussion of related work covering a wide range of papers \n7. Uses road structure as another way to detect OOD data, which is much harder to capture in a straightforward way from the state space. \n\nWeaknesses\n\n1. The biggest weakness in my opinion is the definitions used of OOD and ID data for evaluation. The main difference that is evaluated on is trajectory speed, which is highly relevant for trajectory prediction, but seems relatively easily to detect when velocity is part of the state space. Detecting road structure (roundabout/side of road) is a large improvement. One more would be nice, especially since the baselines seem to perform quite well.\n2. The method proposed seems to be very similar to PostNet with the only difference being additional losses to form ISAP (the name for the proposed method) from PostNet. However, the majority of the paper is focused on how to split uncertainty into these semantically meaningful categories and ensuring that the benefits of that split are reflected in the results. In the rebuttal, the reviewers justified why this paper is a significant improvement on the PostNet backbone,  \n4. Discrete anchors with a predefined number of labels, which I don’t think is a major weakness because discretization often leads to nice properties for learned distributions, and there are many existing methods to do this effectively in a self-supervised manner. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper presents a method that is generally useful in driving domains, where being able to detect epistemic uncertainty is useful for planning.  Having 3 relatively general-purpose interpretable measures of uncertainty that have good quantitative results and sensible qualitative results is useful. The revision clarified some of the technical details and presented some results on a more meaningful OOD/ID axis on road structure. A reviewer who is more familiar with typical AUROC scores in this domain would be better suited to comment, but the differences in scores in the map-based experiment look very similar. ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "1. Scores for various metrics, even the more important ones look very similar across the baselines. \n1. The limitations mentioned on network capacity and difficulty of training normalizing flows seem like relatively small and engineering-focused limitations, especially since most techniques for these high-dimensional datasets that include an uncertainty estimate are already difficult to train. To what extent are the discrete anchors generalizable/scalable? Another significant limitation is that ISAP is a pure prediction method. \n1. Helpful additional information in the appendix. Some language can be made more specific/technical.\n"}}, {"review_id": "XgW1W2xorP-", "reviewer": "Reviewer_xJrB", "summary": "This work proposed a self-aware framework for motion prediction with interpretability of the estimated uncertainty. The proposed method, Interpretable Self-Aware Prediction (ISAP), is able to evaluate the epistemic uncertainty by introducing an evidential deep learning method and then distributing the epistemic uncertainty among several interpretable semantic concepts. \n\nThe key contributions of this paper are:\n\n1. They invest in using evidential deep learning methods for uncertainty estimation for the trajectory prediction task.\n\n2. They add interpretability into epistemic uncertainty estimations by distributing them over low-dimensional interpretable latent spaces.\n\nThe author shows good uncertainty estimation results over AUROC and APR, meaning that it can accurately estimate the epistemic uncertainty of the trajectory prediction model. ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "It is useful to introduce self-awareness to the motion prediction module to let the model tell what is known and what is unknown since over-confident prediction results might be harmful to the downstream modules. Besides, existing methods lack the interpretability of the estimated uncertainty, and it is helpful to introduce semantic concepts to make robust and safe trajectory predictions. \n\nHowever, more baseline of uncertainty estimation methods need to be shown in the experiment, and the influence of the semantic concept of social context is not obvious because of the lack of interactions in the chosen dataset. More discussions of the limitation are needed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper is well written and easy to read. Although the evidential deep learning framework is not original, readers can benefit from the self-aware motion prediction framework, which is rarely mentioned in this area. Furthermore, introducing interpretability is also useful to the downstream modules in the autonomy stack. \n\nHowever, there is a need for a more developed discussion around the limits of the approach.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are not well addressed", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "* Could you please give more details about the training time as you mentioned in 285 they can be brittle and slow to train.\n* Results shown in the paper seem can not show the learned interpretability of social context because of sparse interaction in Nuscenes, which is usually hardest to handle and where the uncertainty estimation is more useful. Therefore, maybe you can try WOMD/INTERACTION dataset which has more meaningful interactions instead of setting different coefficients during training. \n* For the OOD dataset split, why do you choose the l2 distance between the most recent and the earliest waypoints, what if it is a slow turn instead of going straight? And is the difference big enough to consider it as out-of-distribution data? The map-based setting seems more reasonable.\n* Maybe you can add more uncertainty estimation and OOD detection baseline to compare the performance of this particular framework. \n\n"}}, {"review_id": "GkuMnD2p1o2", "reviewer": "Reviewer_UZiy", "summary": "An uncertainty aware trajectory prediction framework of other road participants is presented, showing similar accuracy to deterministic baselines but improved calibration and out-of-distribution detection capabilities.\n\nBoth model (epistemic) and data (aleatoric) uncertainty are estimated and the former is further separated into coming from three sources, namely the agent behavior, the social context and the road configuration.\n\nAs fast inference is important in the autonomous driving setting, the uncertainty is obtained through a single forward pass using ideas from the evidential deep learning literature, i.e. it is modeled using a learned Dirichlet distribution.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strength\n* Uncertainty estimation is important for safety critical applications like autonomous driving\n* Fast, i.e. suitable to the autonomous driving setting\n* Only slight drop in accuracy compared to deterministic and ensemble baselines\n* Evaluated based on calibration and in- and out-of-distribution separation\n* Increased interpretability through decomposition into three sources of uncertainty\n* Clean exposition\n\n## Weaknesses\n* Calibration is measured using the Brier score instead of the expected calibration error which is commonly used.\n* Confidence in wrong predictions is not included which might arguably be of even greater importance than confidence in correct ones in safety critical contexts.\n* In- and out-of-distribution separation could be visualized and exploited using predictive entropy.\n* Choice of uncertainty decomposing variables is not well justified.\n* Out-of-distribution is defined through (arbitrary) speed threshold. Corruption, adversarial attacks or other datasets might yield stronger results.\n* Describing the system as \"self-aware\" seems a bit exaggerated and out of place.\n* Typos:\n  * \"may be not be\" (line 4)\n  * $\\lambda_{agent}$, $\\lambda_{agent}$ and $\\lambda_{agent}$ (line 163)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper shows promising advances for much needed uncertainty estimation in safety-critical, real-time systems. The decomposition of the epistemic uncertainty into different sources and the adaption of evidential deep learning approaches to trajectory prediction are interesting additions to the existing literature. The results are promising but not entirely convincing.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "1. Report common calibration metrics like expected calibration error.\n2. Report confidence in wrong predictions.\n3. Improve exposition of in- and out-of-distribution results.\n4. Properly justify the choice of uncertainty decomposing factors.\n5. Make use of stronger and more realistic out-of-distribution data.\n6. Rethink the use of \"self-awareness\".\n7. Fix typos."}}, {"review_id": "E-J62uUwUTMv", "reviewer": "Reviewer_WAfv", "summary": "This paper proposes a deep-learning based method to estimate epistemic uncertainty over a low-dimensional, interpretable latent space used for trajectory prediction. The key idea of the approach is to map the trajectory prediction data into three interpretable components: the agent, the map, and the social context and obtain a latent state encoding each of these aspects individually. To capture the certainty of each latent state, the proposed neural network outputs parameters of three Dirichlet distributions (one for the agent, the map, and the social context) that are averaged together for a total uncertainty score. Empirical results with the nuScenes dataset (where in-distribution data is slow driving vs. out-of-distribution data is fast driving) show that the proposed approach performs slightly worse than ensemble or alternative baselines on prediction metrics, but outperforms the baselines on uncertainty estimation metrics. \n", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n----------------\n–Important and timely problem choice\n\n–Interesting approach to balancing interpretability of the latent parameters, prediction performance, and epistemic uncertainty estimation\n\n–Evaluation with nuScenes dataset of real highway driving on two examples: datasets where (1) ID data = slow driving vs. OOD data = fast driving and (2) ID data = left-side driving + no roundabouts + small streets vs. OOD data = right-side driving + roundabouts.  \n\nWeaknesses:\n------------------\n–While at a high level the motivation of this work is good, the specifics of why this method enables better prediction and ultimately, better autonomous vehicle behavior, could be significantly strengthened. From the results, it seems like predictive performance is good for the baseline predictors (even marginally better than the proposed method) which would indicate that the baselines may be successful at prediction even on the presented OOD data (and therefore enabling downstream robot behavior to be safe  /effective). This implies that there may be marginal benefits to doing OOD detection with the proposed method for the proposed scenarios. Intuitively, I’d imagine that if the predictor is presented with truly OOD data that is really difficult for it to generalize from the training data, then the prediction performance would significantly suffer. This suggests that perhaps this ID/OOD data split isn’t the best to showcase why measuring epistemic uncertainty is so necessary. For example, I’d posit that slow vs. fast variations in driving behavior are easy to generalize between and seen so frequently / in such large quantities in typical datasets, that SOTA predictors don’t struggle with speed of driving. However, OOD data like drunk drivers, U-Turns, construction zones, etc. intuitively seem like they would be very hard for SOTA predictors to deal with, thereby resulting in poor predictive performance and motivating the need for OOD detection. This paper would be strengthened by highlighting (1) ID/OOD data that SOTA predictors truly struggle with and (2) demonstrating how the proposed OOD method can enable, for example, more “safe” predictions by incorporating robot decision-making that takes safe recourse when presented with an OOD scenario. \n\n–In general, it is challenging to understand the connection between the input $x$, the latent states $z$, and the generation of the dirichlet parameters $\\alpha$ (which are the crux of the uncertainty quantification method). This makes it difficult to deeply understand the key idea of this work and its strengths/weaknesses.\n\n–Overall, the summary of PostNet was quite challenging to follow for a reader who is not deeply familiar with this method. This makes it difficult to appreciate the contribution that hinges on this method. I would suggest changing the flow and adding some more exposition to clarify the key connections between all aspects of this predictor. For example, it would be helpful to start with how PostNet takes as input $x$ (and provide intuition for what this input is – e.g., this is a top-down RGB image of the scene) and this input is converted into a single latent state $z$ via a neural network: $z = f_\\theta(x)$. Then, how does this latent state then get converted into trajectory predictions? The text never clearly states this. Then, you can mention how to obtain uncertainty estimates: PostNet starts by learning a distribution over the latent state, conditioned on a discrete anchor class, c. Here it would be very helpful to provide intuition for why this anchor class is needed – for example, what does it represent in trajectory prediction? Then, you can mention that the shape of this distribution is key to obtaining an uncertainty estimate. The epistemic uncertainty is modeled as a Dirichelt distribution with parameters alpha where $\\alpha = \\beta^{prior} + \\beta_c$. Here is where you leverage the distribution over the latent state: $\\beta_c = N_c \\cdot r(z | c, \\phi)$. As the probability of a latent state given the anchor class goes to zero, then $\\alpha$ approaches $\\beta^{prior}$. Here it would be helpful to provide intuition for why this is a desirable property. Finally, you can describe how you also model the aleatoric uncertainty via the categorical distribution and why this works / is desirable. Then, the ELBO loss can be clearly introduced as the method for how this network is trained. It would also be helpful to provide some intuition for this loss as well. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Although the topic of this work is timely and relevant, the proposed approach warrants deeper investigation to present a strong result. Since the trajectory prediction for the proposed method seems to be on-par with the SOTA methods, it seems to imply that all the predictors can do a decent job of predicting the OOD examples too. So, even though the proposed method can detect OOD examples, the ultimate prediction performance (which would then affect the autonomous vehicle decision-making) seems to not need any OOD detection to achieve good ADE/FDE scores. I think these results warrant (1) investigation into “harder” OOD datasets that clearly showcase what SOTA predictors would suffer in prediction performance and (2) how being able to do OOD detection can be integrated with robot decision-making to achieve safer behaviors than what is achieved without OOD detection. Due to these reasons and the current state of the manuscript, I recommend weak reject. \n\n=====================\n\nAfter discussion, I change to \"weak accept\". ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "Section 2:\n\n–Figure 1: There are several aspects of this figure that could be improved to help the reader understand the method. First, the inputs $[v,a,h]$ are never described in the figure nor in the text. It could also help to label the input representation with x, as is referred to in the text. Second, it would be helpful to explicitly highlight where the latent states $z_{agent}$, $z_{map}$, and $z_{sc}$ are obtained in the pipeline, since these are key to the contribution. Additionally, the block called “Backbone” could be more descriptive. Finally, it could be clarified where the $\\alpha$ Dirichlet parameters come from, since this is another aspect of the contribution. It is also unclear why there are only three predictions as output of the final Dir block? Are these just the three most likely trajectories, or does the method only allow for three predictions? Overall, making the text notation consistent and explicit in Figure 1 would help this figure clarify + enhance the technical content. \n\nSection 3:\n\n–In the summary of PostNet, it was also difficult to understand what the index (i) stands for. Please clarify this.\n\n–In equation (4) why is it desirable to always equally weigh the three alpha parameters? Providing some intuition here would be helpful.  \n\nSection 4:\n\n–Please clarify what each aspect of agent state means: $[v, a, h]$. Also, is this just an instantaneous state or do you obtain the states over time as well? \n\n–Are these “types” of OOD data something that is typically a problem for predictors? It would be more powerful to first present the key “types” of OOD data that often mess up ensembles or other baselines. This would make the types of data used for testing much more convincing. Otherwise, right now it seems like somewhat of an ad-hoc choice to say that in-distribution data is “slow driving” while OOD data is “fast driving” when perhaps these types of variations in driving behavior are seen so frequently and in such large quantities in typical datasets, that SOTA predictors don’t struggle with them. \n\n–Are the latent states $z$ assumed to be discrete or continuous? It would be helpful to clarify this. \n\n–”Post-CoverNet is an ablation….without the interpretability element” does this just mean that there is no separation of the latent state into three components, and instead Post-CoverNet just produces a single latent state z? It would be helpful to clarify this. \n\n–”The ground truth labels are 0 for incorrect predictions and 1 for correct predictions.” It seems like the labels shouldn’t correspond to state prediction accuracy but whether or not a data point is ID or OOD. It would be helpful to clarify this in the text.\n\n–What does subscript $c$ refer to? What are the possible anchor classes?\n\n–In the Brier score definition, $\\bar{p}$ and $d$ seem to be incompatible: $\\bar{p}$ is defined as a distribution while $d$ is a class label. Also, what kind of classes can $d$ represent? Are classes the type of agents (e.g., car, bike, truck etc.) or something else?\n\nSection 5:\n\n–Figure 2: It was difficult to compare the leftmost plot (whose y-axis is total data count) vs. the right three plots (whose y-axis is the $\\alpha_{0, agent}$) since they have different y-axis, but look as though all four subplots should be compared equally. It would be helpful to explain how to compare / interpret these plots, or make the axes all have the same units. \n\n–Figure 3: It’s difficult to interpret the second plots from the left (with $\\alpha_{0, agent}$). Is this a spatial visualization (so the x and y axis represent the x and y of the world) where the red line is some sort of prediction? Or do the axes represent something different?\n\n–Since the trajectory prediction for the proposed method seems to be on-par with the SOTA methods, it seems to imply that all the predictors can do a decent job of predicting the OOD examples too. So, even though the proposed method can detect OOD examples, the ultimate prediction performance (which would then affect the autonomous vehicle decision-making) seems to not need any OOD detection to achieve good ADE/FDE scores. I think these results warrant (1) investigation into “harder” OOD datasets that clearly showcase what SOTA predictors would suffer in prediction performance and (2) how being able to do OOD detection can be integrated with robot decision-making to achieve safer behaviors than what is achieved *without* OOD detection. \n"}}]

Author rebuttal:

Decision: Accept (poster)