Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This work proposed a self-aware framework for motion prediction with interpretability of the estimated uncertainty. The proposed method, Interpretable Self-Aware Prediction (ISAP), is able to evaluate the epistemic uncertainty by introducing an evidential deep learning method and then distributing the epistemic uncertainty among several interpretable semantic concepts. 

The key contributions of this paper are:

1. They invest in using evidential deep learning methods for uncertainty estimation for the trajectory prediction task.

2. They add interpretability into epistemic uncertainty estimations by distributing them over low-dimensional interpretable latent spaces.

The author shows good uncertainty estimation results over AUROC and APR, meaning that it can accurately estimate the epistemic uncertainty of the trajectory prediction model. 
strengths_and_weaknesses: It is useful to introduce self-awareness to the motion prediction module to let the model tell what is known and what is unknown since over-confident prediction results might be harmful to the downstream modules. Besides, existing methods lack the interpretability of the estimated uncertainty, and it is helpful to introduce semantic concepts to make robust and safe trajectory predictions. 

However, more baseline of uncertainty estimation methods need to be shown in the experiment, and the influence of the semantic concept of social context is not obvious because of the lack of interactions in the chosen dataset. More discussions of the limitation are needed.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are not well addressed
summary_of_recommendation: The paper is well written and easy to read. Although the evidential deep learning framework is not original, readers can benefit from the self-aware motion prediction framework, which is rarely mentioned in this area. Furthermore, introducing interpretability is also useful to the downstream modules in the autonomy stack. 

However, there is a need for a more developed discussion around the limits of the approach.
issues: * Could you please give more details about the training time as you mentioned in 285 they can be brittle and slow to train.
* Results shown in the paper seem can not show the learned interpretability of social context because of sparse interaction in Nuscenes, which is usually hardest to handle and where the uncertainty estimation is more useful. Therefore, maybe you can try WOMD/INTERACTION dataset which has more meaningful interactions instead of setting different coefficients during training. 
* For the OOD dataset split, why do you choose the l2 distance between the most recent and the earliest waypoints, what if it is a slow turn instead of going straight? And is the difference big enough to consider it as out-of-distribution data? The map-based setting seems more reasonable.
* Maybe you can add more uncertainty estimation and OOD detection baseline to compare the performance of this particular framework.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes a deep-learning based method to estimate epistemic uncertainty over a low-dimensional, interpretable latent space used for trajectory prediction. The key idea of the approach is to map the trajectory prediction data into three interpretable components: the agent, the map, and the social context and obtain a latent state encoding each of these aspects individually. To capture the certainty of each latent state, the proposed neural network outputs parameters of three Dirichlet distributions (one for the agent, the map, and the social context) that are averaged together for a total uncertainty score. Empirical results with the nuScenes dataset (where in-distribution data is slow driving vs. out-of-distribution data is fast driving) show that the proposed approach performs slightly worse than ensemble or alternative baselines on prediction metrics, but outperforms the baselines on uncertainty estimation metrics. 

strengths_and_weaknesses: Strengths:
----------------
–Important and timely problem choice

–Interesting approach to balancing interpretability of the latent parameters, prediction performance, and epistemic uncertainty estimation

–Evaluation with nuScenes dataset of real highway driving on two examples: datasets where (1) ID data = slow driving vs. OOD data = fast driving and (2) ID data = left-side driving + no roundabouts + small streets vs. OOD data = right-side driving + roundabouts.  

Weaknesses:
------------------
–While at a high level the motivation of this work is good, the specifics of why this method enables better prediction and ultimately, better autonomous vehicle behavior, could be significantly strengthened. From the results, it seems like predictive performance is good for the baseline predictors (even marginally better than the proposed method) which would indicate that the baselines may be successful at prediction even on the presented OOD data (and therefore enabling downstream robot behavior to be safe  /effective). This implies that there may be marginal benefits to doing OOD detection with the proposed method for the proposed scenarios. Intuitively, I’d imagine that if the predictor is presented with truly OOD data that is really difficult for it to generalize from the training data, then the prediction performance would significantly suffer. This suggests that perhaps this ID/OOD data split isn’t the best to showcase why measuring epistemic uncertainty is so necessary. For example, I’d posit that slow vs. fast variations in driving behavior are easy to generalize between and seen so frequently / in such large quantities in typical datasets, that SOTA predictors don’t struggle with speed of driving. However, OOD data like drunk drivers, U-Turns, construction zones, etc. intuitively seem like they would be very hard for SOTA predictors to deal with, thereby resulting in poor predictive performance and motivating the need for OOD detection. This paper would be strengthened by highlighting (1) ID/OOD data that SOTA predictors truly struggle with and (2) demonstrating how the proposed OOD method can enable, for example, more “safe” predictions by incorporating robot decision-making that takes safe recourse when presented with an OOD scenario. 

–In general, it is challenging to understand the connection between the input $x$, the latent states $z$, and the generation of the dirichlet parameters $\alpha$ (which are the crux of the uncertainty quantification method). This makes it difficult to deeply understand the key idea of this work and its strengths/weaknesses.

–Overall, the summary of PostNet was quite challenging to follow for a reader who is not deeply familiar with this method. This makes it difficult to appreciate the contribution that hinges on this method. I would suggest changing the flow and adding some more exposition to clarify the key connections between all aspects of this predictor. For example, it would be helpful to start with how PostNet takes as input $x$ (and provide intuition for what this input is – e.g., this is a top-down RGB image of the scene) and this input is converted into a single latent state $z$ via a neural network: $z = f_\theta(x)$. Then, how does this latent state then get converted into trajectory predictions? The text never clearly states this. Then, you can mention how to obtain uncertainty estimates: PostNet starts by learning a distribution over the latent state, conditioned on a discrete anchor class, c. Here it would be very helpful to provide intuition for why this anchor class is needed – for example, what does it represent in trajectory prediction? Then, you can mention that the shape of this distribution is key to obtaining an uncertainty estimate. The epistemic uncertainty is modeled as a Dirichelt distribution with parameters alpha where $\alpha = \beta^{prior} + \beta_c$. Here is where you leverage the distribution over the latent state: $\beta_c = N_c \cdot r(z | c, \phi)$. As the probability of a latent state given the anchor class goes to zero, then $\alpha$ approaches $\beta^{prior}$. Here it would be helpful to provide intuition for why this is a desirable property. Finally, you can describe how you also model the aleatoric uncertainty via the categorical distribution and why this works / is desirable. Then, the ELBO loss can be clearly introduced as the method for how this network is trained. It would also be helpful to provide some intuition for this loss as well. 

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: Although the topic of this work is timely and relevant, the proposed approach warrants deeper investigation to present a strong result. Since the trajectory prediction for the proposed method seems to be on-par with the SOTA methods, it seems to imply that all the predictors can do a decent job of predicting the OOD examples too. So, even though the proposed method can detect OOD examples, the ultimate prediction performance (which would then affect the autonomous vehicle decision-making) seems to not need any OOD detection to achieve good ADE/FDE scores. I think these results warrant (1) investigation into “harder” OOD datasets that clearly showcase what SOTA predictors would suffer in prediction performance and (2) how being able to do OOD detection can be integrated with robot decision-making to achieve safer behaviors than what is achieved without OOD detection. Due to these reasons and the current state of the manuscript, I recommend weak reject. 

=====================

After discussion, I change to "weak accept". 
issues: Section 2:

–Figure 1: There are several aspects of this figure that could be improved to help the reader understand the method. First, the inputs $[v,a,h]$ are never described in the figure nor in the text. It could also help to label the input representation with x, as is referred to in the text. Second, it would be helpful to explicitly highlight where the latent states $z_{agent}$, $z_{map}$, and $z_{sc}$ are obtained in the pipeline, since these are key to the contribution. Additionally, the block called “Backbone” could be more descriptive. Finally, it could be clarified where the $\alpha$ Dirichlet parameters come from, since this is another aspect of the contribution. It is also unclear why there are only three predictions as output of the final Dir block? Are these just the three most likely trajectories, or does the method only allow for three predictions? Overall, making the text notation consistent and explicit in Figure 1 would help this figure clarify + enhance the technical content. 

Section 3:

–In the summary of PostNet, it was also difficult to understand what the index (i) stands for. Please clarify this.

–In equation (4) why is it desirable to always equally weigh the three alpha parameters? Providing some intuition here would be helpful.  

Section 4:

–Please clarify what each aspect of agent state means: $[v, a, h]$. Also, is this just an instantaneous state or do you obtain the states over time as well? 

–Are these “types” of OOD data something that is typically a problem for predictors? It would be more powerful to first present the key “types” of OOD data that often mess up ensembles or other baselines. This would make the types of data used for testing much more convincing. Otherwise, right now it seems like somewhat of an ad-hoc choice to say that in-distribution data is “slow driving” while OOD data is “fast driving” when perhaps these types of variations in driving behavior are seen so frequently and in such large quantities in typical datasets, that SOTA predictors don’t struggle with them. 

–Are the latent states $z$ assumed to be discrete or continuous? It would be helpful to clarify this. 

–”Post-CoverNet is an ablation….without the interpretability element” does this just mean that there is no separation of the latent state into three components, and instead Post-CoverNet just produces a single latent state z? It would be helpful to clarify this. 

–”The ground truth labels are 0 for incorrect predictions and 1 for correct predictions.” It seems like the labels shouldn’t correspond to state prediction accuracy but whether or not a data point is ID or OOD. It would be helpful to clarify this in the text.

–What does subscript $c$ refer to? What are the possible anchor classes?

–In the Brier score definition, $\bar{p}$ and $d$ seem to be incompatible: $\bar{p}$ is defined as a distribution while $d$ is a class label. Also, what kind of classes can $d$ represent? Are classes the type of agents (e.g., car, bike, truck etc.) or something else?

Section 5:

–Figure 2: It was difficult to compare the leftmost plot (whose y-axis is total data count) vs. the right three plots (whose y-axis is the $\alpha_{0, agent}$) since they have different y-axis, but look as though all four subplots should be compared equally. It would be helpful to explain how to compare / interpret these plots, or make the axes all have the same units. 

–Figure 3: It’s difficult to interpret the second plots from the left (with $\alpha_{0, agent}$). Is this a spatial visualization (so the x and y axis represent the x and y of the world) where the red line is some sort of prediction? Or do the axes represent something different?

–Since the trajectory prediction for the proposed method seems to be on-par with the SOTA methods, it seems to imply that all the predictors can do a decent job of predicting the OOD examples too. So, even though the proposed method can detect OOD examples, the ultimate prediction performance (which would then affect the autonomous vehicle decision-making) seems to not need any OOD detection to achieve good ADE/FDE scores. I think these results warrant (1) investigation into “harder” OOD datasets that clearly showcase what SOTA predictors would suffer in prediction performance and (2) how being able to do OOD detection can be integrated with robot decision-making to achieve safer behaviors than what is achieved *without* OOD detection.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes a framework, named ISAP, for trajectory prediction with interpretable uncertainty estimates. The key idea enabling the interpretability is to split the network architecture into three low-dimensional but general components in the latent space: agents past behaviour, map, and social context. By using normalizing flows, the technique the paper proposes does not require OOD data for training. ISAP is benchmarked against nuScenes where in-distribution data is defined by a measure of the speed of the trajectory. The results show comparable trajectory prediction with baselines with improved uncertainty estimation.
strengths_and_weaknesses: Strengths

1. Algorithm (ISAP) is evaluated on a subset of the nuScenes dataset, which is relatively standard and contains a wide variety of conditions to test uncertainty over the latent space parameters in the method. 
2. Approach is described precisely in a way that seems possible to reproduce. Although I did not run the code, it looks well documented and provides clear instructions for running the experiments in the paper. The description of PostNet in particular was succinct but informative. 
3. Good selection of competitive baselines that the algorithm is compared against across a wide variety of metrics for uncertainty estimation.
3. Experiments show competitive trajectory prediction and superior uncertainty estimation compared to baselines.  
4. Qualitative experiments that show relative magnitudes of the learned uncertainty parameters along with thoughtful discussions interpreting those qualitative results in the context of the data. 
5. The paper is well-written and presented. The figures use space well to illustrate concepts and ground them in the data used. 
6. Very thorough discussion of related work covering a wide range of papers 
7. Uses road structure as another way to detect OOD data, which is much harder to capture in a straightforward way from the state space. 

Weaknesses

1. The biggest weakness in my opinion is the definitions used of OOD and ID data for evaluation. The main difference that is evaluated on is trajectory speed, which is highly relevant for trajectory prediction, but seems relatively easily to detect when velocity is part of the state space. Detecting road structure (roundabout/side of road) is a large improvement. One more would be nice, especially since the baselines seem to perform quite well.
2. The method proposed seems to be very similar to PostNet with the only difference being additional losses to form ISAP (the name for the proposed method) from PostNet. However, the majority of the paper is focused on how to split uncertainty into these semantically meaningful categories and ensuring that the benefits of that split are reflected in the results. In the rebuttal, the reviewers justified why this paper is a significant improvement on the PostNet backbone,  
4. Discrete anchors with a predefined number of labels, which I don’t think is a major weakness because discretization often leads to nice properties for learned distributions, and there are many existing methods to do this effectively in a self-supervised manner. 

robotics_focus: Relevant but unlikely to deploy to hardware in near future
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper presents a method that is generally useful in driving domains, where being able to detect epistemic uncertainty is useful for planning.  Having 3 relatively general-purpose interpretable measures of uncertainty that have good quantitative results and sensible qualitative results is useful. The revision clarified some of the technical details and presented some results on a more meaningful OOD/ID axis on road structure. A reviewer who is more familiar with typical AUROC scores in this domain would be better suited to comment, but the differences in scores in the map-based experiment look very similar. 
issues: 1. Scores for various metrics, even the more important ones look very similar across the baselines. 
1. The limitations mentioned on network capacity and difficulty of training normalizing flows seem like relatively small and engineering-focused limitations, especially since most techniques for these high-dimensional datasets that include an uncertainty estimate are already difficult to train. To what extent are the discrete anchors generalizable/scalable? Another significant limitation is that ISAP is a pure prediction method. 
1. Helpful additional information in the appendix. Some language can be made more specific/technical.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: An uncertainty aware trajectory prediction framework of other road participants is presented, showing similar accuracy to deterministic baselines but improved calibration and out-of-distribution detection capabilities.

Both model (epistemic) and data (aleatoric) uncertainty are estimated and the former is further separated into coming from three sources, namely the agent behavior, the social context and the road configuration.

As fast inference is important in the autonomous driving setting, the uncertainty is obtained through a single forward pass using ideas from the evidential deep learning literature, i.e. it is modeled using a learned Dirichlet distribution.
strengths_and_weaknesses: ## Strength
* Uncertainty estimation is important for safety critical applications like autonomous driving
* Fast, i.e. suitable to the autonomous driving setting
* Only slight drop in accuracy compared to deterministic and ensemble baselines
* Evaluated based on calibration and in- and out-of-distribution separation
* Increased interpretability through decomposition into three sources of uncertainty
* Clean exposition

## Weaknesses
* Calibration is measured using the Brier score instead of the expected calibration error which is commonly used.
* Confidence in wrong predictions is not included which might arguably be of even greater importance than confidence in correct ones in safety critical contexts.
* In- and out-of-distribution separation could be visualized and exploited using predictive entropy.
* Choice of uncertainty decomposing variables is not well justified.
* Out-of-distribution is defined through (arbitrary) speed threshold. Corruption, adversarial attacks or other datasets might yield stronger results.
* Describing the system as "self-aware" seems a bit exaggerated and out of place.
* Typos:
  * "may be not be" (line 4)
  * $\lambda_{agent}$, $\lambda_{agent}$ and $\lambda_{agent}$ (line 163)
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The paper shows promising advances for much needed uncertainty estimation in safety-critical, real-time systems. The decomposition of the epistemic uncertainty into different sources and the adaption of evidential deep learning approaches to trajectory prediction are interesting additions to the existing literature. The results are promising but not entirely convincing.
issues: 1. Report common calibration metrics like expected calibration error.
2. Report confidence in wrong predictions.
3. Improve exposition of in- and out-of-distribution results.
4. Properly justify the choice of uncertainty decomposing factors.
5. Make use of stronger and more realistic out-of-distribution data.
6. Rethink the use of "self-awareness".
7. Fix typos.
review_title: null

Meta-review: metareview: Concluding comments:

The authors present a clearly written framework for epistemic uncertainty estimation with significance in particular to safety critical applications of trajectory prediction methods in autonomous driving. The underlying problem of epistemic uncertainty estimation is clearly of relevance to the community and the presented approach appears to be quite well founded and evaluated on the NuScenes dataset. 

The reviewers were uniform in their recommendation of weak accept x4. Following the initial reviews, the authors have further addressed the concerns from the previous meta review (see below) in their revision/rebuttal to a point where I would also like to recommend the work to be accepted.   
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)