Venue: CoRL 2021 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes a method for anomaly detection for the autonomous driving domain, by learning a representation for joint trajectories of normal driving behaviors, then performing density estimation on the latent representation of the joint trajectory auto-encoder.  At test time, anomalies are detected by test trajectories occurring in low density regions of the learned latent space.  In particular, the proposed method consists of first training a spatio-temporal graph CNN auto-encoder (STGAE) to both represent individual “normal” trajectories in a latent feature space (autoencoder) and model agent (car) interactions over time by computing pairwise agent influences (spatio-temporal graph).  Secondly, it uses kernel density estimation on the latent feature representation of the STGAE to approximate the distribution of the normal trajectories.  The proposed method is evaluated against single-agent and multi-agent baselines, on standard metrics for anomaly detection.  The paper also contributes a new simulated dataset for multi-agent driving trajectories, with both normal and abnormal driving maneuvers.
strengths_and_weaknesses: Identified strengths of paper:
+  Addresses an interesting and under-explored problem in multi-robot systems, anomaly detection given joint continuous trajectories; thus it explores the detection/recognition of anomalous behaviors for multi-robot systems, intended for the autonomous driving domain.  I also wonder how well this approach would generalize to other MA problem domains with continuous action spaces?  E.g. multi-agent nonverbal communication or coordination in a shared environment.  If the authors have insights about this, some discussion of it at the end of the paper would be interesting and potentially useful.
+  Provides extensive empirical analysis, which is important for extracting meaningful insights.  Good that the work evaluates methods using several metrics from the Anomaly Detection literature; provides a stronger basis for hypothesis testing.  Qualitative analysis from Table 2 is useful for providing additional insight into aggregate results and also seems like good evidence for differentiating between the value of single-agent versus multi-agent methods on this problem.
+  Contributes a simulated driving dataset to the MA community, with 146 scenes and 11 different types of driving anomalies, which can be a really useful tool for the community in a problem domain of interest to many.


I would suggest that the authors improve their submission, including the following:
-  For all results reported: Would be extremely useful (and provide more compelling evidence) if some measure of statistical dispersion was included (e.g. standard deviation or standard error), so as to assess statistical significance when comparing the different methods.  Currently, this is not shown at all, only mean data.  So it is difficult to assess the reliability of the findings.  Also, how many seeds/runs are these curves averaged over?
-  The paper briefly explains each of the evaluation metrics used, which is really helpful as context.  However, even with that, it's not entirely obvious to me how to interpret each of the result columns in Table 1.  I understand for certain columns, lower/higher is better, but what do the differences in performance imply?  How much of a difference is significant?  Overall, it would be helpful to provide additional explanation/intuition as to how to interpret the findings, particularly for readers not from the Anomaly Detection community.  Also, still unclear as to *why* all baselines are not shown in Figure 3.
-  From a qualitative analysis perspective, it could be insightful to show an example of where the same trajectory for agent i is normal in the single-agent case but can be anomalous in the multi-agent case (dependent upon the behavior by another agent j).  Would also serve as additional compelling motivation for modeling of joint continuous trajectories, which I’m wondering how much more computationally expensive it is than modeling only individual trajectories? Some discussion about computational complexity would be useful.
-  Minor Clarification:  In Equation 5, remember to define p^i_t

summary_of_recommendation: This paper makes a novel methodological contribution to the space of multi-agent robotic or dynamical systems, by applying spatio-temporal GNNs for anomaly detection.  While the work is intended for a specific application domain (autonomous driving), which serves to limit the generality of its findings, the method considers learning in continuous joint trajectory spaces, which contributes to an under-explored part of the multi-agent reinforcement learning (MARL) problem space.  The application domain itself is of high relevance and interest to the MARL and Robotics communities, particularly as the former is moving toward developing methods for more complex domains.  And in the application domain of interest, the paper is able to show that the proposed method benefits from modeling the interactive (multi-agent) nature of the driving problem, not just considering the single-agent problem.  It also provides insights on the types of driving anomalies where the method performs well.  Finally, it contributes a simulated driving dataset, which is a nice contribution to the community.
issues: Listed above as suggestions for improvement.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The authors develop a system for anomaly detection in driver interactions using a novel spatio-temporal graph neural network architecture combined with a Kernel Density Estimator. 

strengths_and_weaknesses: The definition of a spatio-temporal graph and its use in a GNN architecture is a very interesting contribution of the work. 
The authors provide relevant baselines for comparison, and perform several ablation studies. 
The results clearly demonstrate that considering multi-agent interactions in anomaly detection is essential. The characterization of the performance of the system over various segment lengths is also important. 

One shortcoming of the approach is that the kernel density estimator must use all of the training samples. Many of the most popular autonomous driving datasets have millions of examples, and it is unclear how the KDE methodology would scale to larger datasets. 
I would like to see a discussion of how this approach could scale beyond pairs of agents, since the most challenging and interactive scenarios occur in dense urban traffic. 

The current pipeline for anomaly detection could be used for data mining in a larger dataset to find interesting long-tail cases for training or testing components of autonomous cars. I think this system could also be useful for anomaly prediction, or early detection of situations that may become anomalous in the near future. The use of the max-pooling operation in Eq. 8 may allow for this, especially considering the consistent performance across segment lengths in Fig. 3. If this approach shows promise for this task, then the anomaly detection system could be used on-board an autonomous vehicle to detect when the car would need to revert to a human fall-back driver. 

summary_of_recommendation: The paper describes a novel architecture for processing interactive trajectories and detecting anomalies, but the applicability of it to large modern datasets is limited by the use of KDE.
issues: In Table 2, it would be helpful to indicate which are the single agent anomalies, and which describe interaction between agents. 

It would be good to discuss the possible uses & applications of anomaly detection in autonomous driving to further motivate the work.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper makes two contributions: (1) it proposes a spatio-temporal graph auto-encoder for learning nominal driving behaviors, after which kernel density estimation is performed on the latent state to quantify unlikely (i.e., “anomalous”) latent states, and (2) develops a dataset of nominal and anomalous driving behavior with associated human labels. 
strengths_and_weaknesses: Strengths:
- The contribution of a novel dataset of two-agents driving on a highway with associated human anomaly labels is interesting and relevant to the autonomous vehicle community. 
- The problem domain of anomaly detection in multi-agent trajectories is highly relevant for modern autonomous vehicle systems.

Weaknesses:
- The core contribution in this work is “the ability to simultaneously learn multiple trajectories for a dynamic number of agents” and then “perform kernel density estimation on the latent representation of the [proposed approach].” However, this has been extensively studied in the behavior prediction domain, it is not immediately clear why the proposed approach is a better paradigm since there is no discussion or comparison to state-of-the-art approaches which meet both these criteria. Further explanation is described in the detailed comments. 

Detailed comments:

Section 1:
- “...e.g., recognizing a ghost driver…”, please clarify / define what a ghost driver is, as this is not standard terminology. 
- “the main innovation of STGAE is the ability to simultaneously learn multiple trajectories for a dynamic number of agents'' and  then “perform kernel density estimation on the latent representation of the [proposed approach].” This idea has been studied in various ways by industry behavior prediction teams [1, 2] and academia [3]. In the proposed method, kernel density estimation is performed on the latent representation, and the density estimates are then utilized to detect anomalies. In the aforementioned cited approaches, the distribution over the latent state is often directly estimated, and then coupled with the trajectory forecasting method as a means to quantify likely and unlikely behaviors. For this reason, the novelty of the proposed approach should be discussed / compared to the aforementioned methods which capture similar distributions over latent states as well as produce trajectory predictions for a dynamic number of agents. 
[1] Chai, Yuning, et al. "Multipath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction." arXiv preprint arXiv:1910.05449 (2019).
[2] Tolstaya, Ekaterina, et al. "Identifying driver interactions via conditional behavior prediction." arXiv preprint arXiv:2104.09959 (2021).
[3] Ivanovic, Boris, et al. "Multimodal deep generative models for trajectory prediction: A conditional variational autoencoder approach." IEEE Robotics and Automation Letters 6.2 (2020): 295-302.
- The paper does not upfront define what is meant by “anomaly” in this work. Anomalies can take many different forms in multi-agent trajectory forecasting (e.g., is an agent’s true latent state in the hypothesis space of the model? Is the distribution over states well-calibrated? Assuming the probabilistic model is well-calibrated, is an agent performing a sufficiently unlikely maneuver deemed “anomalous”? What constitutes “sufficiently unlikely”?). It is only in Section 3.2 where the idea of anomalies being samples taken from low-density regions of the latent feature space is described. Being upfront about what anomaly means in this work would strengthen and clarify the narrative. Additionally, a discussion on why this notion of anomaly is valuable compared to alternative notions would also be valuable.  

Section 3:
- Conditional variational autoencoder approaches (CVAE) seem well-suited for the proposed approach, as they both produce a conditional probability distribution given a dataset of driving trajectories but also project the inputs into a lower latent space and a distribution over the latent variable. It would be valuable to discuss why the proposed approach -- which performs kernel density estimation on the latent representation of the STGAE -- is different / more beneficial for autonomous vehicle trajectory modelling compared to the CVAE approaches. 
- “To identify if a frame is normal or abnormal we compute the frame anomaly score… max-pooling avoids missing anomalies compared to the mean, which is not justifiable in automated driving.” While it is true that anomalies compared to the mean are not justifiable, looking at the mean when detecting anomalies may not be sufficient, as anomalies are often tail-events. This section would be strengthened with a further discussion on why looking at the mean during anomaly detection is a good measure of an anomaly. For example, why not detect the most-anomalous agent and latent state?
- At the end of this section, it would be valuable to describe what should be done (from an AV design and/or decision-making perspective) once an anomaly is detected. This would improve the impact of this work for the broader AV community that would have to integrate an anomaly detection framework into the vehicle perception/prediction/planning/control stack. 

Section 4.3:
- Please define AUROC, FPR, and TPR prior to using the abbreviation. 

summary_of_recommendation: I recommended weak rejection because although the domain of anomaly detection for autonomous driving prediction is highly relevant, the proposed method is not clearly described and evaluated compared to state-of-the-art trajectory forecasting methods which both predict multi-agent trajectories but also learn distributions over latent states. This paper would have been stronger with a focus on the design and collection of a novel anomaly-centric dataset, since this is something thoroughly lacking in the autonomous vehicle safety community. 

====================
After the rebuttal phase, I have increased my recommendation to "weak accept".

issues: - Clarify how the proposed approach differs from state-of-the-art trajectory predictors (cited in the detailed comments). 
- Clarify why the measure of anomaly studied in this work is a sufficient metric
review_title: null

Meta-review: The paper introduces a novel anomaly detection mechanism for the multi-agent, autonomous driving domain. All reviewers agree that the paper addresses an important and timely problem for autonomous vehicles. The reviewers expressed concerns about the performed
evaluation that should be addressed in the revised version:
- Clarify the results and the inconsistencies represented in the tables and Fig. 3. It seems that the statistical spread in the performance of the proposed method and other baselines is missing. A more detailed discussion of the presented results will strengthen the paper significantly (see reviewers' suggestions).
- Clarify how the approach scales in terms of multi-agent interactions (as opposed to a pairwise interaction) and as the number of data samples increase.
- Discuss how the proposed approach differs from some of the related multi-agent trajectory prediction efforts pointed out by the reviewers. 
- Address the suggested language improvements.

===== Post rebuttal =====

The authors have sufficiently addressed reviewers' concerns; specifically, ablation studies with multi-agent interactions were particularly interesting. For the final version, I recommend the authors to add some of these studies in the main paper. I would also suggest doing a more thorough statistical analysis (3 trials are not that significant to compute the mean/variance of the performance).
Decision: Accept (Poster)