Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents an approach to safety-aware trajectory planning in the presence of both seen and unseen agents, with a focus on simulated self-driving scenarios. Building upon earlier work, in which an RNN is used to estimate trajectories. In the (previous) approach, future trajectories of the human agents are sampled from a normal distribution trained from data collected offline and the trajectories of the agent are chosen via optimization with respect to those sampled trajectories. In this work, the authors train the system in such a way that the other "human" agents may not always be visible and so predictions about the true state of the system must incorporate that other agents may be in the scene yet not currently observed. The proposed agent is capable of understanding the impact unseen agents may have on their planning in a handful of common driving scenarios, improving performance with respect to baseline approaches that do not consider this term.

strengths_and_weaknesses: Overall, this paper has an interesting (if somewhat small) idea about how to incorporate unseen agents in decision-making. The main contribution seems mostly to be the change in the form of the data and the training process, which now takes into consideration that not all vehicles will be observed at all times and so the unobserved state should not be directly penalized; instead, the latent state used for forward simulation of trajectories during planning consumes the partial state. While this approach is novel, the contribution seems fairly small. The application domain, however, is fairly important and understudied, offsetting this.

The prose in the paper is very clear. However, the paper was hard to follow in places, due primarily to a notable omission: the pseudocode describing the planning procedure exists only in the supplementary material, yet was important for understanding the method. Without it, the paper is not particularly self-contained and so the authors must spend more time describing what this approach looks like in the main text or move the pseudocode into the text; while the equations listed in the text are enough to define the optimization objective, it is a big leap from following along with those equations. Reading through the pseudocode was essential for my understanding of the proposed approach.
Adding in the pseudocode would greatly help understanding.

Smaller questions and comments for clarity.
- Watching the overtake videos in the supplemental material, it appears that (to at least a certain extent), the ability of the agent to succeed during the overtake boils down partially to luck. There does not appear to be sufficient time to successfully overtake if the oncoming car were to arrive sooner. Have the authors experimented with the timing of the arrival of the oncoming car and do they expect this would change the results? (There is a balance here between seeking opportunity and behaving conservatively, and I still believe the results to show relative improvement to the other results. My question touches upon the agent's desire to prioritize safety: is this a tunable parameter, or would it require changing the training data?)
- The experiments seem to be broken into "chunks" of 15, since every result for all planners succeed in 15, 30, or 60 of the 60 trials. Relatedly, is each trial within each experiment identical or are the trials randomized somehow? The authors should include clarity about how these trials are run in the results.

robotics_focus: Relevant but unlikely to deploy to hardware in near future
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: While this approach is novel, the contribution seems fairly small. The application domain, however, is fairly important and understudied. The central issue with the paper as it exists right now is in its clarity, and the overall approach was difficult to fully understand without multiple full read throughs of the paper and reading the supplemental material.
issues: The central issue is the clarity of the paper's description of the overall approach, mainly the omission of the planning pseudocode. As mentioned above, though what is included in the paper is clear, it was difficult to discern how everything came together during planning; the pseudocode helped my understanding of this, though there are perhaps other ways to communicate these ideas as well.

There are other small questions above that, if included, would also help clarity.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The authors propose Active Visual Planning (AVP), a data-driven approach for automated driving that uses context derived from perceptual observations (specifically, lidar scans) along with past vehicle waypoints to predict the distributions of future states for both an ego vehicle and non-ego vehicles, enabling contingency planning which is capable of addressing the partial observability that arises in common driving scenarios. Learning occurs offline, using a dataset of recorded observations, and hand-coded deterministic policies reflective of human driving behaviors. Per the study conducted using the CARLA simulator, AVP permits the scenarios of interest to be handled with increased safety, relative to single and multi-agent models that plan only according to the information that is observable. The authors claim that AVP is the first fully-learned planner that supports active contingency planning over perceptual uncertainty.
strengths_and_weaknesses: + The paper addresses a compelling, well-motivated topic relevant to robot learning, proposing a novel method for predicting and acting upon the existence, positions, and intentions of other vehicles on the road under partial observability, to safely complete key automated driving tasks.

+ The paper is well-organized and clearly written, the code for AVP is made available, and the supplemental details provided on the accompanying website are helpful.  

+ AVP is compared with three other suitably chosen methods, which include simpler, less conservative frameworks (single and multi-agent models that plan only according to the information that is observable) and a game-theoretic framework that considers all of the possible outcomes of the driving scenarios of interest. 

+ The driving scenarios selected for the experimental study are well-motivated scenarios that arise in everyday driving, which also present safety hazards for human drivers. AVP is shown to address these driving scenarios with superior rates of success relative to the selected baselines. 

- It would be helpful to provide more detailed, quantitative results characterizing the plans generating by the four competing methods, as well as to show, qualitatively, what the plans look like that were produced by the three baselines selected.

- The paper would be strengthened by providing clarification on the extent to which the driving scenarios encountered in the training process differed (or did not differ) from the scenarios encountered at test-time. This would lend support to the claim made in the paper that AVP is flexible and can be easily transferred to new scenarios. This is especially important because of the additional claims made that one of the baselines (SOAP) requires significant setup for each individual scenario.

- The paper would also be strengthened by speculating on the pathway by which the planners learned for specific driving scenarios could be integrated into a single unified framework for use in real automated driving. 

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: In AVP, the authors have contributed a novel and well-motivated data-driven framework for prediction and planning that is of potential benefit to complex and hazardous automated driving scenarios, offering improved performance with respect to successful task completion. The paper would be strengthened by providing more implementation details, a more detailed exposition of the experimental results, and evidence for the flexibility/transferability of AVP that is claimed by the authors. 

Post-Rebuttal Comments: I appreciate the revisions and clarification provided by the authors, and I remain in favor of acceptance. Taking the authors' responses and the CoRL review criteria into account, I have decided to keep my rating of weak accept.  
issues: - The authors should provide more evidence to support the claim “our method can be easily transferred to new scenarios”. Perhaps they can do so by providing clarification on the extent to which the driving scenarios encountered in the training process differed (or did not differ) from the scenarios encountered at test-time. For example, what was different about the ego and non-ego trajectories, environments, etc., encountered in training vs. testing. 

- Along these lines, the statement “further details can be found in the supplemental material” in the section on Model Architecture and Training gives the reader the impression that more details on the training procedure will be found in the supplement, but there are no additional details provided.

- The term “visual context” is a little confusing, since vision is not one of the sensing modalities being used in this work. Perhaps “perceptual context” could be used instead for greater generality, or “scan context” for greater specificity. If visual context is a term of art commonly used to describe this, feel free to ignore my suggestion.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper develops a planner for autonomous driving in a multiagent environment. In particular, the paper contributes a method for addressing an imperfect perception system (e.g., sometimes doesn't detect other agents). A learned model of detection of other agents is fed into a planner that aims to compute collision-free paths that reach the goal quickly. Experiments in CARLA simulations of 4 traffic scenarios suggest the proposed method performs better than three prior approaches.
strengths_and_weaknesses: Strengths:
- Important and difficult problem, relevant to this community, well-motivated
- Modeling partial observability as a Bernoulli variable of whether an agent will be detected makes sense
- Incorporating this model of perceptual uncertainty into the planner makes sense

Weaknesses:
- Results section has a lot of words but not much substance. Only a high-level comparison is given on 4 relatively clear scenarios. Since perceptual uncertainty is such an important factor, this section would have been much stronger with more experimental analysis of the detection model, how much uncertainty the detector/planner can handle, how realistic these uncertainties are (maybe using real driving data), etc. Issue with Table 1 described further below.
- Block diagram in Figure 2 should be re-structured. What is the overall output? Where does the planner connect? I don't understand this diagram.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are not well addressed
summary_of_recommendation: Overall this is a cool problem and the approach seems promising, but the experimental results do not provide sufficiently deep insight into what's really going on here.
issues: - Table 1 should be re-generated in a more sensible manner. RG and RG* are super vague concepts that throw away a lot of information here. The important metrics here seem to be reaching the goal quickly and not colliding despite the imperfect knowledge. I suggest reporting how each algorithm does on these metrics (and/or other metrics), and then incorporate a human expert either as a normalization factor (e.g., time to goal relative to human expert) or as another "algorithm" baseline row. Also, since every algorithm gets 15, 30, or 60 out of 60, it suggests there are a few discrete jumps in the difficulty of the scenarios --> this should be elaborated on because it seems unlikely to be a coincidence.
- Additional experiments should be done (e.g., suggestions above) to provide a deeper level of analysis/insights into how uncertainty and the planner interact.
- Block diagram should be improved per comments above.
- The results section claims AVP scales to scenarios with additional agents, but the experiments show a jump from 2 to 3 agents. This is not a strong demonstration of scalability.
- Modify overclaiming/inaccurate statement in the conclusion "Because AVP is learned purely from observations, it does not require expert action labels or domain knowledge (such as state-space discretization or specification of an approximate dynamics model), making it easily generalizable to new environments and scenarios", since "we use hand-coded deterministic policies for all agents, which are designed to generate all plausible modes of realistic driver behavior in the scenario."
- It may be worth adding a reference to the work of Charles Richter (papers circa 2015) that deal with planning conservatively around blind corners (a related form of perceptual uncertainty)?
- The limitations section is not being used as intended. The first sentence describes an advantage of the approach. The third sentence describes future work. Surely there are more limitations than a lack of interpretability.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes an imitation learning like method for autonomous driving. The method outputs contingency plans based on maximum likelihood estimation of ego-agent control parameters in a learned probabilistic multi-agent interaction model. The main novelty of the work is to extend the previous PRECOG model (Rhinehart et al., ICCV 2019) by modelling partial observability via detection events. The method is evaluted in three driving scenarios with the CARLA simulator, showing improved performance over baslines using PRECOG or other alternative approaches.
strengths_and_weaknesses: ** Strengths

- Addresses a highly relevant and interesting problem for autonomous driving: reasoning about future uncertainty and presence of unseen traffic participants, and using this for planning ego-motion.
- Addressing partial observability via modelling detections is interesting, and has the advantage of avoiding need for ground truth positions, which the paper well argues for.
- Paper is overall well written and organized.

** Weaknesses

- The evaluation is rather underwhelming. It focuses on the aggregate performance in the three benchmark tasks, but does not provide any insights into qualitative behavior, or any ablations on the proposed model.

- Especially more effort is needed to evaluate the key novelty of the work, modelling of partial observability. It would be interesting to see how well the model succeeds in predicting future uncertainty, which is key to taking appropriate active sensing actions to potentially reduce that uncertainty. It is not clear what knowledge is represented or how it could be examined/introspected, an empirical demonstration would be welcome. Interpretability is important to have in the target application, but is limited in the proposed method (as is appropriately acknowledge in the paper).

- I also wonder whether the benchmark tasks are too easy, as they are solved perfectly by the proposed method? This gives little clues as to the limits of the proposed method.

- Data association (DA) is an issue in cluttered traffic scenes or with a large number of agents: which measurement originated from which target? DA is assumed solved in the paper, and the detection modelling approach relies on this assumption. It seems unlikely the method can handle complex traffic scenes.

- It seems doubtful the proposed method will scale to more than a handful of agents. Firstly, there is the problem of obtaining training data. The paper is not very clear on how this is done, but Lines 232-233 indicate a substantial amount of manual effort. Second, the paper does not say explicitly, but being based on inference in a graphical model, one could expect scaling to be exponential in the size of the model (but please, correct me if I am mistaken). To be fair, this is a common issue in other methods as well but would be worth commenting on.


** Other comments
- The section on POMDPs and belief space planning is otherwise accurate, but it misses the significant stream of sampling-based Monte Carlo tree search inspired methods, see e.g., POMCP (Silver & Veness, NeurIPS 2010), DESPOT (Somani et al., NeurIPS 2013), and POMCPOW (Sunberg & Kochenderfer, AAAI 2018). These methods have far greater scalability than point-based solvers, and are sometimes applicable in continuous problems. These should be discussed as well.

- Line 102: domain of x_t should be R^{(A+1) x d} to account for ego agent

- How training data is collected is not quite clear (Lines 232-233), this should be properly explained in the paper or supplementary material.

- The paper makes some claims about "closed loop optimality" in the introduction. However, by Section 4.2. it turns out that what is meant is ability to reach human performance (fine, but that's something different). The terminology should be revised, I don't think it is reasonable to claim optimality without a clearly defined reward function.

- What is the relation of this work to methods such as PHD filtering (e.g., Vo & Ma,"The Gaussian mixture probability hypothesis density filter" IEEE Transactions on signal processing)? Is it fair to say the prediction part method approximates a PHD filter with known data association?

**Added after author response**: I suggest the authors also consider the related work Cunningham et al. "MPDM: Multipolicy decision-making in dynamic, uncertain environments for autonomous driving", ICRA 2015, which also models agents with distinctive closed-loop behaviors.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are not well addressed
summary_of_recommendation: This is a work with some novelty (extending PRECOG to partially observable case), but this novel contribution needs a more thorough evaluation. The appropriateness of the evaluation should be checked, and some further insights into what the method has learned or how the knowledge can be examined would be good to have.

**Added after author response**

I thank the authors for their reply. I have also seen the revised paper and supplementary material. This paper has interesting ideas, most notably the approach of modelling detections instead of ground truth positions of other agents. The author response also addresses all the points raised in my "other comments" part well.

However, in my evaluation, the following negative points outweigh the positives:
- Underwhelming evaluation. The evaluation in the main paper simply reports aggregate performance over three scenarios, with perfect scores for the proposed method. This falls short of demonstrating what the limitations of the proposed method are, what are the aspects that are crucial for the proposed method to work (ablation studies), or how it performs with respect to objectives such as maintaining a multi-modal belief over intentions/existence/trajectories of other agents, or actively taking actions to reduce future uncertainty. The revised supplementary material adds qualitative examples, but in my opinion these are key aspects of the proposed method and should be evaluated more thoroughly.
- Lack of scalability. The training data collection requires a manual definition of a control policy for all agents. All combinations of such policies must be present in the training data. Even with 2 distinct behaviours per agent, for n agents the method requires data from 2^n scenarios. This should also be addressed in the limitations, and I agree with Reviewer ec4g that this section can be improved.

Therefore, I am still leaning towards reject.
issues: I would appreciate commenting on/addressing the weaknesses and other comments mentioned above.
review_title: null

Meta-review: metareview: The paper addresses the problem of trajectory planning in scenarios where some of the agents may be partially observed. The central idea in this paper is learning of a latent representation that via forward simulation can explain missing observations from agents that may not be observed at all times. The work builds incrementally on a closely related prior work and extends it to the partial-observability setting. 

The authors present results on simulated self-driving scenarios and provide comparison with a benchmark game theoretic approach. The primary weakness of the paper lies in the results section. Reviewers expressed their concern that the results could have better explored the nature and extent of uncertainty that the proposed method can handle. Further, a more detailed technical exposition for the planning component was required for the paper to be self standing. 

During the rebuttal phase, the authors provided clarification on the experimental setup and relationship with other alternative approaches. The reviewers appreciate the inclusion of the pseudo-code in the manuscript and the additional experimental details. However, it is still felt the experimental evaluation remains a weakness of the paper. In particular, the results are very good on the specific scenarios provided. It is less clear how the approach would generalise to other diverse scenarios. Further, authors are requested to additionally consider providing ablation analysis to better situate the contribution of individual components of the proposed approach. 

confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)