Venue: CoRL 2021 Conference

Reviews:

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper introduced a risk-aware motion planner for autonomous vehicles that learns from expert demonstrations. The learning structure was called LogicRiskNet. In particular, the learned trajectories are guided by temporal logic rules. In addition to evaluating the risks of surrounding vehicles, the learned structure is able to explain why a surrounding vehicle is in a risky situation. Finally, the developed approach was evaluated using a dataset called NuScenes, which was selected due to its rich semantics. The proposed method was compared with a few baselines, including naive behavior cloning, and optimization methods. Results showed that the developed approach produced high success rate, and shorter time to navigate to the goal. 
strengths_and_weaknesses: The main concern is that autonomous driving is a highly interactive domain, but the paper used prerecorded data to simulate the interactions. The differentiable logic-based risk metric is fine, because it doesn't involve any changes to the world (simulated or not). It's becoming problematic when evaluating the proposed control method (risk-adverse and risk-seeking behaviors). It does not make sense to evaluate the performance of a vehicle assuming the surrounding vehicles are not responsive at all. This issue was briefly mentioned in the "Methods of evaluation" paragraph, but it's pointing to future work to address this issue. It's unclear why simulators like CARLA was not used in the paper. 

The other concern is that Sections 2 and 3 were not presented in a way that's friendly to readers who are not very familiar with the relevant mathematics. The reviewer knows basic knowledge representation of AI, and has some experience about LTL, but still find it difficult to follow the symbols and equations. It's good that there are transition sentences that summarize the technical details, which helped the reviewer follow the whole paper. Still, it will be good if the paper can be presented in a way that's mathematically less heavy. 

There were only two formulas used in the evaluation, though the developed approach is supposed to be generally applicable to logic formulas. Also, the explanation part of the work was not evaluated. 

A few minor issues: 

 - Figures show up before their references in the body of the paper. For instance, Figure 2 is on page 6, but it was not mentioned until page 7. Figure 1 is on page 2, and its first mention is on page 4. 

 - Font size in figures is very small, and the embedded text is hardly readable. 

 - Please explain what "ado agent" means the first time it's used. The reviewer couldn't figure it out until reading Figure 1. 

 - The paper has many sentence statements that start with "The authors of [X]..." In general, a reference is to support a statement, but should not be a statement itself. 
summary_of_recommendation: The logic-guided risk evaluation part of the paper (LogicRiskNet) is well motivated, and makes a lot of sense. But the evaluation part has a control component, and was evaluated using a prerecorded dataset, which is questionable and not convincing. 
issues: See above.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes a framework to learn a motion planner (a parametrized distribution of trajectories) that takes into account the risk ok other agents. The proposed approach is to design a risk monitor that can process previous trajectories and compute a risk score to each of the ado vehicles. The monitor is then used to learn the motion planner. The proposed framework is then evaluated using the NuScenes dataset. 
strengths_and_weaknesses: Strengths:
- The motivation behind using risk as a factor in motion planning is interesting.
- The use of ptSTL leads to a formal framework to reason about and quantify risk

Weaknesses:
- The paper lacks several technical details which makes it hard to follow. In particular, the discussion on LogicRiskNet in Section 4 is vague. Examples are "If we use softmax approximations .... we can construct a computation graph ....". It is not clear what the authors are referring to in this paragraph. A more detailed construction is needed to understand the constructions that are verbosely described in this Section.
- One hidden assumption is that previous behavior of ado cars is a good predictor of their risk in the future. The paper failed to explain why this is indeed the case. 
- The evaluation section is based on simple hand-picked risk rules. It is not clear how the performance of the proposed framework is affected by the choice of these rules. 
summary_of_recommendation: The idea of using ptSTL to capture risk and design a risk monitor is interesting. Nevertheless, the paper is lacking deep technical discussion about several of the key components of the framework.
issues: - The authors should provide an empirical evidence of why historical data provide a good predictor of the future risk. The authors may use the same dataset used in evaluation or any other dataset to provide such evidence.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: In this paper, the authors introduce LogicRiskNet, a form of differentiable temporal logic. 
The parameters of LogicRiskNet can be learned from a set of demonstrations.

LogicRiskNet incorporates a measure of the risk connected to the agent's behavior. Furthermore, the logic encoded by LogicRiskNetwork, allows to interpret and explain the agent's choices. 


strengths_and_weaknesses: NOTE: as I declared in the "Reviewer Expertise", my knowledge in the area is limited (I do not have direct experience with temporal logic, and with inverse reinforcement learning or control). 
I think that I understand the general idea of the work and the logic that connects each step. In this sense, I think that the paper is well written. I am however uncertain about the usage of the terminology "risk", and it seems that the algorithm is applicable to a small set of tasks, and this is not reflected in the abstract. 

__Strenghts__

The paper presents a method to mimic the behavior of a demonstrator based on his risk awareness. 
The model provided relies on logic and therefore is interpretable. Furthermore, the model is made differentiable by replacing classic $\max$ and $\min$ with their soft version. The differentiability allows the model to be trained with classic gradient-based techniques. 

__Weakness__

I am not sure that I understood correctly some parts on of the paper. Risk, according to my knowledge, can be seen as a modification of an original distribution $p$, so to give more "probability" to rare "bad" events (risk-averse) or rare "good" events (risk-seeking). Common measures are 
- Value at risk (VaR) (returns the value at a given quantile)
- Average value at risk (AVaR) (returns the average of the distribution from 0 to x quantile)
- Entropic value at risk (EVaR) (returns the average under a modified distribution $q$ such that $\max_q \mathbb{E}[X]$ with $D_{\text{KL}}(q||p) <= const$).

Equation 4 presents a sort of "potential field", and it is related, if I understand correctly, to the distance of two vehicles. The risk measure presented, does not seem to be really a risk measure. Perhaps the name is deceiving. 
Does this "risk" measure apply therefore only to autonomous driving or to more general problems?



In general, the paper seems to be focused only on the particular case described in the example in Figure 1. From the abstract, it seems that the approach is far more general than the autonomous driving example, and autonomous driving has only been used to test it. However, from the paper, it seems that the algorithm is appositely designed for autonomous driving.





summary_of_recommendation: In general, the paper seems reasonable and well done. However, the writing is a bit deceiving.
It is not clear what the risk measure is. And it is not clear if the method is specifically thought of for autonomous driving. 
If so, I'd recommend the authors specify that more clearly in the title and perhaps in the title too.

UPDATE
==

I want to thank the authors for their response. I appreciated the changes in the text. In my opinion, the exposition can be still enhanced, and I still think that the authors could have explained better how to generalize their approach, (i.e., lines 53 and 54 tell us that you can generalize, but not how you do that).
In general, my evaluation of the paper is positive, and I keep my score unchanged. 
issues: As already remarked, I believe that the authors must better clarify 
- what is the risk measure considered (and why it is called risk measure). Does it only apply to autonomous driving or to more general set-up?
- how could one generalize the algorithm to a wider scenario? Is it possible? (for me it is fair that the algorithm focuses only on autonomous driving, but then the motivation of the work should be better specified).
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This work presents an architecture for learning to predict risk associated with other agents in a multi-agent setting (with focus on autonomous driving) in order to produce a risk metric to be used in planning. The architecture is based on a differentiable approximation of a temporal logic description of risks. The method takes a set of logic formulas which describe safe behaviour in terms of features, and is trained via imitation learning to produce risk management behaviour similar to an expert demonstrator. The learned risk model is interpretable. The method is compared against the human expert and 2 other approaches in an autonomous driving simulation, and is closer in performance to the expert driver than the existing approaches.
strengths_and_weaknesses: Strengths:
- Novel and interesting approach
- Applicable to a number of important robotics problems
- Results show a good improvement compared to existing methods

Weaknesses:
- Some parts of explanation could be made clearer
summary_of_recommendation: The quality of the approach and results appear to be good. Presentation clarity is mostly good, with a few areas where clarity could be improved which should be revisited.
issues: - Line 17 - turning across traffic rather than turning left may be clearer for international readers
- Line 38 - should be ‘risk-averse’, not ‘risk-adverse’; same on line 39, 50, 238
- Would be good to briefly introduce 'ego'/'ado' terminology as this doesn't appear to be commonly used in other work
- Possible inconsistent use of \omega}_T between equation 2 and equation 3 - is the same symbol representing expert trajectories in one equation vs agent generated trajectories in the other?
review_title: null

Meta-review: In this paper the authors, by leveraging tools from temporal logic, design a framework to learn risk-aware trajectory planners. The framework is evaluated on a real-world driving dataset.

In general, the reviewers agree that the proposed idea is novel and useful, and I concur with this assessment. However, they also point out some weaknesses, including some lack of details in the exposition of the technical approach and a somewhat weak experimental evaluation (e.g., the experiments only consider two, rather simple logical rules).  The authors should carefully address the reviewers' concerns in their rebuttal.

UPDATE POST DISCUSSION PHASE: I would like to thank the authors for their comments and clarifications during the discussion phase. The reviewers generally agree that this paper presents an interesting idea and provides a useful contribution. On the other hand, some aspects of this paper still need to be improved - in particular, in terms of its exposition clarity (for example, the authors should better discuss to what extent they leverage the methodology presented in [14]). The decision is to accept this paper, but the authors should carefully address all reviewers' comments in the final version of this paper, with a key focus on improving exposition clarity.
Decision: Accept (Poster)