Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This work builds a pipeline for closed-loop planning via behavior cloning.
Their propose to use only past local context (as opposed to SDV state and context in prior work), and use a transformer to predict future states, which is then turned to actions via LQR controller.
They show significant improvements over prior methods on the nuPlan and Lyft closed loop planning benchmark. 
strength_and_weaknesses: ### Strengths

* Experiments show clear improvements over IDM, FLOW, COPO
* Overall mathematical notion is for the most part clear and well-motivated

### Weaknesses

* The main motivation seems quite counter-intuitive at first glance. Taking out ego-state seems like it would harm performance.
* Overall writing could be improved - several sections read off like lists.
* Figures could use some polish and better descriptions.
* nuPlan baselines are **extremely** weak which raises some concern.

### Minor Issues / Questions

* In table 2: it looks like w/o regularization is quite strong.
* How does the method perform without the LQR controller?
clarity,_quality,_novelty_and_reproducibility: **Clarity, Quality:** the results look great but the contributions are not completely clear
**Novelty:** novel in terms of theoretical analysis of closed-loop planning
**Reproducibility:** code is provided and I believe it is reproducible
summary_of_the_review: This work introduces a fairly counter-intuitive but effective approach and show significant improvements on closed-loop planning. However, I believe the paper does not have the appropriate amount of polish for the venue.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents CCIL, a novel behavior cloning approach that learns a driving policy from human demonstrations. It adopts an ego-centric scene representation with a transformer-based architecture. The network predicts a sequence of future ego SDV states, which are converted to control actuations by a LQR. CCIL is evaluated in closed-loop planning driving benchmarks Lyft and nuPlan and attains state-of-the-art performances on both benchmarks.

strength_and_weaknesses: ### Strengths
- The paper is well written and the presentation is clear.
- The proposed method is effective and achieves state-of-the-art performance. This is the strongest aspect of this paper in my opinion.

### Weaknesses
- I am not sure how helpful the theoretical analysis is. The assumptions in equation 3 and 6 which linearizes the state and policy does not match the real world or actual implementation of methods. I am also curious why the example at the end of section 3 particularly chooses the formulation of “u_t=s_t + K_bc s_t”.
- I am afraid that the proposed method has limited novelty. Using an ego oriented coordinate space for observation and predicting future states has been explored in behavior cloning for driving policy learning before [1,2] and is largely a standard practice in motion forecasting [3,4] (although the latter also contains ego state information as opposed to CCIL). 
- I would like to see a latency analysis of the whole system as well as individual components of the methods, and how it compares with the baseline methods.
- I would like to see an ablation where the causal transformer encoder is trained to predict only the current step’s state predictions instead of all H steps.

[1] ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst, Bansal et al.

[2] TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving, Chitta et al.

[3] MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction, Chai et al.

[4] Diverse Multi-Future Prediction and Planning for Self-Driving, Cui et al.

clarity,_quality,_novelty_and_reproducibility: Clarity: great, the paper is well written and the presentation is clear

Quality:good, attains SOTA numbers

Originality: not too great as mentioned in the above section

summary_of_the_review: The paper presents CCIL, a behavior cloning approach for driving policy learning. Although the presented approach is effective and attains state-of-the-art performance on both the Lyft and nuPlan benchmarks, aspects/main claims of the paper are not novel and heavily explored in driving policy learning/autonomous driving. Hence I do not recommend acceptance of the paper at the moment.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposed an offline method to overcome the covariate shift issue in imitation learning. A context-conditioned imitation learning method was proposed, which learns a policy to map context state into  ego vehicle's state without any history ego vehicle's information. To apply the method to urban driving, an ego-perturbed goal-oriented coordinate system was implemented, which helps to reduce the implicit ego information. Finally, model performance was tested on Lyft and nuPlan datasets with several baselines, which demonstrates proposed method can outperform all baselines. 
strength_and_weaknesses: Strength:
This paper proposes a novel offline learning method and a new coordinate system representation, which aims to mitigate the covariate shift problem. The proposed model outperforms the state of the arts in both Lyft and  nuPlan datasets in multiple matrics including collision, off-road, discomfort and L2 position errors with ground truth. In addition, the video and code are released, which will benefit the research of imitation learning. Besides, the paper is well written and has clear framework. In summary, the strength are novelty, performance and opensource.

Weaknesses:
1. Figure
Figure1:
Figure1 is not elegant, impressive or fresh. The authors really need to pay more attention to Figure1 and spend much more time on it.

Figure2:
I do not understand which method the authors choosed as "BC" method in figure2. The authors should explain more about Figure2 in details.
I do not understand why only one "BC" method was selected as comparison with proposed method in toy experiment either.
In Performance Evaluation the authors compared with three methods including Raster-perturb, BC-perturb and UrbanDriver. At least it is required to show the closed-loop trajectories of all these models.

2. Benchmark
Benchmark should take the state of the art offline reinforcement learning algorithms such as Conservative Q-Learning(CQL) into consideration, which makes the result more convincing.
clarity,_quality,_novelty_and_reproducibility: This paper is well written and well organized. The idea that removes all history ego vehicle's information from input to overcome covariate shift problem is full of novelty. I believe there will be more applications of this idea in the future.
summary_of_the_review: Though there exist some problems in Figure and Benchmark, this paper is still considered as full of novelty and beneficial to the field. The paper is marginally above the acceptance threshold.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose a method based on the Past Motion Dropout hyperparameter from ChaufferNet, the behavioral cloning approach to autonomous vehicle motion planning. With the same motivation as ChaufferNet authors, they suggest to set the dropout parameter to 100%, such that neural network policy does not have access to past ego states. Similarly to ChaufferNet, they implement initial state perturbations as well. To compensate for the lack of temporal smoothness, the authors add a linear quadratic regulator. The authors also adopt a common Transformer-based architecture.
strength_and_weaknesses: Strengths:
- the paper is well written and is easy to follow

- it is great that the approach is evaluated on a publicly available planning benchmark

Weaknesses:
- there is no statistical significance evaluation of both the main comparison table and the ablation table. Planning benchmarks are known to have high variance, and it is not clear how training and evaluation randomness affect the results. I suggest the authors follow reinforcement learning experiments best practices, e.g. [1]

- baseline selection: it is explained how the model architectures differ and how many parameters they have. Would be great to add a column with numbers of parameters for every model in Table 1, for example:

|Benchmark|Model | Num params | Comment |
|:--|:--|:--:|:---:|
|Lyft|Raster-perturb | 25M | based on ResNet-50 |
|Lyft |BC-perturb | 2M | based on github colab |
|Lyft | UrbanDriver | 2M | or 3.5M in the paper, not clear |
|Lyft | CCIL | ? | ? |
|nuPlan|Raster | 25M | based on ResNet-50 |
| nuPlan |Vector-perturb | ? | |
| nuPlan |LaneGCN-perturb | ? | |
| nuPlan | CCIL | ? |  |

Based on the open-source code of UrbanDriver, it looks like BC-perturb and UrbanDriver models are small compared to ResNet-50 based Raster-perturb. Depending on the number of parameters in the proposed method, these approaches could be not directly comparable.

[1] Deep Reinforcement Learning at the Edge of the Statistical Precipice, NeurIPS 2021
clarity,_quality,_novelty_and_reproducibility: The novelty as limited as Past Motion Dropout was proposed in ChaufferNet.

The empirical evaluation is done in closed loop simulation only, and is not convincing. ChaufferNet showed that models trained with Past Motion Dropout are able to be deployed both in simulation and in real world. The proposed approach suggests that ChaufferNet policies could be trained without ego history, but only offers a limited evaluation in simulation.

The novelty of the Transformer architecture used for planning is also questionable, as it has been adopted in both motion forecasting and motion planning tasks.

Question to the authors: I did not understand what happens with the future ego trajectory after an initial state perturbation is applied. Is it updated as proposed in ChaufferNet?
summary_of_the_review: My suggestion is borderline due to minor novelty compared to ChaufferNet and the lack of real world testing. It is not clear how many parameters the proposed approach and selected baselines have, and if the results are statistically significant.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This work builds a pipeline for closed-loop planning via behavior cloning. The idea of setting the dropout parameter to 100% is not a new idea (suggested by ChaufferNet) and thus the methodological contribution is rather small. However, given the performance boost, I personally don't think "small" technical contribution is not an issue and also it is worth sharing with the community. The decision is still reject, mainly due to give one more polishing iteration in terms of visualization+writing, but more importantly analysis on where/why the performance gain is happening. This type of analysis can make up the lack of technical contribution.
justification_for_why_not_higher_score: The authors addressed many concerns of the reviewers. However, everyone (reviewers + AC) still thinks that the paper can benefit from one more iteration of experiments and validation.
Decision: Reject