Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This work develops an model-based imitation learning pipeline for autonomous driving.
Their observation network encodes multi-view camera images and route information into a latent embedding which is then used in a variational generative model to predict future BEV semantic segmentation and expert actions.
To my knowledge, this is the first work to show the effectiveness of world models in autonomous driving simulators.
strengths_and_weaknesses: ### Strengths
* Clear writing, technical portion + model architecture easy to understand, since this work seems like FIERY but instead predicts driving actions.
* Their model, which uses cameras only, shows significant improvements over even lidar-based methods.
* Supplementary video shows fairly good short-term predictions.
* I agree with and am glad to see the discussion of driving score - the score is definitely biased towards certain types of routes and adding the reward/normalized reward as another source of quantitative evaluation is useful.

### Weaknesses
* The architecture figure looks more complicated than the architecture actually is. I would suggest adding a simpler toy figure in place of Figure 1, moving the current figure to the technical section.
* I would like to see a bit of discussion or related works on trajectory forecasting. It would be beneficial to discuss some of the difference between world models and trajectory forecasting since there are many recurring themes between these two areas.
* Adding dataset size to Table 1 would be helpful for comparison.
* It seems like a few important experiments refer solely to the supplementary - I'd suggest some spacing edits to try to fit these into the main paper.
questions: * Does this work have a submission to https://leaderboard.carla.org/leaderboard/? This is important to my final rating.
* Related to the trajectory forecasting point - is there any particular reason why rasterized BEV labels were used for future prediction rather than detections (i.e. boxes)?
* BEV labels would require auto-labeling or human annotations - how much driving data is needed for a "good enough" world model?
limitations: yes
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper presents an offline imitation learning approach that simultaneously predicts the evolution of the environment and imitates the expert driver. The proposed approach uses camera-based observations and a semantic segmentation map to build a world representation. The camera images are lifted to 3D and combined into a BeV representation. The environment evolution is performed in a compact latent space. The proposed method achieves better performance and generalization than prior work on a CARLA benchmark.
strengths_and_weaknesses: Strengths:
* The paper is well written and the ideas are easy to follow.
* The problem is well-motivated and the literature review does a good job at contextualizing the paper in prior work.
* The idea behind end-to-end feature learning, while incorporating geometric bias is intuitive.
* Overall, the empirical evaluation of the method is extensive and convincing. The analysis and discussion is thoughtfully constructed, and the different components of the architecture were thoroughly ablated.
* The figures are informative and effectively illustrate the proposed approach and results.

Weaknesses:
* The sentence in the abstract is a bit strong 'So far, such world models have been shown to be highly effective at solving games, but only in simple visual environments with little interaction among agents.'. There has been lots of work in the trajectory prediction setting that builds world models in dynamic environments (e.g., [1,2]).
* The related works section is missing a discussion of trajectory prediction methods which are closely related to imitation learning as presented in the paper.
* One of the motivations for the imitation learning approach over RL is the difficulty in constructing rewards. However, in the evaluation, the most effective metric is determined to be cumulative reward. The considered baselines do not seem to include an RL approach with access to the same input information as MILE.

[1] Zhao, Hang, et al. "TNT: Target-driven Trajectory Prediction." Conference on Robot Learning. PMLR, 2021.
[2] Salzmann, Tim, et al. "Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data." European Conference on Computer Vision. Springer, Cham, 2020.

The paper needs to be proofread for typos. The following is a non-exhaustive list of the typos I found:

1. The BeV acronym is defined in line 30, but not consistently used after that.
2. Line 106: 'a the full description' should read 'the full description'.
3. Line 168: 'indicating the agent where to navigate' should read 'indicating to the agent where to navigate'.
4. Line 234: 'as it obtained' should read 'as it is obtained'.
5. The references should be proofread (e.g., to ensure the year is not entered twice in a citation, the conference venue is listed instead of ArXiv when available, etc.).
questions: In addition to addressing the above listed weaknesses, I have the following questions for the authors. Assuming these questions and concerns are addressed, I am currently inclined to raise the score to an accept.

1. How would the BeV semantic segmentation labels be achieved without LiDAR in a non-simulated setting?
2. What is the explicit definition of the lift operation in line 157?
3. In lines 209 and 210, it is stated that the RL expert was trained using privileged information (e.g., BeV semantic segmentation). However, my understanding was that the proposed model is also trained on BeV semantic segmentation. What is the privileged information available to the RL expert, but not the proposed model?
4. Are any of the considered baselines RL agents outside the expert? One of the motivations behind the proposed method is the ability to avoid constructing a reward. However, the cumulative reward is a key metric of evaluation for the proposed method, thus a comparison to an RL agent trained with this reward while having access to the same input information as MILE would be worthwhile.
5. Is higher better for the infraction penalty?
6. What makes the recurrent approach faster in Sec. 5.4? Were the models for the two evaluation methods trained differently?
limitations: The authors have listed some limitations as part of future work. Some potential additional limitations to include are: imitation learning methods can be subject to adversarial attacks and the model does not have safety guarantees.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a predictive model of visual data for the task of imitation learning in CARLA. The model is trained to forecast RGB images, semantic segmentation images, and expert actions in the variational inference framework. The proposed method incorporates several new components, including a 3D geometry inductive bias that leverages camera intrinsics and extrinsics, as well as a variation on training called “observation dropout”. Experiments illustrate the effectiveness of the proposed approach on several towns in CARLA, relative to other recent approaches. The proposed approach is overall the most performant. Ablations also illustrate the importance of the 3D inductive bias in the single frame prediction setting, the importance of multi-frame prediction, the importance of the semantic segmentation modeling, and the importance of stochasticity.
strengths_and_weaknesses: # Strengths
- The presents the first evaluation of a visual world model on the task of offline imitation learning for urban driving in simulation
- The paper demonstrates compelling performance results of the main method
- The paper presents an interesting approach for “lifting” the representation into 3D using the camera intrinsics and extrinsics was incorporated into the model. 

# Weaknesses
Overall, there are many proposed components and claims of the method that lack quantitative evidence.

- In the main claims of the introduction, L35 claims that the method “scales … by leveraging 3D geometry as an inductive bias”. However, the 3D inductive bias was not ablated from the main method (only from the single-frame prediction). Therefore, the claim has no evidence. Quantitative proof of this claim is needed in order me to agree with the inclusion of this claim, and thus I must weaken my evaluation of the paper's soundness and contributions.
- In the main claims of the introduction, L59 claims that “Our model predicts a distribution of diverse and plausible futures that can be decoded”. However, there is no quantitative evaluation on the diversity or the plausibility of the forecasted futures. This claim also appears in the conclusion. Quantitative proof of this claim is needed in order me to agree with the inclusion of this claim, and thus I must weaken my evaluation of the paper's soundness and contributions.
- A missing point of comparison is an existing model-based imitation learning approach that has been applied to CARLA [A]. This approach models only the future positional trajectory of the ego-vehicle. Although it would require a bit of experimentation, it would be informative to compare against it, or something like it, that purely models the future motion of the ego-vehicle, as opposed to the entire image. I suspect the current approach might perform quite well just by modeling the future positions, as opposed to the future semantic segmentation and RGB observations. I would at least like to see the proposed approach evaluated against itself when just trained to forecast the future positions of the vehicle.
- L100 The high resolution input is quite interesting, but it was not ablated (relative to the standard Dreamer model, which uses 64x64x3 images). The evaluation would be improved if a comparison to simply resizing the larger dimension of the input image to be 64 and running that as a comparison. This would enable us to understand the effect of using a higher resolution image. It is possible there is no effect, or that there is a significant effect. The results are important and relevant in either case.
- “Observation dropout” in S3.5 was a new method that was proposed but not ablated. The proposed approach should ablate this proposed component in order to highlight its efficacy.
- The ablation of the semantic segmentation targets for the main method is missing. The conclusion mentions “another exciting avenue is self-supervision in order to relax the dependency on the bird’s-eye view segmentation labels”. However, this should have been already investigated by simply removing this supervision from the main model and investigating its effect.

## Post-rebuttal update
The weaknesses I described were more-or-less addressed in the revised version and the authors' responses. 

[A] Deep Imitative Models for Flexible Inference, Planning, and Control, ICLR 2020. https://openreview.net/forum?id=Skl4mRNYDr

questions: Aside from addressing the main points in the weaknesses (the two regarding the 'main claims' are the most important)

- The “deterministic temporal” baseline needs more explanation. Is it using the same architecture? What is the precise training objective?
limitations: No, there is no clear "limitations" section. The paper requires some discussion of limitations relative to (a) scalability w.r.t. availability of training data (b) feasibility w.r.t. existing model-free imitation learning methods, among others.
review_title: null

Meta-review: metareview: This work introduced a model-based framework for offline imitation learning of autonomous driving policies in simulated urban environments. The proposed model MILE jointly learns a world model and predicts expert actions using a variational generative model. This paper was reviewed by three expert reviewers. At the initial reviews, the reviewers raised several questions about technical details and gave valuable suggestions on the overall presentation of the paper. In particular, Reviewer W3CA pointed out that some of the claims made in the paper were not sufficiently supported by quantitative evidence, and Reviewer s7YQ suggested an additional discussion of trajectory prediction methods. The authors did a good job drafting a detailed rebuttal and updating the paper revision, which addressed most of the reviewers' concerns. In the end, all three reviewers leaned towards accepting this paper. The AC read the paper, the reviews, and the discussions in detail and believed that this paper had presented a strong showcase of using model-based approaches for challenging vision-based autonomous driving problems. Taking all these into account, the AC recommends accepting this paper at NeurIPS 2022.
confidence: Certain
Decision: Accept