Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper aims to generate adversarial examples for self-driving car. It has two fold contributions. First, the paper analyzes the sensitivity of a learning algorithm w.r.t. different image transformation in a realistic scenario. Next, they also propose a technique to learn from the generated images. 

The paper addresses a very important problem. The reported performance also quite good. However, there are some major concern about this work:

1.	There are many work along this line in other communities. For example, 
a.	Tian, Yuchi, et al. "Deeptest: Automated testing of deep-neural-network-driven autonomous cars." Proceedings of the 40th international conference on software engineering. 2018.
b.	Zhang, Mengshi, et al. "DeepRoad: GAN-based metamorphic testing and input validation framework for autonomous driving systems." 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2018.
The above two papers work on the same premise that the self-driving car learning model is susceptible to many real-world conditions and propose methods to analyze to the sensitivity of the method. Especially, the second paper uses GAN to simulate image close to the original distributions. The paper should compare with these baselines. 

2.	 The re-training method using evolutionary optimization is interesting. However, a similar technique is proposed by Gao, Xiang, et al. "Fuzz testing based data augmentation to improve robustness of deep neural networks." Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering. 2020. The author should compare the retraining method with this technique.

3.	Figure 4 generated images that do not look very real to me.

4.	Only tested for one learning model.

review_title: Adversarial example generations of self-driving car

final_score: rating: 4: Ok but not good enough - rejection
review_content: Summary:
This paper presents an algorithm to improve the model generalization of the task of "learning to steer". First, the sensitivity of a baseline learning algorithm to degraded images in varying qualities caused by different factors is carried out. Some empirical insights are gained. Then, a new training algorithm is proposed to solve a min-max optimization problem, where the most difficult datasets are chosen and used for training at each iteration. Experiments are conducted to validate the effectiveness of the proposed method. 

Pros:
1) The idea of evaluating the sensitivity of a model to different degradation factors in the same metric space is interesting, providing empirical insights for preparing datasets with different degradation levels. 
2) Inspired by adversarial learning, choosing the most difficult datasets for training at each iteration may be informative and potentially improve the model generalization.
3) Empirical study on a base dataset for the "learning to steer" task is carried out. 

Cons:
1) The choice of FID as the unified metric for evaluating the quality of different degraded images caused by different factors is not convincing. Although the authors list some reasons on page 4, it is still unclear how FID could be a fair metric to different factors. Referring to Figure 3, the mean accuracy difference is no so sensitive to blur and noise according to the FID metric. Is it possible that FID is more sensitive to such degradation factors than others? A careful inspection and more discussions are encouraged. Besides, it is recommended to carry out a user study to register the FID or other metrics to the human visual experience. 

2) It seems that the sensitivity analysis is only used as an empirical guideline to discretize the parameters of degradation factors into their corresponding levels. However, the effectiveness of such a guideline has not been validated. The authors can compare it with other discretization methods such as uniformly sampling, unevenly sampling towards heavy degradation, or light degradation. Moreover, if the abovementioned user study is carried out, how is the performance using the discretization method based on the human visual experience levels? Besides, will it be helpful to determine the dataset volumes of different factors, i.e., adjusting the ratio (importance) between different factors? And will the ratio have a significant impact on the performance?

3) The proposed method seems to be not limited to the "learning to steer" task. Considering that some well-established benchmark datasets such as ImageNet-C and different methods have been proposed for studying the impact of degradation factors, it is recommended to carry out experiments on these datasets and included more representative state-of-the-art methods into the comparison.

4) It is unclear how the training cost is since each dataset should be evaluated separately at each iteration in the proposed algorithm. Detailed analysis and comparison are encouraged.

5) On page 5, it is said that the original dataset is also used for training together with the perturbed datasets. However, in the algorithm, only U_p is used, which is defined as the combination of the selected perturbed datasets. Is it the same as the one in Eq. (1)? Besides, what is the meaning of U in the algorithm?

6) In the Experiments Part, both the baseline method and Scenario 1 are called the baseline, which may be confusing in the following description and discussions.

7) Why the model generates strong responses at the distant areas rather than the near road, e.g., markers and boundaries, in the first three columns of Figure 6? Does the vanishing point serve as a cheap feature for "steering"? If so, the structural information of the road should be more useful. Thereby, some degradation factors such as channel perturbations may have a smaller impact on the performance than others such as blur and distortion. However, from Table 2 and Table 3 in the appendix, it seems that distortion and channel perturbations have a larger impact on the generalization performance compared with blur and noise, e.g., at the higher levels. Moreover, the proposed method can effectively deal with the degradation factors that have a smaller impact on structural information such as noise and channel perturbations (i.e., a larger improvement) while being less effective for those structural-related ones (i.e., blur and distortion). Should the degradation factors be divided into different groups, i.e., structural-related ones and structural-irrelevant ones? Will the proposed method have a consistent superiority on the two groups over the common data augmentation technique? More experiments and analysis should be carried out to provide further evidence on the choice of the baseline steering model and benchmark dataset, as well as evaluate the effectiveness of the proposed method comprehensively.
review_title: An interesting subminssion that can be improved further

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposed a novel adaptive data augmentation algorithm that produces random perturbations on the training dataset to train an imitation learning-based self-driving network. It starts with a sensitivity analysis of network performance under different types and levels of perturbations. And a novel automated perturbed training dataset selection mechanism is then proposed to improve the performance. Validation has been conducted over simulated data with both seen and unseen perturbation types. 


Pros:
* Very practical task. Such problems do exist in real-world self-driving, and having learnable perturbations could potentially reduce the generalization gap.
* Great efforts to push and simulate perturbations that could be seen in the real-world. 

My primary concern is that there are quite a few very relevant problem settings tackling similar tasks. And the paper should add discussions and comparisons. These include but are not limited to:
- Learning to reweight different training samples. e.g. [A]
- Automated data augmentation with perturbations, e.g. [B]
- Bayes opt for hyper-params search overall, e.g. [C, D]

"Unseen" perturbations are also re-assembling of several low-level degradation factors. Thus it's not clear whether such a domain gap is significant or not. It will help to understand the question if there exists similar perturbation in training. This could be done by choosing the most similar perturbation type in training to each unseen perturbation in testing in terms of image similarity score between the same images under two different perturbations. 

The baselines used in the experiment are a bit too weak. Please consider adding the ones as mentioned above. Please consider the "oracle" with a training set that reproduces the perturbations used in testing. This is to evaluate whether this proposed method could improve performance when there is no domain gap in perturbation. For instance, could you even perform better on rainy days through a model trained on the proposed training set rather than a dataset of rainy day logs? 

The method was not evaluated on a real dataset. Many real datasets, such as OxfordRoboCar, Raincouver, Canadian Adverse Driving Conditions, contain real-world driving data with challenging weather conditions. Plus, it would be preferable to evaluate perception performance as well, such as detection/segmentation, not only an end-to-end IL driving model, which is less practical at this moment for real-world self-driving. 

The incremental dataset augmentation is greedily selected based on the worst validation performance on that single subset. As a consequence, an over-difficult subset might be preferred. Instead, the subset improves overall performance. Could you comment on that?

Minors:
* Getting the full validation performance at each step could potentially time-consuming. Have you considered improving the efficiency by early stopping criteria? 
* How many time steps do you run in practice? 
* Does the # of training iterations and controlled to be the same across all competing methods to ensure fairness? 
* How about the additional run-time compared against no dataset selection? 

[A] Ren et al. Learning to Reweight Examples for Robust Deep Learning
[B] Cubuk et al. Autoaugment: Learning augmentation policies from data
[C] Tran et al. A Bayesian Data Augmentation Approach for Learning Deep Models
[D] Snoek et al. Practical Bayesian Optimization of Machine Learning Algorithms



-------------------------------------------------------- Post Rebuttal --------------------------------------------------------------------------------------------

I carefully read the reviewer's comments and the rebuttal. I think the author did not get my major concern on competing algorithms. I acknowledge this proposed method is a novel problem setting and my goal is not to ask for summarizing the difference in problem settings. However, the technical approach, which is based on (batch-)training sample selection and reweighting, could be validated through comparing against more comprehensive baselines, as I pointed out, under similar settings.

Furthermore, I am not convinced by the claim that unseen perturbation cannot be justified if there exists a similar perturbation assemble during training. The reference-based perceptual similarity is a well-studied domain which could be directly adopted to evaluate the overall "difficulty" or "out-of-distribution"-level. And I am not convinced by the author that the experiments you have tried cannot be shown in paper due to page limit: at least you could include in supplementary, as they are important to justify if your experiment setting really validates generalization ability. 

In sum, I will keep my current rating. 




review_title: Review for Paper2677

final_score: rating: 6: Marginally above acceptance threshold
review_content: ### **Summary**:
 
This work proposes a new method to improve the generalization of ML models for the task of vehicle steering using a hybrid of data augmentation and adversarial examples. In a nutshell, the proposed method attempts to increase the accuracy of the model by dynamically adding a selection of candidate datasets during training. Each of these “candidates” is created offline applying a transform (e.g. blur, distortion, and changes in color representation) to the original (base) dataset. During training, the method chooses among the K transformed-datasets those who minimize the mean validation accuracy and based on this selection the steering model is retrained. The approach is evaluated on a driving dataset.
 
---

# Final evaluation (post-rebuttal)


I am increasing the score to "marginally above the acceptance threshold" after the discussion with the authors. I thank the authors for including other datasets and considering additional baselines. I still think that Chen's dataset should not be the main dataset to use in the paper, but I am happy to see strong datasets. I also believe that this paper could benefit from using simulation solutions (despite the domain gap) to create controlled experiments. 

--- 
# Original evaluation (pre-rebuttal)


### **Reasons for score**: 
 
My current score is *rejection*. 
 
I went through the main idea a few times and I think it is a nice proposal. Using the FID as a proxy to cherry-pick transformed datasets seems plausible and relatively simple to reproduce.	After seeing Figure 3. I started having doubts about the actual correlation between MA and FID, which may turn problematic for this approach, but I didn’t let that discourage me too much. 				

The real concerns emerged when I reached the experiment section. After reading the experiments I think that the narrative of the paper is not as consistent as it could (should?) be. 
The paper opens with a clear problem motivation: *“To ensure the wide adoption and safety of autonomous driving, vehicles need to be able to drive under various lighting, weather, and visibility conditions in different environments”*. Also, the title of the paper includes the words *“Improving Generalization”*. 

After reading this, I --and probably many readers-- automatically shifted to the mindset of generalization analysis in autonomous driving problems. This typically involves multiple datasets featuring variations such as weather conditions, etc., well-chosen training, validation, and test splits, and strong baselines. However, section 4 does not contain these elements.

All experiments are carried out using Chen’s dataset, and one of the main questions should be, why this dataset? Is this dataset the right one? Honestly, it is hard to tell, because there is not much information about the structure of such dataset. We know that it was collected around Rancho Palos Verdes and San Pedro, (California), but we don’t know much else. How is the dataset split into training, validation, and test? Does it contain strong weather variations? Was the proposed approach tested in a sequence never seen during training or is the method simply overfitting the training routes?

In short, this doesn’t seem to be the right setup t study the problem of generalization. Instead, I would like to recommend a controlled environment, such as the [CARLA simulator](https://github.com/carla-simulator/carla). There you can create datasets in different visual conditions in different maps, and you can also get the steering angle for free. Recent works, such as [Learning Situational Driving](https://openaccess.thecvf.com/content_CVPR_2020/papers/Ohn-Bar_Learning_Situational_Driving_CVPR_2020_paper.pdf) followed this methodology.

---

### **Strengths**:
 
* This paper is after an interesting and relevant problem, fundamental for the progress of the autonomous driving field.
* Using a possible correlation between MA and FID to drive a dynamic “adversarial data augmentation” process is an interesting idea.
* The paper is well written and simple to reproduce

---

### **Weaknesses**:

* As mentioned above, my main concern is that there are possible flaws in the evaluation protocol. The paper uses Chen’s dataset, but it is not clear if the dataset itself is suitable for the task (besides providing labels for the steering task). I believe that it would be beneficial for the paper to switch to other datasets. Synthetic datasets such as those generated by the CARLA simulator, MS AirSim, or LGSVL simulator should be fine. Real datasets, such as the [AUDI A2D2 dataset](https://www.a2d2.audi/content/dam/a2d2/dataset/a2d2-audi-autonomous-driving-dataset.pdf) could be useful too.

* It seems that strong baselines are not present in the experiments. For instance, I am surprised to see that approaches like [AugMIX](https://arxiv.org/pdf/1912.02781.pdf) or even the original paper by [Hendrycks](https://arxiv.org/pdf/1903.12261.pdf) are not included in the comparison.

---

### **Questions to be discussed during the rebuttal period**:

Please, I would appreciate it if you could refer to the points I described in the **Weaknesses** section.

---

### **Other considerations**:

* The text uses the term master-slave → primary - secondary?

---


review_title: Interesting and promising, but the evaluation protocol is not suitable for the task

Meta-review: This meta-review is written after considering the reviews, the authors’ responses, the discussion, and the paper itself.

The paper has 2 main contributions: 1) analysis of the sensitivity of a deep network predicting steering angle from images w.r.t. different synthetic image perturbations, 2) A training method, based on adaptively adjusted data augmentation, which improves the robustness of a model to seen and previously unseen perturbations.

The reviewers’ opinions are somewhat mixed, leaning towards negative. The reviewers point out that the task is important and the methodology makes sense, but the experiments are limited: only one dataset, only synthetic perturbations, only the steering angle prediction task (which is not necessarily very practical), not strong enough baselines, no results on established datasets like ImageNet-C. The authors addressed some of these issues in the updated version of the paper (more datasets, one more baseline), but most reviewers did not change their evaluation.

Based on all this information and reading the paper itself, I recommend rejection at this point. The paper has interesting ideas, but the experimental evaluation is not sufficient. Moreover, I find the use of the steering prediction task confusing - there does not seem to be anything driving-specific in the method, so using standard datasets (like ImageNet-C) would be more convincing. For driving datasets, using real-world (not synthetic) image perturbations would be advisable. As the paper stands, it looks neither like a proper application paper, nor as a fundamental method/analysis paper, but something inbetween, which is not to its favor.

Decision: Concerns raised (can publish with adjustment)