Paper: AutoJoin: Efficient Adversarial Training against Gradient-Free Perturbations for Ro- bust Maneuvering via Denoising Autoencoder and Joint Learning (ICLR-2024)

Reviewer comment (Experiments):
There are several new state-of-the-art adversarial defense mechanisms in the field currently, and they are missed to be included in the paper.

Author response:
The main focus of this work is on gradient-free perturbations, as opposed to gradient-based perturbations. Appendix G discusses some preliminary results of AutoJoin in relation to adversarial attack techniques such as FGSM and PGD. Due to the gradient-free focus, we do not compare to state-of-the-art adversarial defense mechanisms as we feel those belong more in the category of gradient-based adversarial attacks as opposed to gradient-free perturbations as defined in our paper.

Reviewer comment (Novelty):
It is diffult to understand what is the main novelity of the proposed method.

Author response:
We understand that AutoJoin is a technique that lacks novelty. AutoJoin employs simpler techniques, such as its use of DAE, within a robust training pipeline to improve robustness on steering angle prediction as it relates to autonomous driving. However, despite its simplicity, we believe that we are addressing a problem that is interesting to a broad audience; and on the technical front, we believe if a problem can be solved using a simpler approach, that approach should be preferred. In this work, we opt for a relatively simple approach while outperforming SOTA techniques by a significant margin.

Reviewer comment (Experiments):
How deoes this problem might take palce in real-world scenarios?

Author response:
We envision that the perturbations applied to the images simulate potential artifacts that may occur within a faulty camera sensor. For example, we apply a noise perturbation to the images to simulate individual artifacts that may appear in images or we apply a blur perturbation to simulate images that appear blurry due to faulty camera sensors.

Reviewer comment (Experiments):
How does the proposed method comapred with state-of-the-art adversarial training and defence mechanisms?

Author response:
The main focus of this work is on gradient-free perturbations, as opposed to gradient-based perturbations. Appendix G discusses some preliminary results of AutoJoin in relation to adversarial attack techniques such as FGSM and PGD. Due to the gradient-free focus, we do not compare to state-of-the-art adversarial defense mechanisms as we feel those belong more in the category of gradient-based adversarial attacks as opposed to gradient-free perturbations as defined in our paper.

Reviewer comment (Novelty):
What is the main novelity of the propsoed method? new perturbation training or proposing a new roboust model for steer angle prediction?

Author response:
We understand that AutoJoin is a technique that lacks novelty. AutoJoin employs simpler techniques, such as its use of DAE, within a robust training pipeline to improve robustness on steering angle prediction as it relates to autonomous driving. However, despite its simplicity, we believe that we are addressing a problem that is interesting to a broad audience; and on the technical front, we believe if a problem can be solved using a simpler approach, that approach should be preferred. In this work, we opt for a relatively simple approach while outperforming SOTA techniques by a significant margin.

Reviewer comment (Evaluation):
Main weakness is from how the robustness is evaluated. In verifying the robustness, it would be better to explore more types of perturbations. Such perturbations could be from both common image manipulations and adversarial perturbations. Although several image corruptions are applied, they mainly focus on color perspective.

Author response:
Thank you for the review.

The gradient-free perturbations we use during training focus on color perspective; however, we do test on other types of image perturbations such as blur or compression. We had not considered adding those types of image perturbations to the training pipeline to stay inline with Shen et al. so we do not currently have empirical results for what would happen if we used other types of image perturbations during training.

Reviewer comment (Experiments):
The robustness evaluation should consider other types of image corruptions, such as image rotation, perspective warping, and JPEG compression. Such robustness evaluation can be referred from [1].

Author response:
Thank you for the review.

The gradient-free perturbations we use during training focus on color perspective; however, we do test on other types of image perturbations such as blur or compression. We had not considered adding those types of image perturbations to the training pipeline to stay inline with Shen et al. so we do not currently have empirical results for what would happen if we used other types of image perturbations during training.

Reviewer comment (Evaluation):
Such evaluation can also be executed on adversarial perturbations, such as from FGSM, PGD, Autoattack, etc.

Author response:
We do investigate AutoJoin in relation to FGSM and PGD adversarial attacks; however, we place that section within Appendix G due to our main focus for the paper being on gradient-free perturbations.

Reviewer comment (Writing):
There are some typos that need attention. For instance, the first sentence in the introduction, "have" should be replaced with "has".

Author response:
Once again, thank you very much for the suggestions. We have resolved the issues regarding the typos and incomplete citation.

Reviewer comment (Writing):
In the conclusion section, the citation is incomplete. Line 5: a full stop is missing at the end of a sentence.

Author response:
Once again, thank you very much for the suggestions. We have resolved the issues regarding the typos and incomplete citation.

Reviewer comment (Experiments):
Adversarial training is largely dependent on how the images are perturbed, which determines how the adversarially trained model will generalize its robustness. The FID is used only minimally to determine the maximum intensity value of a perturbation. What will happen or be expected if perturbations are kept consistent with [Shen et al, 2021] and [Bengio et al 2009]?

Author response:
For AutoJoin’s performance with using perturbation intensity values that match Shen et al., we investigate that in Appendix D with the model labeled as “w/o RI” (meaning without Random Intensities) within Tables 14 to 17. Our experiments find that using the same perturbation values as Shen et al., there is a loss in performance overall across the datasets compared to random intensities sampled within our defined range. As such, we deploy AutoJoin with random intensities compared to the same intensities as Shen et al. In regard to Bengio et al., we have not considered matching those values so we thank you for the suggestion.