Paper: Adversarial AutoMixup (ICLR-2024)

Reviewer comment (Reproducibility):
The method is harder to implement and it requires more resources than other augmentation techniques, but given the timeline of augmentation, it is expected.

Author response:
**A1:Thank you for your comments on our work.** 

In line with the point you raise, we have added a plot of efficiency against accuracy in Figure 9 in the Appendix in page 16.  From Figure 9, we observe that our approach takes more time than AutoMix for training but it achieves the best performance w.r.t the existing approaches.  For testing, nonetheless, there is no difference between AutoMix and AdAutoMix in terms of time cost.

Reviewer comment (Experiments):
(**W1**) More empirical analysis of the proposed methods can be added. Despite the authors visualizing the mixed samples and CAM maps of various mixup methods, it can only reflect the overall performances and characteristics of different methods. I suggest the authors provide a fine-grained analysis of each proposed module to demonstrate its effectiveness, e.g., plotting the classification accuracy of using adversarial training or not.

Author response:
**A1:** Thank your valuable suggestions. We have plotted the classification accuracy before and after using adversarial training. The classification accuracy is illustrated in Figure 10 in Appendix A.9 The experimental results imply that the proposed adversarial framework is capable of generating harder samples to improve the robustness of the classifier.
|model|wo adversarial|w adversarial|
|---|:--:|:--:|
|ResNet-18|80.45|81.55|
|ResNeXt-50|84.12|84.40|

Reviewer comment (Experiments):
(**W2**) Small-scale experiments. The authors only provide comparison results on CIFAR-10/100, Tiny-ImageNet, and fine-grained classification datasets. More experiments on ImageNet-1K or other large-scale datasets are required. Meanwhile, the evaluation tasks or metrics can be more diverse, such as more robustness evaluations with adversarial attacks and transfer experiments to downstream tasks.

Author response:
**A2:** According to your suggestions, we have carried out comparable experiments on the ImageNet-1K dataset to evaluate the performance of *AdAutoMix*, and the results are listed in Table 1 in page 7. Also, we have shown the robustness evaluations of our approach with adversarial attacks and transfer experiments. We have added a paragraph **“Occlusion Robustness”** in subsection 4.4 and the responding results are shown in Figure 5 in page 9 and Table 7 in Appendix A.6 in page 15. Also, we have done experiments to test our approach against **adversarial attacks** and the corresponding results are detailed in Table 4 in page 9. In addition, we have added **transfer learning** experiments in subsection 4.5, and the accuracy of various approaches are depicted in Table 3 in page 9.

Reviewer comment (Presentation):
(**W3**) Some minor drawbacks in writing formats, and I suggest the authors take more time to polish the writing. As for Section 3, the arrangement of Sec. 3.1 and Sec. 3.2 can be reversed. Or the authors can provide a Preliminary section to introduce the background knowledge (e.g., mixup classification problem). As for equations, the text subscripts (e.g., $argmin_{\theta}$, $L_{amce}$) should be in bold format, i.e., using `\mathrm{}` as $\mathrm{argmin}_{\theta}$. As for tables and figures, there are some untidy arrangements, like Table 3, 4, and 5, and Figure 5 and 6. The author might zoom in on Figure 1 to show the detailed improvement of AdAutoMix.

Author response:
**A3:** Thanks for your suggestions to help us improve the quality of our work.
- We have polished the English writing and carefully checked the grammar errors through the whole manuscript.
- Because of limited space, we did not add a new Preliminary section but have reversed Sec. 3.1 and Sec. 3.2  in the original version (at page 4).
- We have marked text subscripts in all equations in bold format. Also, we have rearranged Table 3,  4, and 5, and Figure 5 and 6 in the revised version in pages 8 and 9. In addition, Figure 1 is zoomed in to clarify the performance improvement of our *AdAutoMix* in page 2.

Reviewer comment (Reproducibility):
(**Q1**) Do the authors provide the hyper-parameter settings of AdAutoMix (e.g., the mixing ratio $\lambda$, the mixed sample number $N$, and $\beta$ in Eq. (12)? The authors might provide a sensitivity analysis of the hyper-parameters in the Appendix.

Author response:
**A4:** In the previous version, we have shown the hyper-parameter settings of AdAutoMix with the mixed sample number N, $\alpha$ and $\beta$. In the revised version, we show the performance of our approach at different mixing ratio $\lambda$ and provide a sensitivity analysis for all hyper-parameters. The experimental results are shown in Figure 6 in page 9, and Tables 9 and 10 in Appendix in page 16. 
|method/$\lambda$ ratio|0.2|1.0|2.0|5.0|10.0|
|---|:--:|:--:|:--:|:--:|:--:|
|ResNet-18|82.27|82.32|81.73|80.02|81.05|
|ResNeXt-50|84.22|84.40|83.99|84.31|83.63|

|method/N samples|N=1|N=2|N=3|N=4|N=5|
|---|:--:|:--:|:--:|:--:|:--:|
|Top1-Acc|78.04|82.16|82.32|81.78|80.79|
|Top5-Acc|94.60|95.88|95.92|95.68|95.80|