Paper: Robustness Guarantees for Adversarial Training on Non-Separable Data (ICLR-2024)

Reviewer comment (Novelty):
It seems that the paper [1] is very relevant to this work, detailed discussion about the similarities, differences, and the superiority of this paper should be added.

Author response:
We thank the reviewer for pointing our the related references and will incorporate them into the final manuscript.

Reviewer comment (Theory):
My main issue with the submission is that the assumptions made by the authors are quite restrictive for most applications. For instance, the assumption that the number of training data is much lower than the dimension of the input space, but still larger than C log(1/\delta) is restrictive for many applications where neural networks are employed and where the amount of data is generally larger than the input dimension.

Author response:
High dimensional settings, wherein $d \gg n$, are common in applications of machine learning; e.g., bioinformatics, computer vision applications involving video/multimedia data, natural language processing, etc. The requirement that $n$ is larger than a constant, i.e., $n > C \log(1/\delta)$ is also very mild. Furthermore, we note that while we limit ourselves to these settings in order to build a theory, in application we may see that our results extend beyond this setting. Also, this paves a path for future work to explore which assumptions are necessary and which are not.

Reviewer comment (Experiments):
Experimental results are only limited to a synthetic example. The authors also report some experiments on MNIST on the Appendix. Why not reporting them in the main text?

Author response:
We report MNIST in the Appendix due to space limitations. We can switch them if the reviewer prefers that.

Reviewer comment (Writing):
Why do you need to overload the notation for the l_2 norm in the first line of Page 3. Can't you simply use always || \cdot || ?

Author response:
Yes we can simply use $\| \cdot \|$.

Reviewer comment (Novelty):
The technical difficulty of this paper is mainly the processing of the adversarial part. However, since the attack intensity $\alpha$ needs to take a very small value, such an expansion does not have great technical difficulty and contribution.

Author response:
**Regarding technical difficulty and contribution.**

Adding the adversarial perturbations requires certain ingenuity for the proofs to go through; they are not straightforward as you can check details from the Appendix. We here point out the contributions
1. We fix mistakes in the proofs 
In prior work of Frei et al. (2022) and Xu and Gu (2023). Please see **discussion in Appendix Section B.1 first paragraph** for details on technical improvements over Frei et al. (2022), and Prop. B.12-Lemma B.15 for a more rigorous concentration argument over Xu and Gu (2023). 
2. The proof of our convergence guarantee also differs from the original -- our Lemma B.11 has a different proof strategy than Lemma 14 in Frei et al. (2022). Following the previous work to give a result in an adversarial setting leads to a vacuous bound. Please see the beginning of Section 4.2. 
3. We establish a robust test error lower bound and explore its relationship with the upper bound, suggesting an almost tight bound under certain conditions. These are interesting findings not present in previous literature.





**Regarding the perturbation size.**

We disagree with the reviewer's comment that ``intensity $\alpha$ needs to take a very small value". For smooth activation function, the perturbation size can be as large as $\alpha=O(\|\mu\|)$ when $d=\Theta(n\|\mu\|^2)$.

Reviewer comment (Theory):
While the authors claim that the results in this paper are applicable to any width, for overparameterized networks which $m\gg n, d$, the results presented may lack significance.

Author response:
**Regarding any width networks.**

 We respectfully disagree with the reviewer's comment. Our guarantees hold for networks of any width. This does not mean that our guarantees become insignficant in overparamterized settings. Furthermore, the convergence and generalization guarantees for overparametrized networks in the lazy regime or the Neural Tangent Kernel (NTK) regime (which is the dominant framework in theoretical deep learning), **do not extend to the adversarial setting.** The work of Wang et al. (2022) shows that networks trained in this regime are not robust. Our work shows that adversarial training actually returns networks that are outside the lazy regime (see Proposition 3.5) and are guaranteed to generalize adversarially robustly.

Reviewer comment (Theory):
The discussion of overfitting with adversarial training is unconvincing. Such a discussion can only show that the results of this paper are not inconsistent with the phenomenon in [3], but it still cannot provide a reasonable explanation for the phenomenon in [3].

Author response:
**Regarding overfitting discussion.**

Nowhere in our discussion do we claim to explain the robust overfitting phenomenon observed in Rice et al. (2020). We simply remark that our results are not in conflict with their findings -- that remark is in no way trying to explain any phenomenon.

Reviewer comment (Theory):
The additional assumption in Section 3, which states that $\phi'(z)z$ and $\phi(z)$ are close, is hard to grasp.

Author response:
In Section 3.1, the additional assumption approximates the homogeneous property, which is used for proving the convergence guarantee. 
This almost homogeneous property is not very important, because it is only used to prove the bound on training error (not test error). From this assumption we showed that if the smooth loss function is close to ReLU or leaky ReLU, we can have a small training error. 

Another example of almost homogeneous function: 
\begin{equation}
\phi(x)= \begin{cases} 
x &x \leq 1,\newline
\sqrt{2x-1} &x>1. \end{cases}
\end{equation}

Reviewer comment (Theory):
Is it another notion on the Liptshitzness on the activation function?

Author response:
In Section 3.1, the additional assumption approximates the homogeneous property, which is used for proving the convergence guarantee. 
This almost homogeneous property is not very important, because it is only used to prove the bound on training error (not test error). From this assumption we showed that if the smooth loss function is close to ReLU or leaky ReLU, we can have a small training error. 

Another example of almost homogeneous function: 
\begin{equation}
\phi(x)= \begin{cases} 
x &x \leq 1,\newline
\sqrt{2x-1} &x>1. \end{cases}
\end{equation}

Reviewer comment (Theory):
How strong is it to assume that $c_1,c_2=0$ in ReLU network, for example?

Author response:
We are not assuming $c_1=c_2=0$ for ReLU. 
The approximate homogeneous property holds trivially for ReLU.