Paper: Efficient local linearity regularization to overcome catastrophic overfitting (ICLR-2024)

Reviewer comment (Novelty):
The main idea presented is not conceptually novel. Local linearity has been presented as a solution to catastrophic overfitting several times in the literature (which the paper also discusses). The only novelty is regularizing for linearity using an objective that does not involve gradients. I am unsure how novel or interesting this part itself is – for instance, one can just as well use any other local linearity regularizer (for example, LLR or GradAlign), and replace the gradient terms therein using finite differences. This would eliminate the need for the “double backpropagation” as well and meet their desiderata. The paper needs to discuss why these simple solutions are undesirable if at all they are.

Author response:
**(A1):** We respectfully disagree with this statement. Admittedly, the spirit of enforcing local linearity has been well applied as a solution to CO. However, there are several key contributions introduced in our paper:
- Our method is the first to avoid the 3x time cost coming from Double Backpropagation while being based on the same local linearity principle. 
- We theoretically link our regularization term and LLR [1] as approximants of second-order directional derivatives (Prop. 1 and 3). Compared to LLR, ELLE poses the advantage of not requiring double backpropagation.
- We develop the first method to adaptively regularize local linearity and showcase its benefits.
- We showcase the appearance of Delayed CO for several other methods and demonstrate that adding our regularization can resolve this issue.

While our approach maintains simplicity, the results in the paper demonstrate the robustness of the method to tackle CO and outperform SOTA for large $\epsilon$. If the reviewer has noticed any of the previous contributions in related papers, we are happy to reevaluate this list. 

- **(Q2): Why not use Finite Differences to approximate LLR or GradAlign?**

**(A2):** Let us explain why Finite Differences don’t fit well in this setup. We theoretically link LLR and ELLE by showing both regularization terms approximate second-order directional derivatives (see Prop. 3 and 1). Therefore, using a Finite Differences approximation of the gradient in the LLR case is unnecessary.

We elaborate on the approximation of GradAlign with Finite Differences in Appendix B.14. Estimating the gradients via Finite Differences is computationally expensive with $d+1$ function evaluations needed for estimating every gradient ($d = 248,832$ for ImageNet). Additionally, when more efficient approximations are involved [2], the regularization term becomes non-smooth due to the appearance of the sign function and it is not directly differentiable, as we would desire for a regularization term. Moreover, it requires one more function evaluation than ELLE. If the reviewer has in mind an alternative way to compute the finite differences, we would be happy to discuss this further.

Reviewer comment (Novelty):
The paper misses a couple of key references (see [1,2]), which aim to achieve the same objective as this given paper. It would be great if the authors could comment on how their method compares with the approaches presented in these works.  
[1] Singla et al., Low Curvature Activations Reduce Overfitting in Adversarial Training, 2021  
[2] Srinivas et al., Efficient Training of Low-Curvature Neural Networks, 2022

Author response:
**(A3):** We are thankful for the references that link curvature and robust generalization. We accordingly cite [3] and [4] in the introduction. However, we emphasize that there exist distinct differences between these papers and our method.

[3] analyze the role of the curvature of the activation functions in robust generalization when performing AT. Our method is not based on architectural design, but the optimization method used to train a given architecture. The combination of low curvature architecture design and regularization terms to enforce local linearity is an interesting future direction.

[4] propose a method for controlling the global curvature of twice differentiable neural networks to improve robust generalization. Our method controls the local curvature of the network and is not restricted to twice-differentiable classifiers.

Reviewer comment (Experiments):
The experimental section, while demonstrating CO and the effect of its method, fails to present comparisons with the canonical method of PGD [Madry et al., 2018]. While speed comparisons are made with PGD in Figure 1, there do not appear to be accuracy comparisons (AA, Clean accuracy) or comparisons with regard to the ability of PGD to locally linearize the model or usage of the proposed method (ELLE) with PGD. This notable omission of PGD significantly weakens the paper. Instead, the paper focuses primarily on improving FGSM-based defenses (although it also discusses improving GAT and N-FGSM in the experiments), which is a significantly weaker defense than PGD.

Author response:
**(A4):** AT is an effective but expensive defense. This is precisely the reason that faster single-step alternatives flourished [6,7], trading off performance for a 10x speed up. The main target of our work is handling critical failures and weaknesses of single-step adversarial training methods. Ideally, we would like single-step methods as reliable as AT PGD-10 but without the computational burden of PGD. 

To address the reviewer’s concern, we have included the AT PGD-10 evaluation in [Fig 1](https://imgur.com/a/xjV7Bdc), [Fig 2](https://imgur.com/a/sMToIKd), [Fig 3](https://imgur.com/a/aFpRbkz) and [Fig 5](https://imgur.com/a/e7Z8mQ1). The ability of AT PGD-10 to generate locally linear models is well known and displayed in Fig. 2, where both the gradient misalignment and our local linear approximation error are controlled during training. N-FGSM+ELLE-A is the method that attains the closest performance to AT PGD-10.

Reviewer comment (Theory):
The authors claim in Section 3.3 that the GAT and NuAT methods are not relevant since they attempt to make models locally constant as opposed to locally linear. However, local constancy is a special case of local linearity, and thus it is not inconceivable to compare the two methods on equal grounds, as is done for LLR and GradAlign.

Author response:
**(A5):** We thank the reviewer for the attentive reading. We have revised the text in section 3.3.

Let us clarify that in the original submission, we already had comparisons against GAT. As studied in [5], GAT and NuAT are not able to overcome CO for large values of $\epsilon$. In [Fig 5](https://imgur.com/a/e7Z8mQ1), we show that GAT suffers from CO for all cases except PRN at $\epsilon=8/255$. When combined with ELLE-A, GAT training is stabilized and CO does not appear.

We will be happy to answer any other comments or questions the reviewer might have.

Reviewer comment (Experiments):
There are several SOTA AT methods currently being used which have demonstrated significant improvement in this area. It would be beneficial to have other SOTA adversarial attacks evaluation in the paper as well to show how this approach helps against other SOTA attacks as well.

Author response:
**(A1):** Our models are evaluated with AutoAttack [1] (See Tab. 1 and 2 and Fig. 5), which contains the SOTA attacks FAB [2] and Square [3]. Moreover, AutoAttack is the standard evaluation metric in the [RobustBench](https://robustbench.github.io/) benchmark. We are not aware of more effective adversarial attacks than the AutoAttack ensemble. We will be happy to include any other SOTA adversarial attacks in the evaluation if the reviewer has any concrete methods in mind.

Reviewer comment (Experiments):
The reviewer questions how the proposed method compares with SOTA adversarial attack algorithms, and how this method can be combined with other AT algorithms beyond FGSM, and whether it would be effective in that sense or not.

Author response:
**(A2):** Yes, please let us explain how we have already integrated ELLE to other methods. In this paper, we present a thorough evaluation against SOTA single-step adversarial training methods. ELLE(-A) can be integrated as a plug-in regularization term into any other adversarial training method. This is demonstrated in the paper by combining ELLE-A with FGSM, GAT [4], and N-FGSM [5] (See Fig. 3 to 5). As we demonstrate, our method is effective to overcome CO when combined with all these methods.  

Additionally, during the rebuttal we have included an analysis of the integration of ELLE-A with single-step variants of AdvMixUp [6] and TRADES [7]. These two methods are only able to overcome CO when in combination with ELLE-A, see [Tab. 3](https://imgur.com/a/lhXoSZN) and [Fig. 20](https://imgur.com/a/SkItddC).  

We will be happy to answer further comments the reviewer might have.

Reviewer comment (Theory):
The definition of Equation 1 is excessively simplistic, lacking any explanation or validation. It's important to note that the majority of deep models and training losses do not conform to linear mappings, which leads me to question this particular configuration. Also, the authors claim that ''CO appears when the loss suddenly becomes non-linear, ...'' (on page 4) I find this description somewhat confusing. Since the loss function usually remains constant throughout the training process, how does this transition from linear to non-linear occur in the loss?

Author response:
**(A1):** We thank the reviewer for highlighting this issue. We agree with the reviewer in that deep models and training losses are usually non-linear. However, even though they are globally non-linear, it has been observed that the loss of robust models behaves linearly in small regions around data points, see [1] and related citations in the third paragraph of the introduction. As an example, the output of Deep NNs with the ReLU activation function is highly non-linear, but piecewise linear, and thus they are locally linear in small regions.

We have clarified this in the preamble of equation (1) and accordingly updated equation (1) to better capture this intuition.

**(A2):**  “The loss function usually remains constant”: We believe the reviewer refers to the minimization objective, e.g., the cross entropy loss. This is indeed kept the same throughout the training process in AT [2]. Nevertheless, the behavior of the loss landscape with respect to the input data evolves in time depending on the parameters of the model [1]. Let $h(\theta_t, x, y) = L(f_{\theta_t}(x), y)$, we are interested on the local linearity of $h$ w.r.t. $x$. The loss transitions from locally linear to non locally linear in the sense that for some $t = t_1$, we have that $h$ behaves linearly with respect to $x$, but it does not for some other $t = t_2 > t_1$. This behavior is clearly displayed in Fig. 2 (b-c), where the local linear approximation error suddenly increases at epoch 15 when training with FGSM.

Reviewer comment (Experiments):
The proposed local linear approximation error is similar to the mixup, which has been studied in adversrial training [a]. But, the comparison and analysis are missing.

Author response:
**(A3):** We are thankful to the reviewer for the suggestion. We conduct the related experiment in Appendix B.13. The convex combination used in the design of the input points and labels given by AdvMixUp resembles the convex combination used for obtaining the point $x_c$ in our method. However, there are fundamental differences between AdvMixup and ELLE. Let $\delta_{\text{adv}}$ be the adversarial perturbation given by some adversarial attack:

- AdvMixUp assigns the one-hot vector corresponding to the true class to  $x$ and uniform class probabilities to the point $x + 2\delta_{\text{adv}}$. Then, every point in the convex combination of $x$ and $x + 2\delta_{\text{adv}}$ is assigned a convex combination of the aforementioned labels.
- ELLE enforces the training loss to be locally linear in the whole region $x + \delta: ||\delta|| \leq \epsilon$ via a loss involving a convex combination of terms.
- In the single-step scenario, i.e. $\delta_{\text{adv}} = \epsilon \cdot \text{sign}(\nabla_{x}L(f_{\theta}(x), y))$, AdvMixUp suffers from CO while our method does not.

We include an experimental evaluation of AdvMixUp and its combination with ELLE-A in Appendix B.13. Showing that combined with ELLE-A, CO is avoided, see [Tab. 3](https://imgur.com/a/lhXoSZN) and [Fig. 20](https://imgur.com/a/SkItddC).

Reviewer comment (Evaluation):
According to the results shown in Figure 4, I found that ELLE-A is not better than ELLE, but I cannot find any clear explanation.

Author response:
**(A4):** We thank the reviewer for highlighting this missing explanation in the text. This is expected because the ELLE-A regularization is weaker. So overall, the model can still overfit in long schedules. ELLE is a stronger regularization and test performance decreases much less with time. We have included a related sentence in section 4.4.

Reviewer comment (Experiments):
In the review process, I attempted to use mathematical tools to show that the regularization introduced in this paper can be used for the approximate estimation of LLR. That is, 

   $L(f_\theta(x+\delta),y)-L(f_\theta(x),y)-\delta^T\nabla_xL(f_\theta(x),y)\approx L(f_\theta(x_c),y)-(1-\alpha)L(f_\theta(x_a),y)-\alpha L(f_\theta(x_b),y)$.

   So, I think ELLE is an effective approximation for LLR. And it's better to compare LLR and ELLE in the all experiments. BTW, the authors only report some results in Figure 1, when considering methods enforcing local linearity. But, I cannot find any comparison in the experiments. I think this lack of comparison to be unfair.

Author response:
**(A7):** We thank the reviewer for the suggestion. Both LLR and ELLE approximate the second order derivatives up to constant factors. This is proven in Prop. 3 (End of the appendix) and Prop. 1 respectively. Considering this fact, the lower computational cost of ELLE favors its usage. The comparison against LLR is usually not included in the literature [1,3]. Moreover, our results in Fig. 1 show that LLR attains a significantly lower adversarial accuracy than ELLE.

We are currently running additional experiments on CIFAR100 and SVHN. We will post those results and revise the respective tables during the rebuttal period. We include an ablation on the $\lambda$ parameter for LLR in Appendix B.12, [Fig. 19](https://imgur.com/a/SXLapTO), and the corresponding CIFAR10 results in [Fig. 3 (b)](https://imgur.com/a/Gz20Tfn). LLR does not match the performance of ELLE(-A), especially for large $\epsilon$.

We will be happy to answer any other comments or questions the reviewer might have.