A Closer Look at the Adversarial Robustness of Deep Equilibrium Models
Abstract
Deep equilibrium models (DEQs) refrain from the traditional layer-stacking paradigm and turn to find the fixed point of a single layer.
DEQs have achieved promising performance on different applications with featured memory efficiency.
At the same time, the adversarial vulnerability of DEQs raises concerns.
Several works propose to certify robustness for monotone DEQs.
However, limited efforts are devoted to studying empirical robustness for general DEQs.
To this end, we observe that an adversarially trained DEQ requires more forward steps to arrive at the equilibrium state, or even violates its fixed-point structure.
Besides, the forward and backward tracks of DEQs are misaligned due to the black-box solvers.
These facts cause gradient obfuscation when applying the ready-made attacks to evaluate or adversarially train DEQs.
Given this, we develop approaches to estimate the intermediate gradients of DEQs and integrate them into the attacking pipelines.
Our approaches facilitate fully white-box evaluations and lead to effective adversarial defense for DEQs.
Extensive experiments on CIFAR-10 validate the adversarial robustness of DEQs competitive with deep networks of similar sizes.
Introduction
Conventional deep networks employ multiple stacked layers to process data in a feedforward manner [17].
During training, network parameters are optimized by backpropagating loss updates through the consecutive layers [36].
Recently, [3] propose deep equilibrium models (DEQs), whose forward pass involves finding the fixed point (i.e., equilibrium state) of a single layer.
With implicit differentiation, the backward pass of DEQs is formulated as another linear fixed-point system.
Training DEQs with black-box root solvers only consumes O (1) memory, which enables DEQs to achieve performance competitive with conventional networks in large-scale applications, including language modelling [3], image classification and segmentation [4], density modelling [24, 16], and graph modelling [23].
Considering the fixed point as a local attractor, DEQs are expected to be stable to small input perturbations.
However, empirical observations show the opposite that a vanilla DEQ is also vulnerable to adversarial attacks [16].
Along this routine, several works are proposed to investigate the certified robustness for monotone DEQs [40, 34, 27, 20, 28, 10].
Inspired from the monotone operator splitting theories, monotone DEQs are designed with the guarantee of existence and convergence of equilibrium points.
However, the layer parameterization of monotone DEQs and the limited scalability of certification methods narrow the scope of these previous studies.
On the other hand, [16] explore the adversarial robustness for general DEQs.
They incorporate the adversarial generation process into the equilibrium solver to accelerate the PGD attack [25].
Nevertheless, the PGD attack is originally designed for deep networks, requiring for end-to-end white-box differentiation.
In contrast, DEQs rely on black-box solvers and could obfuscate the gradients used in PGD: as shown in Fig.
2-(a), in
DEQs trained with different configurations, the intermediate states always exhibit higher robustness than the final state under ready-made PGD attacks.
Compared to the extensive literature on the adversarial robustness of deep networks [6, 38, 15, 26, 22, 25, 42, 35, 29], much less is known about the adversarial robustness of general DEQs, especially under a well-elaborate white-box setting.
This motivates us to disentangle the modules in DEQs and provide a fair evaluation of their robustness.
In this paper, we first summarize the challenges of training robust DEQs (see Sec. 3), including ( i ) convergence of the black-box solvers and ( ii ) misalignment between the forward and backward passes.
The off-the-shelf attacks work in a gray-box setting as they have no access to the intermediate states in the forward pass.
To thoroughly evaluate the robustness, we propose two methods for intermediate gradient estimation: the first one is iterating adjoint gradient estimations simultaneously in the forward pass, as formally described in Sec. 4.1; the second one is estimating intermediate gradients by unrolling, as seen in Sec.
Then in Sec. 5, we develop approaches to integrate the estimated gradients into the ready-made attacks towards fully white-box adversaries.
We also design defense strategies for DEQs to boost their robustness under white-box attacks.
We use PGD-AT to train large-sized and XL-sized DEQs on CIFAR-10.
To benchmark their robustness [12], the parameter sizes of the DEQs are set to be comparable with ResNet-18 [18] and WideResNet34-10 [41], respectively.
We observe that the adversarially trained DEQs with the exact gradient [3] require more forward steps to arrive at the equilibrium state, or even violate their fixed-point structures.
We also find an intriguing robustness accumulation effect that the intermediate states in the forward pass are more robust under ready-made attacks.
These phenomena exhibit gradient obfuscation [2], which verifies the necessity of intermediate gradient estimation to construct white-box attacks and defense strategies.
Robustness performance under the white-box evaluation shows that DEQs achieve competitive or stronger adversarial robustness than deep networks of similar parameter amounts.
Our investigation sheds light on the pros and cons with respect to the adversarial robustness of DEQs.
Background
This section includes the background on DEQs and adversarial robustness for deep networks.
2.1 Deep equilibrium models
We first briefly introduce the modelling of deep equilibrium models (DEQs) [3, 4].
Consider a T -layer weight-tied input-injected neural network:
[Equation]
where x ∈ R l is the input, z n ∈ R
d is the output of the n -th layer, and θ is the network weights shared across different layers.
One can cast the evolution of { z n } as a fixed-point iteration process.
When n → ∞ , z n converges to the fixed point z ∗ which satisfies the equation z ∗ = f θ ( z ∗ ; x ) .
Deep equilibrium models rely on the fixed-point equation and leverage a black-box solver to directly solve for z ∗ in the forward pass.
The backward pass of DEQs can also be formulated as a fixed-point iteration process.
With the loss function L ( z ∗ , y ) and implicit differentiation, we can compute the gradient with respect to θ or x with ∂L
[Equation]
where ( ∂ a /∂ b ) ij = ∂ a j /∂ b i and u ∗ satisfies
[Equation]
According to Eq.
(3), the backward pass can also be executed with a black-box fixed-point solver, and this iteration process is independent of that in the forward pass.
Several techniques have been proposed to improve the training stability of DEQs.
[5] propose to regularize the Jacobian matrix in Eq.
(2) during training so that the nonlinear forward system and the backward linear system enjoy appropriate contractivity.
[14] propose unrolling-based and gradients
The unrolling-based phantom gradient is defined as
[Equation]
where
[Equation]
are the k unrolling steps with 1 ≤ t ≤ k , starting from ˆ z N = z ∗ returned by the forward solver.
Eq.
(4) is calculated by the automatic differentiation framework [30] on the computational subgraph in Eq.
It is demonstrated that the unrolling-based phantom gradient imposes implicit Jacobian regularization effect to DEQ training [14].
DEQs trained by either the exact or the phantom gradients are competitive to deep neural networks in terms of natural accuracy.
In our work, we leverage adversarial defense strategies to train DEQs to improve their robustness.
2.2 Adversarial robustness for deep networks
Much research has been dedicated to adversarial attacks and defenses of deep neural networks.
On the one hand, white-box adversarial attack techniques like PGD [25] construct adversaries by iteratively perturbing inputs in the gradient ascent direction.
The robustness of deep networks is benchmarked by AutoAttack [12], which consists of four attacks including two PGD variants with adaptive stepsize and the query-based SQUARE attack [1].
On the other hand, adversarial training (AT) [25] is one of the most effective defense strategies.
By early stopping the training procedure as in [35], the primary PGD-AT framework still achieves competitive robustness performance compared with the state-of-the-art defense techniques like TRADES [42].
It is worth mentioning that many defense approaches claim robustness improvement by obfuscating gradients, which proves to be a false sense of security under adaptive attacks designed specifically [2].
In our work, we train DEQs with PGD-AT and investigate their adversarial robustness by designing customized defenses and adaptive attacks.
Challenges for robust general DEQs
This section describes the challenges encountered when we aim to train robust general DEQs.
Misalignment between forward & backward passes.
The central idea of DEQs is directly solving for the equilibrium state z ∗ and differentiating through the fixed point equation z ∗ =
f θ ( z ∗ ; x ) for efficient forward and backward passes.
Fig.
1-(a) sketches the calculation of the exact gradient [3].
Independent from the forward iterations (the blue curve), the exact gradient is acquired by solving for a linear fixed-point system that only depends on the equilibrium state z ∗ (the orange curve).
Fig.
1-(b) shows the calculation of the unrolling-based phantom gradient [14].
z ∗ as the final state in the forward pass is unrolled (the gray iteration), and the gradient is obtained from the automatic differentiation on the loss function.
However, when iterating the gradient computations, the intermediate states { z n } in the forward pass are bypassed by both methods.
The misalignment between the forward and backward tracks results in a gray-box setting for the ready-made attacks.
Convergence of the black-box solvers.
In contrast with monotone DEQs, there is no guarantee for the existence and convergence of the equilibrium states in general DEQs.
It is thus unknown whether the black-box solvers in DEQs still converge to equilibrium states under input perturbations.
Adversarial training also adds the concern on equilibrium convergence.
The well-known effect of adversarial training for deep networks is the trade-off between robustness and accuracy [37, 39, 42, 32].
A similar drop in standard accuracy (from 78 % to 55 %) is also observed for tiny-sized adversarially-trained
DEQs [16].
The robustness-accuracy trade-off brings training instability for general DEQs, which may take more iterations in the solvers for equilibrium convergence, or even violate their fixed-point structures.
Finally, the robustness comparison is still under-explored between large-sized general
DEQs and deep networks with similar parameter counts.
On intermediate gradient estimation
As the forward and backward tracks in DEQs are misaligned, the intermediate states in the forward pass are inaccessible to off-the-shelf attacks, which causes gradient obfuscation and results in false positive robustness.
Therefore, it is necessary to estimate the intermediate gradients.
With the integration of the estimated gradients, the attacks can validate the robustness of DEQs in a fully white-box setting.
In this section, we propose two methods for intermediate gradient estimation.
4.1 Simultaneous adjoint in the forward pass
Inspired by the adjoint process in neural ODE models [9], we propose the adjoint process for intermediate gradient estimation in DEQs.
The adjoint process in neural ODE models is characterized by an adjoint ODE [31].
For DEQs, we propose to iterate the updates of adjoint states subject to z n in the forward pass.
We investigate the simultaneous adjoint with Broyden’s method [7] as the forward solver.
In the forward pass, Broyden’s method updates the intermediate state z n based on the residual g θ ( z n ; x ) =
f θ ( z n ; x ) − z n and B n , the low-rank approximation of the Jacobian inverse:
[Equation]
where 0 ≤ n ≤ N − 1 , B
= − I , ∆ z n +1 =
z n +1 − z n , ∆ g n +1 =
g θ
( z n +1 ; x ) − g θ
( z n ; x ) , and α is the step size.
To maintain a simultaneous adjoint, we start from u 0 = 0 and use Broyden’s method to solve Eq.
Similar with the residual function g θ ( · ; x ) for z n , the fixed-point equation in Eq.
(3) defines the residual of the adjoint state.
However, we propose to replace the z ∗ in Eq.
(3) by z n , and integrate the approximated Jacobian inverse B n to force the alignment of the adjoint state update:
[Equation]
where v n is the residual at iteration n , u n is the updated adjoint state, and β > 0 is the step size.
We use the following surrogate gradients to construct attacks on the intermediate state z n :
[Equation]
An illustration for the simultaneous adjoint process is shown in Fig.
1-(c).
In the following, we refer to this method as simultaneous adjoint when constructing intermediate state attacks in Sec 5.1.
Remark 4.1 .
We show in Appendix B that under mild assumptions, the { u n } converges to u ∗ when 0 < β < 1 in Eq.
However in practice, we do not require the convergence of u n as we only use them in Eq.
(11) as gradient estimations to construct intermediate attacks .
Remark 4.2 .
Similar with the update of u n in our approach, [16] propose augmented DEQs as an integration of the iterative updating process of z , u , and x as a whole:
[Equation]
The augmented DEQs leverage a black-box solver (e.g., Broyden’s method) to find the equilibrium of the whole state [ z ∗ , u ∗ , x ∗ ] T .
However, several cross-terms exist in the joint Jacobian due to the coupling of the three iteration processes, which further hinders the convergence.
In contrast, the simultaneous adjoint update in Eq.
(10) does not include the update of x .
Furthermore, we reuse the
Jacobian inverse approximation matrix B n in the update of u n , which is easy to implement.
Because of the disentanglement of z n and u n in the joint Jacobian, our method also enjoys better efficiency and flexibility as one can early exit the adjoint process without affecting the updates of z n ’s.
Remark 4.3 .
Concurrent work [33] also explores the idea of sharing approximated Jacobian inverse
B n in bi-level optimization problems.
While their motivation is to accelerate DEQ training, we use the adjoint states to construct gradient estimation and facilitate white-box attacks.
We also compare our work with theirs in terms of intermediate state attacks; See Appendix D for details.
4.2 Unrolling the intermediate states
We also propose to estimate the gradient at the state z n by unrolling.
Depicted in Fig.
1-(d), z n is involved in an artificially constructed computational graph.
We can thus estimate the intermediate gradient by backpropagation with automatic differentiation.
Formally, applying Eq.
(4) to z n yields
[Equation]
where
[Equation]
+
[Equation]
I
[Equation]
, (15) and the state sequence ˆ z n , ˆ z n +1 , · · · , ˆ z n + k represents the damped unrolling iteration:
[Equation]
16) with t = 1 , 2 , · · · , k and ˆ z n = z n .
While Eq.
(4) is proposed as an approximation of the exact gradient, we unroll the states z n for intermediate gradient estimation.
Similar to the case of Eq.
(11), we use Eq.
(13) as estimation to design intermediate attacks for DEQs.
We refer to this method as unrolled intermediates in the following when incorporating Eq.
(13) into the white-box attacks.
White-box attacks and defenses for DEQs
This section describes different types of white-box attacks and defense strategies for DEQs.
5.1 White-box attacks for DEQs
The existing attacks leverage the gradients calculated at the final state outputted by the forward solver.
Based on the surrogate intermediate gradients in Eq.
(11) or Eq.
(13), we can involve the z n in the forward pass into the construction of adversaries.
A direct white-box approach is to use the estimated gradient at an early state z n as an alternative for input perturbations.
Another simple yet effective method is to average the intermediate gradients as the gradient ensemble for attacks.
For example, the average of all intermediate gradients along the simultaneous adjoint process is given by
[Equation]
The gradient ensemble can be viewed as the fusion of all perturbation directions indicated by all z n ’s.
5.2 Defenses with intermediate states
In addition to the final state z ∗ , the unused intermediate states can be leveraged as well for the defenses of DEQs.
A simple yet effective defense strategy is to early exit the forward solver during
Exact (8/7) Unroll (8/7)
Exact (18/20)
R ea dy - M a d e P
GDA cc .
R e l a ti v e E rr o r
Forward/Backward #Iterations
Intermediate State (a) (b) 8/ 18/20 30/30 50/ 8/
A cc u r ac y ( % )
Final state Intermediate state
ExactTrained
TrainedUnrolling inference.
We can evaluate the robustness of DEQs with the early state z n in the forward pass, as z n and z ∗ have the same shape.
We determine the optimal timing for early exit by selecting the top robustness performance of all z n ’s on the development set under the ready-made PGD-10 attack.
The input-injected neural network provides an interpretation for the early-state defense.
From Eq.
(1), the distortion of z n comes from both the perturbed z n − 1 and the biased transformation f θ ( · ; x +∆ x ) .
By early exiting the forward process, one obtains a less distorted intermediate state.
Another defense strategy for DEQs is leveraging the ensemble of intermediate states.
Similar with Eq.
(17), we average the intermediate states { z n } to defend against attacks.
Instead of early stopping, the intermediate state ensemble exploits the state representations at all iterations in the forward solver.
While the proposed defense techniques leverage the intermediate states, they still require only O (1) memory.
For early state defense, we determine the optimal time to early exit the solver on the development set offline for once and then fix the early exit step during testing.
For ensemble state defense, we maintain an accumulator to sum up { z n } along the forward pass without storing them.
Experiments
Following the settings in [5], we experiment with the large-sized DEQ with its parameter count similar to ResNet-18 [17].
We also experiment with an XL-sized DEQ with its parameter count similar to WideResNet-34-10 [41] to enable a fair comparison with the empirical robustness of the deep networks.
The detailed experimental settings are listed in Appendix A. We first train DEQs on
CIFAR-10 [21] with the PGD-AT framework [25], then test the adaptive attacks and defense strategies proposed in Sec.
5 on the adversarially-trained DEQs.
We refer to a DEQ as “exact-trained” when using the exact gradient, and “unrolling-trained” when using the unrolling-based phantom gradient in the PGD-AT framework to generate adversaries and optimize for model parameters.
Unless specified, all DEQs are adversarially trained in this paper.
During training, we use 10 -step PGD with the step size of 2 / 255 to generate adversaries within the range of ℓ ∞ = 8 / 255 .
For the specific type of attacks, we use PGD and AutoAttack (AA) [13] to instantiate the white-box attacks in Sec. 5.1.
6.1 The retention of the fixed-point structure
We start with the observation on the fixed-point structure.
Shown in Fig. 2-(b), the lines illustrate the relative error ∥
f θ ( z n ; x ) − z n ∥ 2 / ∥
f θ ( z n ; x ) ∥ 2 for each z n 2 .
We find that for the exact-trained
DEQ with small iteration settings (8 forward / 7 backward iterations), all the relative errors are larger than 0 .
75 , i.e., the forward solver in the DEQ fails to converge to an equilibrium.
Such phenomenon
[Table]
reflect the challenge on the convergence of the black-box solvers in DEQs mentioned in Section 3.
It is shown that a larger iteration setting is required (18 forward / 20 backward iterations) for exact-trained DEQs to retain the fixed-point structure.
In contrast, we find the small iteration setting (8/7) is enough for the unrolling-trained DEQ to retain the fixed-point structure.
PGD-10 with the exact gradient is deteriorated using the unrolling-based phantom gradient.
Leveraging the query-based SQUARE leads to even lower accuracy.
These observations indicate that the
DEQ with violated fixed-point structure suffers severe robustness degradation.
[Table]
It is necessary to retain the fixed-point structure, otherwise leading to gradient obfuscation issues.
As is derived from the implicit differentiation on the fixed-point equation z ∗ = f θ ( z ∗ ; x ) , the exact gradient in Eq.
(2) becomes inexact when the equilibrium point z ∗ is not reached.
Tableshows the empirical performance of the exacttrained DEQ under the small (8/7) iteration setting.
The severe performance degradation under alternative gradient formulations as well as the
SQUARE attack also indicates gradient obfuscation, as suggested in [2] and [8].
While large iterations for the exact-trained
DEQs keep the fixed-point structure, it inevitably slows down the training speed (detailed in Appendix E.3).
For the exact-trained DEQs with the small (8/7) iteration setting, we have also tried with varied Jacobian regularization weights to impose stricter Lipschitz constraints during training, but found the DEQ solver still diverged.
We have also analyzed the instability by tracing the variation of Lipschitz constant during the adversarial training of DEQs; See Appendix E.4 for details.
By comparison, the unrolling-trained DEQ requires fewer iterations in the forward solver to converge.
According to the green line in Fig. 2-(b), the relative errors become lower consequently and reach 0 .
04 at the final state.
The results coincide with [14] that the unrolling-based phantom gradient invokes implicit Jacobian regularization during training.
6.2 Robustness of DEQs under white-box attacks
Intriguingly, we discover the robustness accumulation effect in both the exact-trained and the unrolling-trained DEQs.
We plot the highest robustness under the ready-made PGD-10 among all the intermediate states in Fig.
2-(a), with comparison to the final state robustness.
It is shown that the intermediate states always exhibit much higher robustness.
The accumulated robustness comes from gradient obfuscation, as the ready-made attacks fail to “directly” attack the intermediate states due to misaligned gradients.
This resonates with the first challenge in Sec. 3, and similar results are observed as well in adversarially-trained neural ODEs: the large error tolerance from the ODE solvers with adaptive step sizes allows gradient masking after adversarial training [19].
The exact-trained DEQs, as we have discussed in Sec. 6.1, require larger iterations in the solver.
However, it is noticed in Fig.
2-(a) that the larger the iteration is in the exact-trained DEQs, the more robust the intermediate states are under ready-made PGD-10.
On the contrary, the (8/7) unrollingtrained DEQ still achieves the highest robustness at the final state.
To benchmark the white-box robustness, in this section, we compare the (50/50) exact-trained DEQ-Large with the (8/7) unrolling-
[Table]
trained one 3 .
We integrate the estimated intermediate gradients in Sec. 4 as different alternatives in
PGD-10 for white-box evaluation.
Among all the attack candidates based on intermediate gradients, we select the one that leads to the largest robustness deterioration on the early-state defense in Sec.
5.2 and report the results in Table 2.
Ablation studies on the performance of the attack candidates with estimated gradients at different intermediates can be found in Sec.
We also include the report of memory usage for the defense strategies in Appendix F.1, and the running time complexity analysis for the white-box attacks in Appendix F.2.
Shown in Table 2, for the unrolling-trained DEQ, unrolling the intermediate states results in the strongest attack to the final state and early state defenses.
While the robustness accuracies under final and intermediate attacks are improved and better balanced with the ensemble state defense, the ensemble attack leads to the largest performance drop in this case, arriving at the overall white-box robustness of 47 .
The estimated intermediate gradients based on simultaneous adjoint process also shows significant attack performance for the exact-trained DEQ with large solver iterations.
After maximizing the minimum robustness under all attacks across all defense techniques, the overall white-box robustness is 39 .
In addition, all attacks significantly deteriorate the robustness of the
DEQs without adversarial training, indicating that the attacks leveraged in Table 2 are reliably strong (detailed in Sec. 6.5).
Considering the superior robustness of the unrolling-trained DEQ as well as its training efficiency, we proceed to experiment with unrolling-trained DEQs for further evaluation.
6.3 Comparison between DEQs and deep networks
In this section, we further provide a thorough evaluation by benchmarking the white-box robustness performance of the unrolling-trained DEQ-Large and DEQ-XL under both PGD-10 and AutoAttack.
Table 3 shows the robustness performance of the unrolling-trained DEQ-Large and DEQ-XL.
According to the results, the gradient ensemble attacks are more effective in defeating the early-state defense than the final-state defense.
The ensemble attack is the most threatening on the ensemble defense in DEQ-Large.
The attack with the gradient at the final state leads to the most substantial performance drop on the ensemble defense in DEQ-XL.
weakest robustness under all attacks in Table 3 is reported.
For the deep networks, we report the results in [29].
Arch.
[Table]
In Table 3, we find that the PGD-10 attack brings more significant performance drops than AutoAttack does in many settings.
The results differ from the case in the robustness of deep networks [13].
The phenomenon originates from the difference between intermediate-state attacks and alternative defense strategies.
AutoAttack will overfit to the provided gradients at the intermediate or the averaged states, thus generating less threatening adversaries on the defenses based on other states.
In addition, the intermediate gradients can also be inaccurate, as they only serve as approximations.
The minimum robustness under all types of attacks represents the robustness of a defense strategy.
We thus take the most robust defenses for DEQ-Large and DEQ-XL, and compare them with the shown in Sec.
6.1 and Table
[Figure]
deep networks of similar parameter counts.
Shown in Table 4, the empirical robustness of the DEQs is competitive with or even slightly higher than that of the ResNet-18 and WRN-34-10 models with the PGD-AT framework, respectively.
6.4 Ablation study on different intermediate gradients
In this section, we study the effect of the white-box attacks with gradients estimated at different intermediate states in the forward solver of the large-sized DEQs.
We first inspect the attacks with intermediate gradients acquired from each adjoint state.
Fig.
3-(a) plots the robustness of the early-state and final-state defense in both the unrolling-trained and the exact-trained DEQs.
For the exact-trained DEQ, due to its violated fixed-point structure, u 8 results in the strongest attack for both the early-state and the final-state defenses.
For the unrolling-trained DEQ, the estimated gradients at the consecutive adjoint states { u n } form increasingly stronger attacks on the robustness of the final state.
On the robustness at the early state ( z 3 ), the state u 4 gives rise to the strongest attack, which coincides with Eq.
(9) and Eq.
(10) that u n +1 directly depends on z n .
We further explore whether the simultaneous adjoint process are aligned with the forward pass.
We use each intermediate adjoint state u n as the gradient surrogate in the PGD-10 attack for the unrolling-trained DEQ.
Fig.shows the robustness performance of all the intermediate states z n in the forward pass of the unrolling-trained
DEQ-Large.
According to Fig. 4, it always follows that u n +1 results in the largest robustness drop of z n .
As u n + directly depends on z n in Eq.
(10), this validates that the simultaneous adjoint process is aligned with the forward pass at each iteration in terms of adversarial robustness.
We also study the effect of unrolling different intermediate states { z n } for surrogate gradient estimation.
Fig.
3- (b)/(c) illustrates the robustness of the early state z 3 in the unrolling-trained DEQ under white-box attacks in different settings.
It is shown that the number of unrolling steps for intermediate states like z 1 and z 2 should not be too much in order to obtain a powerful intermediate attack.
The reason of this might be the inaccuracy of the unrolled intermediate gradient estimates.
The gradient estimated by unrolling z 1 leads to the most vigorous attack on the robustness of z 3 .
To understand the circumstance, we note that the unrolling-based intermediate gradient reflects only the feedback from the loss function at the unrolled state in Eq.
(13) and Eq.
As a result, the estimated gradient may be misaligned with the unrolled state: gradients by unrolling z 3 compose weak attack in terms of the robustness of z 3 .
It is inferred that the perturbation from the unrolled intermediate gradients must still be propagated in the forward pass to induce enough threatening
[Table] [Table]
distortion.
This explains the delay that the unrolled gradient at z 1 affects the robustness of z 3 .
More ablation studies on the unrolled intermediates can be found in Appendix E.1.
6.5 Performance of the proposed attacks on vanilla DEQ models
In this section
[Table]
we evaluate the performance of the proposed attacks on the DEQ models without adversarial training.
We train a DEQ-Large on CIFAR-10 with standard training following the recipe in [5]
[Table]
and use the ready-made PGD-10 to attack the model.
The clean accuracy of each state z n , as well as its robust accuracy is shown in Table 5.
Different from the robustness accumulation effect in the adversarially-trained DEQs (shown in Sec.
6.2, Fig.
2-(a), and Appendix C), the ready-made
PGD-10 already has a dramatic effect in attacking all the states in the standardly-trained DEQ.
We proceed to apply all the proposed attacks and defense strategies.
Following Sec.
5.2, we determine the optimal timing for early exiting the standardly-trained DEQ as state z 1 .
Shown in Table 6, it can be seen that all the proposed attacks can defeat the DEQ by standard training.
As the white-box robustness of DEQs is assessed by the strongest defense under all attacks (minimum over all columns in a row, then maximum over the minimum of the rows), the white-box robustness of the vanilla DEQ is 1.31% with a 38.81% clean accuracy using the early-state defense.
When using the final-state and the ensemble-state defense, the robustness is 0%.
These results validate that all the proposed attacks are reliably strong as they all defeat the DEQ models without adversarial training.
Conclusion
We study the adversarial robustness of general DEQs, using the exact gradient and the unrolling-based phantom gradient in adversarial training for DEQs, respectively.
We observe the gradient obfuscation issues in DEQs under ready-made attacks.
Based on the misalignment between the forward and backward tracks, we leverage intermediate states in the forward pass to construct white-box attacks and defense strategies and benchmark the white-box robustness performance of DEQs.
While we have performed a serious comparison of white-box robustness between DEQs and deep networks, it can be seen that the performance of DEQs is on par with that of deep networks.
Our empirical observations indicate that we should explore more advanced AT mechanisms for DEQs, in order to exploit their local attractor structures.
A potential way is to explicitly encourage closed-loop control during training, similar to the mechanism introduced in [11].
To this end, the gradient estimation method proposed in this paper would be one of the critical ingredients for solving the misalignment between the forward/backward pass of DEQs.
Acknowledgements
This work was supported by the National Natural Science Foundation of China (No.61925601) and
Beijing Academy of Artificial Intelligence (BAAI).
We appreciate all of the anonymous reviewers for their comments and suggestions on this work.