Fine-Grained Iterative Adversarial Attacks with Limited Computation Budget
Authors: Zhichao Hou
Weizhi Gao
Xiaorui Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Adversarial Attack
Efficiency
Robustness
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: This work tackles a critical challenge in AI safety research under limited compute: given a fixed computation budget, how can one maximize the strength of iterative adversarial attacks? Coarsely reducing the number of attack iterations lowers cost but substantially weakens effectiveness. To fulfill the attainable attack efficacy within a constrained budget, we propose a fine-grained control mechanism that selectively recomputes layer activations across both iteration-wise and layer-wise levels. Extensive experiments show that our method consistently outperforms existing baselines at equal cost. Moreover, when integrated into adversarial training, it attains comparable  performance with only 30\% of the original budget.

Published as a conference paper at ICLR 2026
FINN-GRAINED ITERATIVE ADVERSARIAL ATTACKS WITH LIMITED COMPUTATION BUDGET
Zhichao Hou', Weizhi Xiaorui Liu' North Carolina State University { (zhou4, w www. (ww.:eexu...eeenuu.eeee @ncsu.edu
ABSTRACT
This work tackles a critical challenge in Al safety research under limited compute: given a fixed computation budget, how can one maximize the strength of iterative adversarial attacks? Coarsely reducing the number of attack iterations lowers cost but substantially weakens effectiveness. To fulfill the attainable attack efficacy within a constrained budget, propose fine-grate fine-grained control mechanism that selectively recomputes layer activations across both iteration-wise and layer-wise levels. Extensive experiments show that our method consistently outperforms ex- isting baselines at equal cost. Moreover, when integrated into adversarial training, it attains comparable performance with only 30% of the original budget.
INTRODUCTION
Adversarial attacks, which craft imperceptible perturbations to input data to degrade the perfor- mance of deep learning models, have become central topic in the safety and robustness of AT systems. From the attack perspective, iterative adversarial methods such as Projected Gradient De- scent (PGD) (Madry et al., (0117) are widely adopted as strong oracles to benchmark the robustness of modern neural networks. From the defense perspective, adversarial training (Madry et al. 2017] Zhang et al., 2019 Wang et al. 2019) has been validated as the most effective method, which criti- cally depends on strong iterative attacks to generate challenging adversaries during training.
Despite their broad impact, iterative attacks are notoriously expensive because each step requires both a forward and backward pass. In robustness evaluation, PGD with 20, 50, or even 100 steps typically used, which incurs roughly 40-200x the cost of natural inference. In adversarial training. the standard 10 procedure already costs around 10x more than natural training. As model architectures and datasets continue to grow, this heavy computational burden severely limits the scalability of both attacks and defenses for large-scale models. Normally, more computation in attacks (e.g., more iterative PGD steps) means stronger attack. This raises a critical question: Given preseribed computation budget, how can maximize attack strength?
This question is crucial in the era of increasingly large models and limited computational re- sources, especially for researchers and practitioners outside major industrial labs. To enhance attack strength and transferability. prior work has explored several complementary directions, including momentum-based updates (Dong al. 2017 Lin al. 2019). diverse input transformations (Xic) et al. 2019). and translation- invariaat gradients (Dog et al. 2019). Nevertheless, these approaches still incur substantial performance degradation when the iteration count is reduced and fail sustain high attack efficacy under strict computational constraints.
We attribute this limitation to common design choice: existing methods perform full- peel-preoisoon computation across all layers and treat iteration count the sole, coarse-grained lever for con- trolling cost. From a combinatoriallopiiiizztii
perspective, it unlikely to be optimal to restrict computation allocation to a fixed number of iterations applied uniformly every layer. As ob- serve in Section (322 during iterative attacks the activations across successive steps quickly become highly correlated, and the residual change rate varies across layers. This suggests that should employ more fine grained computation control at both the iteration-wist wiie and layer- wise levels.
Motivated by this insight, we propose the Spiking Iterative Attack event-driven iterative scheme that adaptively executes fine-grain -raigra control over activation computation. Specifically, the spiking mechanism performs full-precision computation only when the relative activation change exceeds

---

Published as a conference paper at ICLR 2026
threshold; otherwise, the previous output is reused. To avoid gradient vanishing caused by naive reuse, we introduce a virtual surrogate gradient that preserves informative backward signals while retaining most of the forward-pass savings. Our main contributions can be summarized as follows:
We identify and quantify substantial computation redundancy in multi-step iterative attacks, demonstrating that intermediate activations are highly correlated with strong layer-wise similarity.
We introduce a combinatorial optimization perspective to reveal limitations of existing attacks: the coarse graiiee computation allocation limits attack strength given computation budget.
We propose a spiking forward computation scheme to reduce redundant computation by adaptive computation. Moreover, a virtual surrogate gradient technique is proposed to preserve backward signals despite activation reuse. Together, the proposed attack algorithm (Spiking-PON) -GD)) ensures effective adversarial updates and enhances attack effectiveness under given computation budget.
We empirically demonstrate that Spiking- expands the tier: it achieves comparable or superior attack success rates at significantly lower computational cost on both vision and graph benchmarks. Moreover, incorporating the spiking mechanism into adversarial training reduces training cost without degrading final clean or robust accuracy.
effectiveness Pareto fron-
2 RELATED WORK
Adversarial attacks have become a central topic in the safety and robustness of AI systems, with a large body of work dedicated to improving their attack efficacy and efficiency. The Fast Gradient Sign Method (FGSM) (Goodfellow et al. 2014) introduced a simple one-step gradient update, while its iterative variant I-FGSM (Kurakin al. (2018) achieved higher success rates by applying multiple small steps. Building on these foundations, several refinements have enhanced attack strength and transferability while requiring a small number of iterations. The Projected Gradient Descent (PGD) attack (Madry et al., 2017), often considered the canonical iterative variant of FGSM, adds a random start to the update rule. MI-FGSM (Dong et al. 2017) incorporates momentum to stabilize gradient updates, DI-FGSM (Xie et al. 2019) applies diverse input transformations to improve transferability,
and extensions such as TI-FGSM (Dong et al. 2019) and NI-FGSM (Lin et al., 2019) further refine the optimization by introducing translation invariance and Nesterov momentum, respectively.
Beyond the white-box setting, efficiency has also been studied in black-box scenarios where query complexity is the dominant cost. Bandit-based methods (Ilyas et al. 2018) leverage gradient priors to reduce the number of queries, while decision based attacks such as Boundary Attack (Brendel et al.) 2017) and HopSkipJumpAAttakk (Chen et al.) 2020) progressively refine adversarial examples using only hard-label feedback. More recently, Square Attack (Andriushchenko et al.) (020)) demonstrated that simple random search over patchwise perturbations can achieve remarkable query efficiency. Sparse adversarial attacks, such as the One-Pixel Attack (Su et al., (019)) and SparseFool (Modas) al. 2019). further reduce perturbation cost by focusing on minimal, geometry- driven modifica- tions.
In summary, prior research has explored efficiency either by reducing the number of gradient steps, minimizing queries, or constraining perturbations. In contrast, our proposed Spiking iterative attack seeks to improve efficiency from a complementary and orthogonal perspective: by exerting fine- grained control over activation computation, it selectively skips redundant operations during iterative updates while retaining the strong adversarial strength.
PRELIMINARY
In this section, first present the necessary technical background gradient-based iterative ad- versarial attacks and their associated computational overhead. We then conduct a preliminary study that reveals significant redundancy in the computations across attack iterations.
Notations. Consider an L-layer model, where (-) denotes the 1-th layer in the model (1 E {I..... LJ). The input and output activations of (1 are denoted by (!) and (0 = (a) respectively. Consider a T step iterative attack that passes through the model T times, then we have o
(1) AA (a.)) at time step t (t E {I,...,7)).
2

---

Published as a conference paper at ICLR 2026
3.1
EXPENSIVE COMPUTATION ITERATIVE ADVERSARIAL ATTACKS.
Gradient-based iterative attacks (Madry et al.
2017 Kurakin et al. 2018 Dong et al.) 2017.
Algorithm Gradient-based Adversarial Attack
2019) construct adversarial examples by itera-
Require: clean input æ, label y, adversarial loss
tively perturbing the input increase the adver- sarial loss. Although individual methods differ only in minor implementation details, they fol- low the general procedure summarized in Al- gorithm [ Starting from initial point æ1, it performs a forward propagation to evaluate the loss and a backward propagation to obtain the gradient with respect to input data æ. The pro- jected gradient update step \ is the following: Le+1 = Ig, (2; + V (xx, ()), where L denotes the adversarial loss (e.g., cross-
L. attack budget e, step size a, iteration T.
Ensure: adversarial example x
1: σ1 <
initialization
2: for = 1,..,, do
(L) + L
3:
0: 02
Ot
O:
4:
80,
ac
Bo 1
Do [1)
ac
do D
ac
L
5: I++1 (x)
It
+
ac
6: end for
7: return 5 + -T+1
entropy), a is the step size, and (TB(()) projects
the perturbed data back onto the allowed perturbation set (x) given the attack budget e.
With T' iterations, iterative attacks produce much stronger adversaries than single-step methods, at the cost of T forward and backward passes. Although iterative adversarial attacks are among the most effective methods for evaluating model robustness, they are also extremely expensive to perform multiple gradient calculations per input, with each step involving both a forward and back- ward pass through the network. Furthermore, when these attacks are incorporated into adversarial training, the cost compounds, as each training step must generate new adversarial examples through repeated iterations, which limits its applicability in real-world large-scale deployments.
3.2
COMPUTATION REDUNDANCY IN ITERATIVE ADVERSARIAL ATTACKS.
Relative Error (x10-2)
x10-2
Normal Model
4.0
0.4
Robust Model
Relative Change
12-
435
814
1:45
211
2:41
1.58
2:10
3.25
2:55
1.00
0.35
315
132
3.5
0.3
Attack Iteration
T4
24-2222
100
1.0
L30
175
9.99
1.4
3
2.54
2.54
2:2
0.2
T5
LG
145
1.14
1.1
0.1
17
0.7
L3
133
LI
143
651
3:00
002
C77
LXX
3.75
1:2
212
125
L11
0.0
79
km
693
EMT
AA3 9.99
644
1.14
9%
WAS
1
2
3
4
6
7
8
9
10
T10
AM
0:38
0.3
0.30
0.2
X20
0.5
Attack Iteration
ls
L1
L12
133 L14
L17
Layer
Figure 1: Activation relative change - a1-111//ll -a-all
||ae|| || for R8 on CIFAR-10. Left: per- layer curves (light color) and layer average (dark color) for normally trained (red) and adversarially trained (blue) models. Right: heatmap of relative change across layers and attack iterations.
||a,-
As shown in Algorithm [ iterative attacks typically update the adversarial perturbation via pro- jected gradient steps until convergence. As a result, the perturbed input and the network's ' hidden activations change smoothly, especially in the later stages when the optimizer has largely stabilized. Motivated by this intuition, we conduct a preliminary study on CIFAR- 10 (Krizhevsky et al. 2009) with Re (He et al. 2016) to validate our argument. To be specific,
measure the relative between consecutive attack iterations t - and for each layer I. We refer to standard model trained without adversarial examples the "Normal Model", and to an adversarially trained model as the "Robust Model". Figure [ (Left) visualizes the relative activation change per layer using light-colored lines, while the dark lines represent the layer-wise average. Additionally, a heat map in Figure [] (Right) is used to illustrate the distribution of relative changes across iterations and layers. We can make the following observations from the results: (1) Figure (Left) shows that the activations (a." )T become highly similar after a small number of iterations, i.e., ||a." a.",l1/ ll ("I decreases rapidly. This indicates substantial redundancy in repeated computations across attack iterations. Additionally, the "Robust Model" exhibits markedly smaller activation changes than the "Normal Model"; (2) Figure [ (Right) shows that while all layers
activation change ||a ( a [| (")
3

---

Published as a conference paper at ICLR 2026
follow the same overall decreasing trend, the decay rate of relative activation change varies across layer. These findings motivate tterationwwis and layer-wise fine- one-grained control of computation.
4 SPIKING ITERATIVE ATTACK
In this section, we first formulate the combinatorial optimization problem underlying activation computation in iterative attacks and highlight the limitations of existing approaches, which impose coarse-grained constraints on the search space (tection[.11) We then introduce novel and efficient Spiking Iterative Attack that leverages fine-grained spiking- forward computation (Section 4.2)
in conjunction with faithful virtual surrogate gradient for the backward computation (Section 4.3
11
X2
2
41
x2
13
2
M
0
11
07
@
12)
(2)
X
4
(1)
03
X5
0
X4
4
33
1
X4
1
X5
5
(1)
12)
(2)
Early Stopping at Step S
X4
0
0
Xy
11
0f
(1)
0
U
(a) Vanilla Iterative Attack
(b) Coarse-Grained Attack
(c) Fine-Grained Attack
Figure 2: Three attack execution patterns: (a) Vanilla iterative attack: every layer I E [L] is fully computed at every attack iteration E [T]; (b) Coarse- Coarse-grained attack: all layers are computed for t S S and skipped for > S; and (c) Fine-graite gene-graind attack: selectively controls the computation scheme to maximize attack strength under given computational budget.
4.1 ITERATIVE ATTACK AS COMBINATORIAL OPTIMIZATION UNDER LIMITED BUDGET
Multi-step step iterative adversarial attacks are costly because each attack step typically performs a full forward and backward pass through the network. Let C, denote the computational cost at iteration t,
then the total cost scales as Σ Ct Under strict computation budget Croual. common strategy to reduce the iteration count in iterative attacks to some S < T (carly stopping; see Figure 2b),
which leads to the following coarse-grained problem:
max SE(1,..,77 ((x5-111) S.t. ΣG E Crotal:
(1)
where Ø5-1 denotes the perturbed input after S attack full iterations. enforces a block-structured computation pattern, which severely restricts the admissible computation schedules, making the coarse- mouse-grriined solution tend to be suboptimal.
This carly-stop constraint
We instead consider fine-grained control over which layer computations are executed at each it- eration (Figure 2 (c)). Decomposing the per-iteration cost by layer, write C. Σ C..-- Let A = (ōe,) 1)T'X be binary mask where ò,, 1 indicates that layer / fully com- puted at attack iteration f, while εt,1 = 0 indicates reuse of previously computed activations. The corresponding fine-grained grained optimization is the following:
T
L
max AE(0,1)Tx2
(((T+1(A),))
S.t.
ΣΣ [=1==
Cip 8eg S Crotal:
where (A) denotes the final perturbed input after T attack steps executed under mask A. The following proposition formalizes the relationship between the two formulations: the coarse- coarse-grained problem Eq. ( is subproblem of the fine fine-grained problem Eq. (2)
Proposition 4.1. Let Veoarse and Vine denote the optimal objective values of Eq. (7) and Eq. ( respectively, then we have: Voearse I Vine- The feasible set of Eq. ① can be embedded into the
feasible set of Eq. (2) by masks that compute layers for iterations t S S and skip layers for I. >>> S.
4

---

Published as a conference paper at ICLR 2026
Remark 4.2. Proposition 4. shows that etrly-stopping is block-structurred subset of the more expressive layer-wise masking decisions. The lineggrinee formulation can therefore improve attack strength under the same budget, at the expense of larger combinatorial search space.
So far we have introduced fine-grained layer-wise optimization as alternative coarse- coarse-grained early stopping. Despite its intuitive appeal, solving the fine-grained problem in Eq. 22) is challeng-
ing for the following reasons: (1) Vast search space: The binary mask A E {0, 1j x induces combinatorial search space of size (22xxL) lead exhaustive search; (2) Expensive objective evaluations: Each evaluation of the attack objective ((x (4). v) requires simulating the entire masked forward/backward computation under A, eliminates the practicality of classic optimization methods; and (3) Broken gradient flow: Reusing cached activations breaks the usual computation graph, leading to vanishing gradient with respect to the input perturbation (See Section 4.3
To address these issues, we propose a novel spiking forward computation scheme (Section 4.2) that enables fine-grained control of per-layer computation based on our preliminary study of redundant computation during iterative attacks. To mitigate the resulting
we introduce vir- tual surrogate gradient (Section 4.3 that preserves gradient information during back-propagation.
4.2 SPIKING FORWARD COMPUTATION
To solve the combinatorial optimization in Eq. (2). one may naturally consider range of classical approaches such as brute-force enumeration, greedy heuristics, dynamic programming (Bellman 1957), continuous relaxations (Raghavan & Thompson| 1987).
variety of meta-heurise algo-
rithms (Holland 1975 Koza 1992] Hansen & Ostermeier] [200]] Kirkpatrick et al. 1983). However, in our setting they become impractical: the decision variable A E [0,1) induces an exponen- tially large search space, and each candidate mask requires an expensive end-to-end attack simula- tion evaluate C(xT+1(A) y). Motivated by the empirical redundancy observed in Section 3.2 we propose an efficient spiking forward computation scheme that enforces layer-wise recomputation via a single tunable threshold parameter p E [0. 1]. This threshold controls lightweight per-layer selector to enable strong attack efficacy while meeting a prescribed computational budget.
Consider linear mapping A in the model, with sequential inputs (a, (a!")T_ Yr= and outputs VT where ( = (a,") Exploiting the linearity of A vanilla forward computation can be reformulated shown in Figure (left), where each output is expressed the accumulation of the previous output plus the residual between consecutive activations. This view naturally suggests that many computations are redundant when residuals are small.
o ① = (a
o o = ("
0 ( (a = (a a! + (9
0
02
= =1
(1)
+ o
of (a (a - + (
(1)
0 0000
(S,
of (a (a
of of A 1
Figure 3: Comparison of vanilla forward computation and spiking forward computation for adver- sarial attacks. Left: In vanilla forward computation, the current output is composed of the previous output and the activation residual, leveraging the linearity of ( Right: Spiking forward compu- tation applies a spiking function S, to the residual to selectively update the activations.
Motivated by empirical observations in Section 3.2 that activations across attack iterations are highly correlated, we introduce a spiking mechanism determine whether an update should be triggered:
(a: a:-1), |a: aa-11///|aaa 2 P: (a. 1 a,-1,, ||a a--ll//a/aa < p:
a.-1) (0-
(3)
where p is a pre-defined threshold controlling the sensitivity of the spiking gate. Intuitively, the gate "fires" only when the relative change in activation is sufficiently large, thereby avoiding unnecessary recomputation for negligible updates.
This spiking formulation leads to two distinct cases: (1) If ||a, - -a-11l//i 11/l1///ee/ || is larger than p. we
compute a new output: (! (a) (1)
a t2 A0 (a( ); and (2) if ||a at- |aall
5

---

Published as a conference paper at ICLR 2026
smaller than p. the update degenerates to: 0.") ( +0 o:-1 reusing the previous output and skipping redundant computation. Through this mechanism, the forward pass becomes event-driven rather than iteration-driven, leading substantial savings in computation. Importantly the spiking threshold p allows explicit control over the trade-off between efficiency and efficacy, enabling the attack to flexibly adapt different resource budgets.
4.3
VIRTUAL BACKWARD COMPUTATION
0 --1
AT
0
t-1
1 O2
A
0
0
at
a2
C
a2
4 C
A
(a) Exact Gradient
(b) Vanishing Gradient
(c) Virtual Gradient
Figure 4: Gradient computation mechanisms. (a) Exact gradient #C
A
BC
when the layer
Sa.
D0,
output o, is freshly computed, the gradient w.r.t a follows the standard chain rule. (b) Vanishing 8C O: when o, is reused from Ôz-1, the spiking gate blocks the path to S6.
gradient #c Ta.
yieldinigg zero gradient. (c) Virtual gradient BC = AAT AC_ our virtual surrogate gradient
0a.
(Jo.-1)
manually restores the backward path by manually applying A to the upstream gradient.
Gradient Vanishing, When computing backward gradient in the spiking computation described
in Section 4.2 the default torch. autoozz backward pass follows the chain rule through the spiking gate. For a linear layer (1),, we obtain
OC (J) da!
Oc
ac 06
|a. a
| 2 p. + Figure (a)
Sp -
00,
0,
la: at a.a-1l/ll1ll a|/ p, Figure
where S is the derivative of the spiking mask. The second case corresponds to Figure (b): because
the activation reused, the forward value o!") set to the cached 0 and the aut ograd engine produces a zero gradient a!") Practically, this results in gradient vanishing for at and thus prevents the input perturbation æ from receiving a useful update via this layer.
Virtual Surrogate Gradient. To address this problem, we introduce a virtual surrogate gradient that restores a more faithful backward signal when the spiking gate suppresses the standard gradi- ent. Conceptually (Figure + (c)), we manually inject an approximate backward mapping from the reused output 6,-1 back to the current input a;, replacing the zero that autogras rad would other- wise propagate. Concretely, our implementation proceeds in two steps: (1) Step One: We mark the cached output "!"h as requiring gradients via requires aC/06("] and store it during the backward pass; (2) Step Two: When the layer activation is reused, we retrieve the cached upstream gradient OC/00 and manually compute the surrogate gradi- ent A " (ac/a6 by placing the A00) between 0f21 and ( in the computational graph via eegister.hook. In practice, this implemented efficiently using the corresponding low- level gradient primitives (c.g., torch.nn. nn. ww......nnnaa........mm condd.corvecc innuu for convolutional layers, or matrix-mi multipliitiiiiitiinnn for dense layers).
() so that we can capture
With the surrogate gradient in place. the backward rule becomes a piecewise function: if a layer is fully computed, we rely on the true backward provided by autograd: if reused, we substitute the surrogate gradient via register hook. This hybrid rule restores meaningful gradient flow to the input while preserving the forward-pass savings. overview of the full Spiking Iterative Attack with PGD is given in Algorithm
EXPERIMENT
In this section, we first validate the effectiveness of the proposed Spiking Iterative Attack in vision and graph domains. We then incorporate Spiking-PGD into adversarial training to substantially improve efficiency without compromising final performance. Finally, we conduct several ablation studies to analyze the underlying mechanisms and behaviors of Spiking-PGD.

---

Published as a conference paper at ICLR 2026
Algorithm 2 Spiking-PGD
Require: clean input label y, adversarial loss L, attack budget e, step size a, iterations T. Ensure: adversarial example
1: 11 < +
optional random start ò s.t. ||ó11 S else ô <- 0
2: for = I....,T T do
3: for ( == 1 to L do
(2)
1-
(8)
ift > 1 and
P. P.
4:
0
aa
forward propagation
A at
if = or
la
5: end for
6:
for -= to 1 do
AD
0C
if > 1 and
11 (
0.-1 p.
7:
da
ac
15
A
ac
if
or
||
||
p.
> backward propagation
86
||a
8:
end for
9: D++1 < (x)
Ts + a sign( (i,
> adversary update
10: end for
11: return ã t -T++1
5.1 EXPERIMENTAL SETTINGS
Datasets & Backbone Models. We conduct the experiments several datasets: for vision task,
use CIFAR 10 Krizhevsky et al. 2009). CIFAR 100 (Krizhessky et al.
2009) and Iiny-Im (Le)
& Yang 2015;; ; for graph task,
Cora and Citeseer (Sen al.
2008). For backbone models,
we select R8 (He et al. 2016) for vision task and GCN Kipf
2016) for graph task.
Baseline Attacks. For the vision task, we evaluate iterative attacks including PGD (Madry al., 2017). I-FGSM (Kurakin et al.) 2018), and MI-FGSM (Dong et al.] (017).. For the graph task, we evaluate the PGD topology attack (Xu al., 2019). We follow all the settings in the original papers except for the number of attack iterations T.
Computation Cost. We set the reference number of iterations To 20 for vision tasks and To = 200 for graph tasks. For baseline iterative attacks with T iterations, the relative computational cost defined as T T/T, x 100%. For our Spiking-PGD the relative computational cost is measured by the proportion of full-precision operations executed over the entire iterative attack in the model.
5.2 ADVERSARIAL ATTACK STRENGTH
To evaluate the strength of different adversarial attacks under any prescribed budget, report model accuracy under attack across a range of relative computational costs (see Section 5. for details on cost measurement). For baseline iterative attacks. we vary the number of iterations T span low- to high-budget regimes, corresponding to relative costs of T/To x 100%. For our Spiking-PGD. we adjust the spiking threshold p E [0, 1] obtain operating points corresponding to different proportions of full updates. We conduct our evaluation on vision tasks (CIFAR-10. CIFAR-100, and Tiny- ImegeNe)) and graph tasks (Cora and Citeseer).
CIFAR-10
CIFAR-100
Imny--mageNee
HFGSM
I-FOSM
46
I-GSMM
Accurscy (%5)
RGD
(90)
PGD
Accurscc P50
44
PGD
80
SpikingPGO AutoAttack
accurso
30
AutoAttack
Spiking-GGD
82
AutoAttack
Spiking-PG
90
30
10
20
30
40
50
60
70
80
90
100
10
20
30
40
50
60
70
80
90
100
10 20
30
40 50 60 70 100
Computation
(%)
Computation
Figure 5: Comparison of model accuracy under attack versus computation cost on CIFAR-10, CIFAR-100, and Tiny-ImageNer. Spiking-PGDD consistently achieves better attack strength than baseline iterative attacks (I-FGSM, MI-FGSM, PGD, AutoAttack), with the performance gap most pronounced in the low-computation regime.
7

---

Published as a conference paper at ICLR 2026
Pixel Perturbation on Images. FirureSssreeents comparison of attack strength with various com- putation costs for our Spiking-PGD and several baseline methods across three vision benchmarks: CIFAR-10. CIFAR-100, and Inn-eem..NNe.. Several key observations can be made:
Across all datasets, for any given computation cost, Spiking-PGD consistently achieves higher attack strength than baseline iterative attacks. The performance gap is especially pronounced in the low-budget regime, highlighting that Spiking-PGD utilizes computation more efficiently. This advantage stems from its fine- one-grained control over computation via the spiking threshold, in contrast the coarse- coarse-grained iteration based control used by conventional methods.
Among baseline attacks, MI-FGSM outperforms I-FGSM and PGD due to the use of momen- tum in gradient updates to stabilizes optimization. However, when the number of iterations is reduced to lower computation cost, MI-FGSM suffers a significant drop in attack strength. This indicates that iteration reduction alone cannot preserve effectiveness, leaving considerable room for improvement- an efficiency gap that Spiking-PCO PGD bridges through fine- ine-grained computation allocation. Additionally, although AutoAttack offers strong attack strength, it does so at a sub- stantial computational cost. Under an equal computation budget, however, AutoAttack performs worse than the other attacks.
Structure Attack on Graphs. Fig- ure [ shows results on two citation network benchmarks (Cora, Citeseer) for PGD topology attacks and our Spiking-PGD We can make similar observations as vision task: (1) PGD attack strength falls sharply when reduce computational effort. (2) The Spiking-PGD achieves notice- ably higher attack strength than PGD across most budgets. The gap is par- ticularly clear at small-to-moderate budgets, indicating that reusing com- puted quantities in graph updates pre-
Cora
Citeseer
85
75
PGD Spiking-GDD
Accuracy (%)
80
Spicing-PGD
Accuracy (16
70
65
60
70
55
30
20
Computation Cost (%)
Computation Cost (%)
Figure 6: Comparison of model accuracy under attack ver- sus computation cost on graph datasets (Cora and Citeseer). Spiking-PGre PGD consistently achieves stronger attack perfor- mance than PGD across all computation budgets, with a par- ticularly large advantage in the low-cost regime.
serves enough signal to guide structure perturbations effectively.
5.3 EFFICIENT ADVERSARIAL TRAINING
100
Insl. A-1.0
80
1.10
Nettrral (ierall))
Natural (4m22000
50
Avg.
Robrot (hanifal
Rickaai (km2.00 20
Inst. A-5.0
Oi
Nationll 3.00
Preciston (%)
3.0.0
40
Accuracy
30
10
25 300 75
30C
125
130
275
200
25
50
73
100
325
230
175
200
Epoch
Epoch
Figure 7: Accuracy curves of adversarial train- ing on CIFAR-10. Clean (solid) and robust (dot- ted) accuracy for vanilla PGD-AT and Spiking- PGD with different decay rates X.
Figure 8: Precision curves of adversarial train- ing on CIFAR-10. Instantaneous (shaded) and average (dotted) precision for Spiking-PGr with different decay rates X.
Beyond evaluating iterative attacks, we also integrate Spiking-PGD into adversarial training to re- duce the cost of generating perturbations. In this setting, we investigate two scheduling strategies for controlling the spiking threshold p: a constant threshold and an exponential decay schedule. For the latter, the threshold at epoch t is defined as p(t) Po threshold, N = 200 is the total number of epochs, and X is a hyperparameter that controls the decay rate. By construction, p(0) = Pu and P(T) = 0. Intuitively, this schedule produces weaker (and cheaper) perturbations in the early stages of training, while progressively increasing attack strength and precision as training proceeds. This gradual refinement allows the model to stabilize early while still converging robust solution.
-M/N- where Po 0.1 is the initial
8

---

Published as a conference paper at ICLR 2026
To better understand the training dynamics, visualize how accuracy and precision evolve across epochs during adversarial training with Spiking-PGD, For accuracy curves in Figure 7 we track both clean and robust accuracy throughout training. For precision curves in Figure we distinguish between the instantaneous precision at each epoch and the cumulative average precision over time. From the results, we can make key observations follows: (1) Between epochs 25-50, robust accu- racy drops sharply as low thresholds cause frequent reuses. As p decays and more full computations are introduced, robust accuracy recovers and aligns with PGD-AT, showing that adversarial training can correct early inaccuracies once stronger perturbations appear. (2) The decay parameter À con- trols how quickly P approaches zero. Smaller (e.g.. 1.0) prolongs efficiency but delays robustness recovery, while larger À (e.g., 3.0) accelerates refinement, yielding smoother curves closer to PGD- AT. (3) With = 2.0, Spiking-PGD nearly matches PGD-AT best clean and robust accuracy while using under 30% of the computation.
5.4
ABLATION STUDY
100
Robust Accuracy
70
w/a Virtual Gradient
0000-0
Accuracy Cost
80
Computation Cost
Accuracy (%)
Virtual Gradient
Accuracy 1%6
60
60
2/255
40
50
50
4/255
8/255
20
40
40
15
0 10 20 30 40 50 60 70 80 90100
0. 10 20 30 40 50 60 70 80 90100 Computation Cost (%)
Threshold p (x10-2)
Computation Cost (%)
(a) Ablation on threshold p
(b) Ablation on virtual gradient.
(c) Ablation on attack radius.
Figure 9: Ablation studies of Spiking-PGr PGD.. (a) Effect of spiking threshold on robust accuracy and computation cost. (b) Impact of the virtual surrogate gradient. (c) Effect of attack radius (2/255, 4/255, 8/255), where smaller radii lead to slower degradation in accuracy.
Threshold p. Figure 9a illustrates the sensitivity of attack performance to the spiking threshold p. As p increases, computation cost decreases substantially since more operations skipped, while attack strength decreases only moderately. This trade-off indicates that a higher threshold yields significant efficiency gains at the cost of slight reduction in attack effectiveness.
Virtual Surrogate Gradient. To validate the effectiveness of our virtual surrogate gradient, we conduct an ablation study shown in Figure 9b With the virtual surrogate gradient, attack strength improves substantially, especially under low computation cost. This demonstrates that the virtual surrogate gradient provides faithful signal that enables stronger adversarial perturbations when computational resources are limited.
Attack Radius e. To examine the effect of the attack radius on Spiking-PGD. we evaluate three set- tings: é = 2/255, 4/255, and 8/255. From Figure 9c, we observe that Spiking-PGD yields greater efficiency improvements under smaller radii. With small radius, attack strength decreases more gradually as computation cost is reduced, since the induced activation changes are less pronounced compared to larger radii.
6 CONCLUSION
Adversarial attacks have become cornerstone of Al safety research, serving reliable means for robustness evaluation and system safeguarding. However, iterative attacks introduce substan- tial computational overhead, which limits both academic research and industrial deployment in the fields of Al safety and robustness. While reducing the number of attack iterations can significantly cut costs, comes the expense of attack strength. In this work, we propose spiking iterative at- tacks, which employ fine-grained forward computation to reduce overhead and virtual surrogate gradient to preserve backward signals during activation reuse. Our experiments demonstrate the effectiveness of this method across various baselines under constrained budgets. Moreover, when integrated into adversarial training, our approach reduces computational cost by up to 70% without compromising final performance. In addition, the method naturally extends to black-box attacks that rely on iterative gradient estimation, since these attacks also produce temporally correlated queries that benefit from activation reuse. Overall, this method offers orthogonal and complementary per- spective to existing techniques, expanding the crucial implications for the advancement of AI safety.
Pareto frontier and carrying
9

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This paper investigates efficient attack methods, which are valuable for evaluating and enhancing model robustness. However, acknowledge the potential risk that such insights could be misused to target existing secure models. Beyond this, we have not identified any ethical concerns related to human subjects, data release practices, conflicts of interest sponsorship, discrimination, bias or fairness, or issues of research integrity.
REPRODUCIBILITY STATEMENT
We provide comprehensive details to facilitate the reproduction of our experiments. Specifically, the datasets, models, and attack methods described in Section 5.1 along with the hyperparameters used in our proposed method. The code will be released upon paper acceptance.
ACKNOWLEDGMENT
Weizhi Gao, Zhichao Hou, and Xiaorui Liu are supported by the National Science Foundation (NSF) under grant number IIS-2443182.
REFERENCES
Maksym Andriushchenko. Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square tack: a query-efficient black-bor box adversarial attack via random search. In European conference computer vision, pp. 484 501. Springer, 2020.
R. Bellman. Dynamic Programming. Princeton University Press, 1957.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. arXiv preprini arXiv: 2017.
Jianbo Chen, Michael I Jordan, and Martin J Wainwright. Hopskipjumpattack: A query-efficient decision-based attack. In 2020 ieee symposium on security and privacy (sp), pp. 1277-1294. IEEE, 2020.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Discovering adversarial examples with momentum. arXiv preprint arXiv:1710.066 5, 2017.
Yinpeng Dong, Tianyu Pang Hang Su, and Jun Zhu. Evading defenses transferable adversar- ial examples by conslation-invariii attacks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4312-4321, 2019.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint
2014.
Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 9(2):159-195, 2001.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
John H. Holland. Adaptation Natural and Artificial Systems. University of Michigan Press, 1975.
Andrew Ilyas, Logan Engstrom, and Aleksander Madry. Prior convictions: Black-box adversarial attacks with bandits and priors. arXiv preprint arXiv:1807.07978. 2018.
TN Kipf. Semi-supervised classification with graph convolutional networks. arXiv preprint 1009.09000 2016.
Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. Science, 220 (4598):671-680, 1983.
10

---

Published as a conference paper at ICLR 2026
John R. Koza. Genetic Programming: On the Programming Computers by Means of Natural Selection. MIT Press, 1992.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples the physical world. In Artificial intelligence safety and security, pp. 99-112. Chapman and Hall/CRC, 2018.
Yann Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, (7):3,2015
Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E Hopcroft. Nesterov accelerated gradient and scale invariance for adversarial attacks. arXiv preprint arXiv: 06281, 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. arXiv preprint arXiv:1706.06083 2017.
Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Sparsefool: few pixels make big difference. Proceedings of the IEEE/CVF conference on computer vision and pattem recognition, pp. 9087-9096, 2019.
P. Raghavan and C. D. Thompson. Randomized rounding: A technique for provably good algorithms and algorithmic proofs. Combinatorica, 7:365-374, 1987.
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad Rad. Collective classification network data. magazine, 29(3):93-9)) 93, 2008
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 23(5):828 841, 2019.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International conference on learning represenlations, 2019.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang. Zhou Ren, and Alan L Yuille. Improving transferability of adversarial examples with input diversity. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2730-2739, 2019.
Kaidi Xu, Hongge Chen, Sijia Liu, Pin-Yu Chen, Tsui-Wei Wei Weng. Mingyi Hong, and Xue Lin. Topology attack and defense for graph neural networks: An optimization perspective. arXiv 2019.
preprint arXiv:1906.044
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pp. 7482-7482. PMLR, 2019.

---

Published as a conference paper at ICLR 2026
A REDUNDANCY IN ITERATIVE ATTACK.
To reveal the redundancy present in iterative attacks, we report the relative changes of activations and gradients in Table D and Table 2 respectively. The results show that both activations and gradients quickly converge to highly similar values after only a few iterations. This indicates that repeatedly recomputing them across all iterations computationally wasteful. Moreover, the decay rates are not uniform across layers- some layers stabilize much faster than others. These observations highlight the need for fine-grained computation control, both at the iteration- wwie and layer-wise levels, in order to effectively reduce redundancy while preserving attack strength.
Nornal Model
lteration Layer
3
5
0
5
10
11
12
13
14
16
17
18
0,42
0,072
0.129
0.100
0,164
0.235
0.112
0213
0.365
0.385
0.009
0.078
0.046
0.081
0.062
0.098
0.143
0.136
0.171
0.166
0.088
0.166
0.198
0.327
0.198
0,008
0.022
0.036
0.005
0.064
0,049
0.068
0.075
0.104
0.096
0.108
0.091
0.056
0.094
0.105
0.100
0.164
0.100
0.007 0.019 0.031 0.030
0.055
0,042
0.057
0.063
0.089
0.08]
0.089
0.077
0.046
0.077
0.087
0,081
0.120
0.081
0,022
0.007 0.019 0.030
0.029
0.084
0,012
0,588
0.066
0.094
0.085
0.02
0.077
0.44
0.076
0.81
0.071
0.091
0.071
0.007 0.019 0.031 0.030
0.056
0.043
0.061
0.069 0.099
0.089
0.977
0.082
0.099
0.082
0.067
0.078
0.000
0.078
0.007 0.099 0.031
0.010
0,056
0.041
0.061
0.000 0.099
0.080
0.077
0.083
0.050
0.082
0.87
0.079
0.099
0.079
0.007
0.019
0.032
0.031
0.057
0,044
0.061
0.069
0.088
0.068
0.095
0.079
0.046
0.078
0.082
0.076
0.088
0.076
Robust Midill
Teration Layer
0.012 01011
0.072
0.002
0.083
0.027
000300
0.050
0.044
0.037
0.946
0.053
0.074
0.065
0.083
0.065
0.001
0.065
0.007 (1.006
0.009
0.000
0.015
0.000
0.011
0.010
0.014
0,012
0.015
0.016
0.023
0,020
0.026
0.020
0.026
0/020
0,005
0.006
0.006
0.009
0.010
0.004
0.003
0.004
0.006
0.005
0.05
8.004
0.006
0.066
0.009
0.008
0.006
0.007
0.004 01003
0.004
0.004
0.005
0.004
0.004
8.004
0.005
01004
0.05
0.005
0.077
0.007
0.088
0.006
0.006
0.006
0.004 0.033
0,004
0.004
0.005
0,044
0.004
0.044
0.055
0.004
0.055
0.005
0.07
0,007
0.088
0.00
01066
0.00
0,004
0.066
0.003
0.004
0.005
0.004
8:044
0.006
0.066
0.009
0.008
0.066
0.007
0.004
0.001
0.004
0.004
0.005
0.004
0.004
0.004
0.006
0.005
0.006
0.006
0.009
0.008
0.010
0.005
0.007
0.006
0.004
0.003
0.004
0.004
0.005
0.004
0.004
0.004
0.006
0.005
0.006
0,806
0.009
0,008
0.009
0.005
0.007
0.006
Table 1: Activation relative change ||t+ 0 |//l| for R8 on CIFAR-10.
Nomaa Midel
Itention Layer
11
16
0.376 0.387
0.726
0.531
0.569
0.440
0.585
0.604
0.606
0.630
0.663
0.648
0.661
0.669
0.688
0.679
0.700
0.768
0.167
0.190
0.420
0.334
0,346
0.244
0.370
0.415
0:457
0.485
0.516
0.505
0,516
0.526
0.552
0,541
0.572
0.661
0.172
0.321
0.017
0.042
0.188
0.231
0.097
0.314
0.380
0.423
0.444
0.457
0.476
0.400
0.514
0.562
0.670
0.015
0.041
0.224
0.239
0.266
0.100
0.400
0.331
0.392
0.422
0.184
0.473
0.488
0.508
0.536
0.533
0.576
0.684
0,020 0.042
0.153 0.239
0.346
0.102
0.388
0.327
0.393
0442
0.499
0.471
0.487
0.504
0.538
0,529
0.579
0.687
0.217
0.251
0.099
0.321
0.484
0.510
0.59
0.026 0.022
0.160
0.235
0.263
0.101
0.744
0.323
0.402
0.443
0.888
0.479
0.497
0.513
0.554
0.543
0.606
0.712
0.018 0.455
0.194
0.237
0.247
0.102
0.422
0.331
0.405
0.449
0.499
0.491
0.510
0.528
0.563
0.560
0.617
0.716
0,013
0,012
0.214
0,253
0.103
0.440
0.330
0.414
0.452
0.309
0.522
0.539
0.583
0.572
0.635
0,736
Model
Iteration Laver
11
12
13
14
16
17
18
0.058 0.183
0:411
0.218
0.345
0.255
0.233
0.011
0.086
0.067
0.102
0.096
0.097
0.136
0.154
0.157
0.159
0.155
0.13
0.154
0.171
0.159
0.163
0.201
0.007
0.058
0.040
0.069
0.098
0.003
0.155
0.125
0.143
0.148
0.122
0.145
0.145
0.145
0.149
0.148
0.153
0.189
0.006
0.050
0,054
0.071
0.0%
0.004
0.163
0.130
0.142
0.142
0.156
0.144
0.147
0.144
0.347
0.147
0.150
0.189
0.805
0.071
0.054
0.100
0.044
0.128
139
0,141
0.140
0.140
0.143
0.151
0.888
0.005
0.088
0.142
0.087
0.095
0.093
0.178
0.135
0.155
0.155
0.162
0.137
0.55
0.159
0.154
0.159
0.100
0.198
0.007
0.065
0.122
0.080
0.103
0.095
0.174
0.138
148
155
166
0.157
0.157
0.159
0.162
0.161
0.163
0.200
0.034
0.143
0.165
0,005
0.009
0.052
0.099
0.095
0.137
0,151
0.158
0.165
0.159
0.156
0.160
0.361
0.163
0.364
0.201
Table 2: Gradient relative change Vo. Vor,ffl/111
ClI for ResNet-18 on CIFAR-10.
12

---

Published as a conference paper at ICLR 2026
B ADVERSARIAL TRAINING
We integrate Spiking-PGin into adversarial training to reduce the cost of generating perturbations. In this setting, investigate scheduling strategies for controlling the spiking threshold pt constant threshold and an exponential decay schedule. For the latter, the threshold at epoch is defined p(t) po e-M/N-e" where Po 0.1 is the initial threshold, N 200 is the total number of epochs, and hyperparameter that controls the decay rate. By construction, p(0) Po and p(T) O.
Performance Analysis. Table summarizes clean aceuracy, robust accuracy, their sum, and the effective precision under different schedules (constant schedule and exponential decay schedule). Several trends emerge:
Spike-PGi achieves substantial computational savings while maintaining accuracy comparable to standard PGD-base adversarial training.
Efficiency:
Stability: Exponential decay schedules consistently outperform constant thresholds, offering smoother convergence and better robustness.
Trade-off control: By tuning X, practitioners can balance computational cost against training precision. Larger yields faster threshold decay, higher precision, and more accurate training dynamics, at the cost of increased computation.
Method
Clean (%)
Robust (%)
Sum. (%)
Precision (5)
Vanilla PGD-AT
80.18
48.75
128. 93
100
0.0
80.18
48.75
128.93
100
p= 0.01
80.47
49.04
129.51
98. 98.9934369883219
0.015
79.64
48.52
128. 16
95.7153006 1299882
p= 0.02
80.0
48.63
128.63
64, 64.66005577743944
Constant
0.025
80.52
48.78
129.3
45, 45.2051 19898780769
p= 0.03
81.38
45.37
126.75
32. 1615555407 (997
== 0.035
82.
41.15
123.26
51 51,5596288823900
p= 0.04
85,47
39.95
125.42
52.9638 (547788199 197
p= 0.045
88.38
39.72
128.1
26.0033435916685 665889
p=
86.48
37.59
124.07
14.2857 142857 428
1=
85,25
43,66
128.91
28.4462983220927
2=
82.33
47,45
129.78
44.056306580611
A=3.0
81.09
48,88
129.97
55.72839957238934
A
80.75
48.49
129.24
64.775761 5123344111334
Exponential
A=5.0
80,46
48.76
129.22
1L.832419994499
6=
80.0
48.64
128.64
71.76493522666
7=
80.4
48.87
129.27
1470022670225588
=
80.27
48.69
128.96
18.9746140001199
9=0
80.19
48.59
128.78
81.21360234915223
10.0
80.12
48.77
128.89
83.07328921906928
Table 3: Adversarial training results on CIFAR-10. Spike-PGD with exponential decay schedules consistently achieves competitive performance with significantly reduced computation.
13

---

Published as a conference paper at ICLR 2026
Accuracy Curves. To better understand the training dynamics under different schedules, vi- sualize the evolution of accuracy across epochs during adversarial training with Spiking-PGD in Figure T0 and Figure [I] Both clean and robust accuracy tracked throughout training. The results show that the exponential decay schedule leads to more consistent final performance with baseline compared the constant schedule.
80
Natural (Vanilla)
Robust (Vanilla)
Natural (p-0.02)
60
Robust (p=0.02)
Accuracy (%)
(p=0.025)
Robust (p=0.025)
Natural (p-0.03)
Robust (p=0.03)
Natural (p=0.035)
Robust (p-0.0335
Natural (p=0.04)
Robust (p-0.04)
Natural (p=0.045)
Rebust (p=0. (p-0.045)
Natural (p-0.05)
Robust (p=0.05)
25
50
75
100
125
150
175
200
Epoch
Figure 10: Accuracy curves of adversarial training on CIFAR-10. 10. Clean (solid) and robust (dotted)
accuracy for vanilla PGD-AT and Spiking-PGD with different threshold in constant schedule.
Natural (Vanilla)
Robust (Vanilla)
Natural (A=1.0)
Accuracy (%)
Robust (A-1.0)
Natural (A-2.0)
Robust (A-2.0
Natural (A=3.0)
Robust (A=3.0)
Natural (A=4.0)
Robust (A (4-4.0)
Natural AA-5.00
Robust (A=5.00
(A=6.00
Robust (6
Natural -
Robust (107.0)
25
50
100
125
150
175
200
Epoch
Figure 11: Accuracy curves of adversarial training on CIFAR-10. Clean (solid) and robust (dotted) accuracy for vanilla PGD-AT and Spiking-PGD with different decay rate À in exponential decay schedule.
14

---

Published as a conference paper at ICLR 2026
Precision Curves. Furthermore, visualize the evolution of precision across epochs during ad- versarial training with Spiking-PGD in Figure [2 and Figure [3] corresponding to the constant and exponential decay schedules, respectively. The results show that the exponential decay schedule achieves more stable precision control throughout training, whereas the constant schedule is highly sensitive the threshold setting.
80
70
Precision (%)
60
p-0.02
50
p=0.025
p=0.03
p-0.035
p=0.04
p=0.045
30
20
100 Epoch
200
Figure 12: Precision curves of adversarial training on CIFAR-10 with different threshold in con- stant schedule.
100
Precision (%)
A=1.0
A=2.0
A=3.0
A=4.0
A=5.0
A-6.0
A-7.0
0
50
75
100 Epoch
125
150
175
200
Figure 13: Precision curves of adversarial training on CIFAR-10 with different decay rate X in exponential decay schedule.
15

---

Published as a conference paper at ICLR 2026
C THEORETICAL PROOF
C.I PROOF OF PROPOSITION 1.1
Proposition 4.1 Let Veoarse and Vine denote the optimal objective values of Eq. 1 and Eq. 2
respectively, then we have:
Vecarse Vine-
The feasible set of Eq. ( can be embedded into the feasible of Eq. by masks that compute layers for iterations t < S and skip layers for t > S.
Proof. For any S E 10, :0....,,Ty T feasible in Eq. 1 (i.e., E Ct S Ctotal), define the correspond- ing mask:
11, t S S,
(5)
(t. T). LE (I....,.).
0, t> S,
The cost of A (8) equals Σ El= Ci. cu== E C₁, hence ( is feasible for Eq. 2 Moreover, executing the attack with A ( results in no updates after iteration so TT+1
øs+1-
Thus every feasible S for Eq. [ admits corresponding feasible mask A (5) with identical objective
value. Therefore the optimum over all S cannot exceed the optimum over all masks A, i.c. Veoarse Vitue.
16


Reviews:
[{"review_id": "bNIrZ4P6rq", "reviewer": "Reviewer_kGj7", "summary": "This paper proposes Spiking-PGD, a method for performing iterative adversarial attacks under a limited computational budget. The key idea is to skip layer computations in iterations where activations change little compared to previous iterations.", "questions": "1 Could the authors provide a more concrete breakdown of the computational cost in practical units, such as GPU FLOPs or runtime, instead of reporting it only as a proportion of full-precision operations? This would make it easier to evaluate the real-world efficiency of the proposed method.\n2 If feasible, please include experiments on larger models and datasets (e.g., ImageNet with ResNet50 or Vision Transformers) to verify whether the claimed improvements under a fixed computational budget generalize to realistic, large-scale settings.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1 The claim addresses a practically important problem: iterative adversarial attacks (and adversarial training) are expensive, especially for larger models/datasets. \n\n2 The redundancy study is a good supporting piece: showing that intermediate activations across iterations become similar, giving plausibility to “skip some computations” idea.", "weaknesses": "1 Computation Budget Definition Ambiguity.\nThe proposed method claims reduced computational cost by selectively skipping layer computations based on activation similarity. However, to determine whether to skip a layer, one must first assess the change in activations, which itself appears to require computing the activation. The paper does not clarify how this comparison is implemented without incurring similar cost to a standard forward pass. Consequently, the reported budget may underestimate the true computational cost.\n2 It is unclear whether the approach generalizes to other popular architectures (like Transformers in vision/NLP).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MZtOeqjUIb", "reviewer": "Reviewer_jTdB", "summary": "The paper proposes Spiking Iterative Attacks, a method to make iterative adversarial attacks more efficient under limited computation. By observing that layer activations have minor changes across attack iterations, the authors introduce fine-grained, iteration and layer-wise control that recomputes activations only when necessary, and use a virtual surrogate gradient to maintain gradient flow when activations are reused. This approach achieves comparable or stronger attacks at significantly lower computational cost and also speeds up adversarial training.", "questions": "1. Why did the authors choose not to include AutoAttack (AA) [ext_ref_1] or other modern robust evaluation suites in their experiments, given that these are widely considered standard baselines for assessing adversarial robustness today? Can the authors integrate AA into their experiments during rebuttal?\n2. Can the authors test the method on other architectures (e.g., VGG, WideResNet, or even better on transformer-based models) to validate whether the proposed approach generalizes across architectures?\n3. The authors show that activations across iterations become highly similar, yet iterative attacks continue to improve as iterations increase. How do the authors reconcile this apparent contradiction? What mechanisms explain the continued effectiveness of attacks even when activations stabilize?\n4. Can the authors elaborate more on why do standard relaxations or methods fail in solving equation 2?\n\nOverall, I liked the way in which the paper was presented and the provided method/contribution. However, there are some important limitations from the experimental perspective, as well as concepts that need further elaboration from my point of view, thus justifying my score. I am open to increase my score, however, based on how the authors will conduct rebuttal.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and easy to read. \n- The contribution is novel and interesting.", "weaknesses": "**Missing important baselines induces vague lack of contemporaneity:** The paper compares against standard iterative attacks (PGD, MI-FGSM, I-FGSM) but omits stronger, widely-used robust evaluation suites such as AutoAttack (AA) [ext_ref_1]. Without AA (or similar), it’s hard to evaluate the actual impact of Spiking-PGD. AutoAttack is now a de-facto standard for robust evaluation, and since it only allows modifying iterations, skipping it weakens the claim that Spiking-PGD genuinely expands the efficiency–effectiveness frontier. Unfortunately, the same can be said for the choice of using only ResNet18 models and usual datasets and not extending to transformers. Even by accepting the choice of not using modern architectures or datasets, to broaden the impact of the proposed approach it is significant to extend beyond the single ResNet18 architecture with different Deep Neural Network architectures, such as VGG and others. \n\n**Slight lack of clarity in some aspects:** \n- They show activations quickly become similar across iterations, motivating their methodology. Yet, iterative attacks still improve with more iterations. The paper lacks a clear mechanistic explanation reconciling these two facts. If activations stabilize, why do further iterations still yield stronger adversaries?  \n- Equation (2) is a discrete, combinatorial, non-convex optimization (NP-hard). The paper frames this but then presents a limited discussion on why other well-known strategies (continuous relaxations, differentiable masks, Gumbel-Softmax, integer programming relaxations, STE, or pruning literature methods) are unsuitable, or how they compare.\n\n[ext_ref_1]: Croce, Francesco, and Matthias Hein. \"Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks.\" International conference on machine learning. PMLR, 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JeodYXmKhJ", "reviewer": "Reviewer_uH8u", "summary": "This paper tackles the efficiency problem of iterative adversarial attacks, which are computationally expensive due to repeated forward and backward passes. The authors propose Spiking-PGD, featuring Spiking Forward Computation, which skips redundant layer computations based on activation changes, and Virtual Surrogate Gradient, which restores gradient flow when activations are reused. Formulated as a layer–iteration joint optimization problem, this approach enables fine-grained computation allocation. Experiments on multiple datasets show that Spiking-PGD significantly improves attack success rates under the same computational budget and reduces adversarial training costs by up to 70% without sacrificing performance.", "questions": "1. Please compare the GPU memory overhead with baselines.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1.The work demonstrates strong originality and innovation.\n2.The method is elegantly designed.\n3.The paper is well-structured and clearly written.", "weaknesses": "1.The paper lacks a released code repository, which limits reproducibility.\n2.My main concern is that since the proposed method reuses and stores previous activations, it is likely to increase memory consumption. As GPU memory is often a more critical bottleneck than time, corresponding experiments or analyses on memory overhead should be included.\n3.The vision experiments are limited to ResNet-18; it is recommended to include and discuss more diverse architectures, especially Transformer-based models.\n4.The evaluation should be extended to more diverse modalities, such as audio and NLP datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ezu31kRKwL", "reviewer": "Reviewer_3r6R", "summary": "This paper presents a timely approach to improving the computational efficiency of iterative adversarial attacks. To this end, authors replace the coarse-grained control of iteration count with a fine-grained, layer-wise spiking mechanism.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The topic this paper focused on is very interesting and novel. Besides, this paper is well-written.", "weaknesses": "1. Authors should report the magnitude of the error brought by using S_{\\rho}. That is, authors should examine the approximation error between the left and right part of Figure 3.\n\n2. Can the proposed method applied to the attack on large-scale dataset, such as the ImageNet dataset?\n\n3. Can the proposed method be well generalized to more iterative attacks, such as C&W? It would be better authors add a discussion for the application to the black-box attacks.\n\nminor:\n1. The condition in Algorithm 2 \"if $t=1$ or ...\" seems to have a typo (likely should be if $t>1$ or ... for the second case). The notation $∂^(l)_t$ is also not explicitly defined before its use. A more polished and commented algorithm would improve clarity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)