DRIFT: Divergent Response in Filtered Transformations for Robust Adversarial Defense
Authors: Amira Guesmi
Muhammad Shafique
Venue: ICLR 2026 Poster (2026)
Keywords: Adversarial Robustness
Transferability of Adversarial Attacks
Randomized Defenses
Gradient Consensus
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Deep neural networks remain highly vulnerable to adversarial examples, and most defenses collapse once gradients can be reliably estimated. We identify \emph{gradient consensus}—the tendency of randomized transformations to yield aligned gradients—as a key driver of adversarial transferability. Attackers exploit this consensus to construct perturbations that remain effective across transformations. We introduce \textbf{DRIFT} (Divergent Response in Filtered Transformations), a stochastic ensemble of lightweight, learnable filters trained to actively disrupt gradient consensus. Unlike prior randomized defenses that rely on gradient masking, DRIFT enforces \emph{gradient dissonance} by maximizing divergence in Jacobian- and logit-space responses while preserving natural predictions. Our contributions are threefold: (i) we formalize gradient consensus and provide a theoretical analysis linking consensus to transferability; (ii) we propose a consensus-divergence training strategy combining prediction consistency, Jacobian separation, logit-space separation, and adversarial robustness; and (iii) we show that DRIFT achieves substantial robustness gains on ImageNet across CNNs and Vision Transformers, outperforming state-of-the-art preprocessing, adversarial training, and diffusion-based defenses under adaptive white-box, transfer-based, and gradient-free attacks. DRIFT delivers these improvements with negligible runtime and memory cost, establishing gradient divergence as a practical and generalizable principle for adversarial defense.

Published as a conference paper at ICLR 2026
DRIFT: DIVERGENT RESPONSE IN FILTERED TRANSFORMATIONS FOR ROBUST ADVERSARIAL DEFENSE
Amira Guesmi & Muhammad Shafique eBrain Lab, Engineering Department New York University Abu Dhabi Abu Dhabi, UAE
ABSTRACT
Deep neural networks remain highly vulnerable to adversarial examples, and most defenses collapse once gradients can be reliably estimated. We identify gradient consensus- tte tendency of randomized transformations to yield aligned gradi- ents-as key driver of adversarial transferability. Attackers exploit this consen- sus to construct perturbations that remain effective across transformations. We introduce DRIFT (Divergent Response in Filtered Transformations), stochastic ensemble of lightweight, learnable filters trained to actively disrupt gradient con- sensus. Unlike prior randomized defenses that rely on gradient masking, DRIFT enforces gradieni dissonance by maximizing divergence in Jacobian- and logit- space responses while preserving natural predictions. Our contributions are three- fold: (i) we formalize gradient consensus and provide theoretical analysis link- ing consensus to transferability; (ii) we propose consensus-divergeno train- ing strategy combining prediction consistency, Jacobian separation, logit-space separation, and adversarial robustness; and (iii) we show that DRIFT achieves substantial robustness gains on ImageNet across CNNs and Vision Transformers, outperforming state-of-the-art preprocessing. adversarial training, and diffusion- based defenses under adaptive white-box, transfer- bansfrr-based, and gradient-free at- tacks. DRIFT delivers these improvements with negligible runtime and memory cost, establishing gradient divergence as practical and generalizable principle for adversarial defense.
INTRODUCTION
Despite the steady progress of adversarial defenses, most existing strategies collapse under adaptive attacks. Input transformations such as JPEG compression (Guo et al., 2018), randomized ensembles like BaRT (Raff et al., 2019), SIT Guesmi al. (2021), and even randomized smoothing (Cohen) eet al. 2019) all share critical weakness: their defenses still exhibit gradient consensus. An adap- tive adversary can approximate gradients across these transformations (c.g., via Expectation over Transformation (EoT) (Athalye et al. 2018)) and exploit the consistent directions that emerge, lead- ing to transferable adversarial examples. This vulnerability stems not from insufficient randomness, but from the fact that most stochastic defenses still preserve a coherent, low-variance surrogate gra- dient landscape.
We argue that true robustness requires not masking gradients, but destroying their alignment. If the gradients through different transformations diverge, then an attacker aggregating them obtains noisy and incoherent signals, severely limiting transferability. Crucially, this principle stands in contrast to prior defenses: unlike BaRT, we do not rely on hand-designed random transformations; unlike randomized smoothing, do not certify robustness by averaging over smooth noise distributions; and unlike obfuscated defenses (Athalye et al. 2018), we remain fully differentiable, avoiding the pitfall of false robustness.
Recent approaches such as DifiPure and DiffDefense (Nie et al. 2022 Silva et al. 2023) attempt to reverse adversarial perturbations using diffusion models, effectively projecting inputs back onto

---

Published as a conference paper at ICLR 2026
Base-PGD (M-only)
adv ΣΣ <(M (x 6x))).
+ 8M
Jacobian-Space Separation
(x)
M
ss Eg |cos² (H.o,{f,o)
M
Lc - Σ, e(M(f())..))
[0, 1]
Pairwise
VJPs
{(t,))
fz
M
Logit- Separation (pairs)
M
4 Lupp = Bw (V, (w, (a ()))
Logit- VIP vs Identity L Lvjp = C.E.lloo' fú. (2(M fra. w))]
Total loss
L af + BisLis + Pavi Livi
Lavsp
+ AC adv
Figure I: DRIFT methodology. Left: ensemble learnable filters (plus identity) feeding frozen base model M. Right: separation losses Jacobian VJPs and logit VJPs (including vs. identity). The base-only PGD and baseline clean performance preservation loss. The total objective sums all terms with weights (a, 3p, Syipp A).
the data manifold. While these methods achieve strong robustness on small- and medium seale datasets, they computationally prohibitive for ImageNet-scale tasks and unsuitable for real-time deployment. Moreover, their robustness stems from reconstructing "clean" versions of adversar- ial examples, which remains vulnerable if the attacker incorporates the purification step into the optimization loop. In contrast, DRIFT is lightweight and online: instead of purifying inputs, we adversarially train differentiable filters that directly disrupt gradient alignment, providing robust- ness without heavy generative modeling or inference overhead. We introduce DRIFT (Divergent Response in Filtered Transformations). lightweight and aechitecture-annssii an ensemble of differentiable, learnable filters to explicitly maximize gradient divergence while preserving clean accuracy. Each filter is small, efficient, and operates as a front-end to frozen pretrained model. Through a tailored training objective combining (i) prediction consistency, (ii) Jacobian-space divergence, (iii) logit-space divergence, and (iv) adversarial robustness, DRIFT en- sures that while clean predictions remain stable, adversarial optimization encounters conflicting, decorrelated gradient directions. This breaks the attacker's ability to rely on gradient consensus, under BPDA and EoT.
defense that trains
Our contributions are threefold:
We formalize the concept of gradient consensus and prove that reducing gradient alignment across transformations directly lowers adversarial transfcrability.
We propose DRIFT, the first differentiable and adversarially trained filter-ensemble de- fense that explicitly enforces gradient divergence without modifying retraining the back- bone classifier.
We demonstrate on ImageNet-scall models (ResNet-v2, Inception-v3, DeiT-S, ViT-B/16) that DRIFT consistently outperforms state-of-the-arr wrassormationn-ased and stochastic defenses against strong adaptive attacks, including PGD-EoT, AutoAttack, transfer-based attacks, and BPDA.
Our findings show that breaking gradient consensus a general principle for reliable stochastic defenses. By making gradients diverge rather than disappear, DRIFT provides robustness that is both effective and compatible with real-world, large-scale classifiers.
2 RELATED WORK
Defending deep neural networks against adversarial attacks has inspired broad range of strategics, including input transformations, stochastic preprocessing, adversarial training, architectural modifi-
2

---

Published as a conference paper at ICLR 2026
cations, and generative purification. Below summarize the most relevant categories, focusing on methods included in our evaluation. Early works rely on fixed transformations to suppress adversar- ial noise. JPEG compression (Dziugaite al. 2016) reduces wigh-frequence perturbations but often degrades clean accuracy and collapses under adaptive white-box attacks. BakT (Raff et al., (019) applies randomized blurs, noise, and color shifts at inference obfuscate gradients. While offer- ing some robustness, these transformations are non-differentiable and not optimized for adversarial resilience, leaving them vulnerable BPDA-styLe attacks.
Adversarial Training (AT) (Madry et al. 2018) remains one of the most widely adopted defenses, retraining models directly on adversarial examples. Despite its robustness improvements. AT is computationally costly, tends to reduce clean accuracy, and does not generalize well to unseen threat models. Several variants attempt to mitigate these issues, but the core trade-offs remain. Architec- tural modifications incorporate robustness into the model design itself. ANF (Suresh et al., 2025) inserts large-kemell convolutional filters and pooling layers at the input to denoise perturbations, but the approach is deterministic and static. Frequency-basee strategies such as FFR (Lukasik et al.) 2023) regularize filters in the Fourier domain to suppress high-frequency vulnerabilities. These methods improve robustness for CNNs but do not easily transfer to more diverse architectures such as Vision Transformers. Generative purification strategies, such as DiffPure (Nie et al.] (022),, re- verse adversarial perturbations by projecting inputs back onto the data manifold via score-based diffusion models. While highly effective on small datasets Silva et al. (203). these approaches are computationally prohibitive for large-scale or real-time scenarios, limiting their practicality.
In contrast to these strategies, our defense DRIFT introduces a stochastic, differentiable front-end composed of an ensemble of learnable filters. Crucially, these filters are trained to maximize gradi- ent divergence across members, directly disrupting adversarial transferability. Unlike BaRT, DRIFT does not rely on fixed randomness but learns diverse filters optimized for robustness. Unlike ad- versarial training or diffusion-basee purification. DRIFT is lightweight, modular, and alug-and-play with any pretrained classifier, requiring no modification or retraining of the backbone. This makes DRIFT both practical and theoretically grounded against strong adaptive attacks such as BPDA and EOT.
GRADIENT CONSENSUS ANALYSIS
3.1 CORE COMPONENTS
Notation. Let M IRd + R be pretrained classifier that maps an input x. E Rd to logits (z) E R with supervised loss {(z.y) for label y E {1..... K). We introduce a bank of lightweight, differentiable, dimension-preserving filters (f:) each fi = and define the
filtered pipeline (x) = (x)(x)). Throughout, Jy(x)= = dg(2)
Bi denotes the Jacobian of g at x.
Chain rule and gradient factors. By the chain rule, the input gradient of the loss through pipeline F; decomposes as:
V. (F(())))))) = (x) Vgl(z,)) (x) Ju((f(z)) Vzl(z,
with (ff(z)). Eq. makes explicit that adversarial directions shaped jointly by: (i) the logis-space factor V.f. and (ii) the input-output Jacobians Jas and J DRIFT exploits this factorization by decoupling these components across filters, so that shared (consensus) adversarial directions are disrupted.
Vector-Jacobian products (VJP). Forming full Jacobians is infeasible in high dimensions. Instead, reverse-mode AD computes (x) E Rd, v E R" without ever materializing Jş(x). These VJPs quantify how output directions v backpropagate the input. Sampling from Rademacher/GGasssin distributions (Hutchinson probing) yields scalable estimates of Jacobian (dmiiaaiyy across filters without storing Jacobians.
RRK probing with random w E K gives the logit-space VJP: gi (z; w) = V. (2(x), w) (x) w. This provides a tractable surrogate of how each fi reshapes sensitivity of decision-space directions with respect to the input. In practice, (x w) closely tracks adversarial update directions produced by first-order attacks.
Logit-space probing. For the composed mapping x + 2(x) [te)=M((()))
Adversarial examples and transfer. An adversarial example x' = x + with ||11 < é is typically found by iterative first-order methods (e.g., PGD). Transferability is the tendency of 8 crafted
3

---

Published as a conference paper at ICLR 2026
surrogate to fool a different target. By minimizing cross- filter consensus in Eq. DRIFT makes harder for a single surrogate surrogate-induced induued direction to generalize across pipelines or models, thus reducing black-box success.
Dimension-preserviing residual filters. Each filter fú keeps the input shape and is implemented with lightweight residual block so (x) R is square and M processes fi()) without architectural changes. This design keeps runtime small while providing enough flexibility to steer gradient geometry.
3.2 GRADIENT CONSENSUS
Attack Success Probability, For an input (r,y), let gi(x)) v=((M(((()))) y) denote the gradient of the supervised loss through filter fi and base model M. Given perturbation with || ooo < 6, we define the attack success probability on pipeline F.(x) = M (f((x)) as pz( 6)
IP arg max, M (fi(x +))e + v] A perturbation is transferable if pj (z, 6) is high
even when 5 was optimized on fr.
Definition 3.1 (Gradient Consensus). The gradient consensus between two filters fi fj input is defined as
[f(,ff;)) = -11 (1)())) (z)
This squared cosine similarity lies in [0, 1]. High values indicate that fi and fn share adversarially useful directions (high transferability), while low values indicate divergence of gradient subspaces (low transferability)
3.3 ASSUMPTIONS
Assumption 3.2 (Smoothness). The base model M and filters {fi} L-smooth: their gradients are Lipschitz continuous with constant L.
Assumption 3.3 (Bounded Gradients). There exists G such that [19(())]12 for all i and all inputs x.
3.4 THEORETICAL RESULTS
Lemma 3.4 (Transferability and Consensus). Let ê be an adversarial perturbation of size |0]1oo S crafted using gradient Then, under Assumptions 1-2, the expected attack success probability on fg satisfies 6, S C- I'ff, fyj x some constant C depending on the Lipschitz constant L and the loss margin at æ.
Proof sketch. A first-order Taylor expansion gives {(M(s{z + 8)),9) (MM() + (g, (x). Choosing 8 aligned with (x(x) yields an inner product (z))) normalizing and squaring the cosine similarity, the transfer effect is scaled by l'(fi, fziz). Ø). Bound- edness (=)]l G) and smoothness ensure the residual terms controlled, leading to the probability bound up to a constant factor C.
By
D
Theorem 3.5 (Breaking Consensus Reduces Transferability). Suppose filters {fi..... fnd satisfy Assumptions 1-2. If the expected consensus satisfies Eiz; [f fūiz)] I p < 1, then for any adversarial perturbation crafted on a single filter fe, expected transfer success probability on the other fiterss satisfies Ejzi 8)] S O(GGp).
Remark 3.6 (Identity Path). Including the identity mapping fia(x) = in training ensures that adversaries relying solely on M's 's gradients are also discouraged. This removes the "M-only" blind spot and forces robustness even against attacks that ignore filter structure.
This theory motivates the DRIFT objective: minimizing empirical gradi- ent consensus. Concretely, the Jacobian separation loss L.s reduces alignment feature space, and the logit-VJP separation loss LLVJP reduces alignment in decision space. Together, they enforce the low-p regime required by Theorem 3.5 thereby provably reducing transferability of adversarial examples across filters.
Connection to DRIFT.
4

---

Published as a conference paper at ICLR 2026
4 METHODOLOGY: DRIFT
The consensus analysis of Section 3 shows that transferability is controlled by the alignment of gra- dients across filters. To make adversarial examples -on-transferable, must explicitly reduce this alignment. As shown in Figure 11 DRIFT (Divergent Response in Filtered Transformations) opera- tionalizes this insight by introducing trainable preprocessing filters and losses that enforce Jacobian- and logit-level divergence. Together with adversarial training on base gradients, this creates a system robust both non-adaptive and adaptive attacks.
4.1 Loss COMPONENTS
DRIFT integrates four complementary objectives designed to balance clean accuracy with robust- ness and gradient diversity:
Cross-Entropy Loss. To preserve baseline predictive performance, we apply standard supervised
training across all filters: Le
Ei=1 ((M((. 3)
Jacobian Separation Loss. To reduce cross-filter transferabilitty, we explicitly penalize alignment between the vector-Jacobiaa products (VJPs) of different filters:
LJs = Eiey E. [coss"(()) (Jr, (x) v, Jr, (x) v)] where e is a random probe vector. High LJs implies shared adversarial directions, which DRIFT suppresses.
Logit- VJP Separation Loss. Beyond raw Jacobians, we also enforce divergence at the deci- sion level. By probing the logit space with random directions w E R² we obtain gradients (f, w). We then penalize their pairwise cosine similarity:
LLVJP = Eies Ew [cos" (Vo vV (M (f)(z)) w), (f) w))] This term ensures that filters remain diverse with respect to how input perturbations propagate into class decisions.
Adversarial Training Loss. To maintain robustness under direct attack, we craft adversarial per- turbations EAS using PGD on the base model M alone. The filters are then trained resist these perturbations: Lado attacks crafted in the base model's gradient space.
max; (f + ()r)).).. This enforces that each filter can withstand
Total Objective. The complete training loss is weighted combination:
= a LCE + BJs LJs + JL LL + X Ladv. By jointly optimizing these terms, DRIFT encourages filters that (i) preserve clean accuracy, (ii) diverge in Jacobian subspaces, and (ii) resist both base-model and cross-filt filttr attacks.
lightweight residual convolutional block: 3 x convolution expanding from 3 to 16 channels, a ReLU nonlinearity, and second 3 3x3 convo- lution projecting back to 3 channels. The filter output is added to the input via skip connection: This design ensures that each filter remains close to the identity mapping while still being capable of learning meaningful transformations that diversify adversarial gradients. A detailed exploration of the filter architecture (Appendix (A.4) and the effect of ensemble size on robustness (Appendix AA3)) further support the design decisions of DRIFT
Filter Architecture. Each filter fi is implemented
((x) x- =
EXPERIMENTAL SETUP
Dataset. We conduct experiments the ImageNet dataset (Krizhevsky et al. 2017). Following common practice, we use a randomly selected subset of the validation set for training the filter en- semble, while reserving the remaining portion exclusively for evaluation. This ensures that the filters are trained without access to test samples, thereby providing fair assessment of robustness.
Models. We evaluate DRIFT across both convolutional and transformer-based architectures high- light its generality. Specifically, we use two widely adopted CNN models: Inception-v3 (Inc-v3) and ResNet-v2-50 (Res-v2) (Szegedy et al., 2016 He et al. 2016). For transformer- baased architectures, we include ViT-B/16 (Dosovitskiy et al. (022)) as a representative Vision Transformer and DeiT- (Touvron et al. 2021), data-efficient variant trained with distillation.
Baselines. We benchmark DRIFT against comprehensive set of strong baseline defenses spanning multiple categories. Input preprocessing defenses include deterministic and stochastic transforma- tions such as JPEG compression (Dziugaite et al.) (016)) and BaRT (Raff et al. 2019). Generative defenses are represented by biffusion-based purification via DiffPure (Ne et al. (022). Architecture- level defenses include adversarial noise filtering (ANF) (Suresh et al. 2025 and frequency-baar based
regularization strategies such as filter frequency regularization (FFR) (Lukasik et al.) (023). Finally,
include the widely adopted adversarial training (AT) (Madry al. 2018 canonical robust-
5

---

Published as a conference paper at ICLR 2026
ness baseline.
Evaluation Metries. We evaluate defense performance using the standard Robust Accuracy (RA), defined as the proportion of adversarial examples that are successfully classified by the target model. Higher RA (t) indicates stronger defense and baseline (standard) accuracy, performance of the model in benign setting (i.e., no attack).
Parameter Settings. For training the set of filters, we use four filters formed by two convolu- tion layers, we use e 4/255. The number of PGD iterations size of = c/T = 0.4/255.
T 10, with step 1, 8js 0.5, Stejp = 0.5, À = 1, jss.sssssprrrr probs = 5, 5, epochs 100. For the optimizer, we use AdamW with ir le 3,
lejp_num huppnummprobb
weighlldecce
le - 4.
Attacks. We evaluate DRIFT under comprehensive suite of strong white-box and black-box at- tacks. Gradient-baae attucks include the canonical &o Projected Gradient Descent (PGD) (Madry) cct al.) 2018). momentum- basedd MI-FGSM (MIM) (Dong et al.) 2018). variance-reduced VMI- FGSM (VMI) (Wang & He 202I). and gradient-smoothing Skip Gradient Method (SGM) (Wu
ct al. 2020). To benchmark against state-of-the-ait evaluation protocols, we also include AutoAt-
tack (AA) (Croce & Hein 2020b). Gradient-free attacks are represented by the Square Attack (An-
driushchenko et al. (020)) and the Fast Adaptive Boundary Attack (FAB) (Croce & Hein 2020a).
which probe robustness without relying on gradient information. Finally, model adaptive adver- saries aware of the defense mechanism, we incorporate BPDA (Backward Pass Differentiable Ap- proximation) and EOT (Expectation over Transformation) (Athalye et al.) 2018), which are widely recognized for breaking obfuscated or randomized defenses. For completeness, Appendix A.I presents the full threat model and detailed implementation our method, along with training pseudocode.
RESULTS AND ANALYSIS
6.1 NON-ADAPTIVE ATTACKS
We first evaluate DRIFT in a non-adaptive setting: the adversary has full white-box access the base classifier (architecture and weights) but is unaware of the deployed defense. Table I reports robust accuracy across four backbone models (ResNet-v2,, Inception-v3, DeiT-S, and ViT-B/16) against eight commonly used attacks at fixed perturbation budget of e 4/255 for -bsed attacks and
1 for l attacks. The results show that DRIFT consistently preserves baseline performance, unlike JPEG compression and BaRT, which significantly degrade accuracy even in the absence of attacks. For example, on ResNet-v2, DRIFT maintains 84.66% clean accuracy compared to only 44.97% with JPEG q 50. At the same time, DRIFT provides substantially higher robustness across all attacks. Against AutoAttack, DRIFT achieves 74.30% robust accuracy on ResNet-v2, sur- passing DiffPure (67.01% and far exceeding JPEG (14.29% at q 75). These results demonstrate that DRIFT offers a favorable trade-off: it preserves clean accuracy while achieving of-t-o---he-arr robustness across convolutional and transformer-basee models. In contrast, existing preprocessing- based methods either distort the input distribution or over-regularize the model, leading to severe drops in standard performance.
6.2 ADAPTIVE ATTACKS
Prior work has shown that many input-transformation defenses collapse under adaptive threat mod- els. In particular, (Athalye al. 2018) demonstrated that defenses relying on gradient masking or non-differentiabiiiy can be bypassed by BPDA (Backward Pass Differentiable Approximation), which substitutes the gradient of the transformation with the identity or Average Pool during back- propagation. Moreover, randomness-bes based defenses can be defeated with Expectation over Trans- formation (EOT), where the attacker averages gradients across multiple stochastic passes to approxi- mate the true gradient. We evaluate DRIFT against such adaptive attacks by considering both BPDA and BOT settings. As shown Table 2 classical input transformations like JPEG and BaRT col- lapse under adaptive PGD and Attack, with robust accuracy dropping close to zero across all models. Adversarial training variants (AT, FFR+AT, ANF+AT) exhibit partial robustness but incur significant drops in clean accuracy and remain vulnerable when the attacker leverages EOT. DiffPure achieves moderate robustness but is highly sensitive to configuration. For DiffPure, computing full gradients under BPDA+EOT was infeasible our hardware due to the prohibitive memory required

---

Published as a conference paper at ICLR 2026
Table I: Robust accuracy (%) various defenses against seven attacks at noise budget e = 4/255
for &o and 1 for E2 attacks. Best results per row block (model) in bold.
Model
Defense
Config.
Attack
PGD E.
PGD fg
MIM
VMI
SGM
AA
FAB
Square
JPEG
62.56
35.45
41.00
6.88
19.05
37.04
14.
74.07
15.34
JPEG
44.97
41.27
62.43
23.81
29.65
40.74
8.99
(5.61
8.47
ResNet-v2
BaRT
À =
50.79
23.28
37.57
13.76
24.34
21.87
12.70
43.39
15.34
BakT
38.10
24.34
18.52
22.22
9.52
43.39
12.17
DiftPure
e = 0.15
67.79
65.43
70.64
48.73
45.66
47.20
67.01
63. 12
62.88
Ours
84.66
76.19
79.53
67.20
53.44
71.43
74.30
81.16
JPEG
==
78.31
6.88
28.77
2.65
3.17
5.82
1.23
58.90
6.30
JPEG
76.72
37.57
39.12
14,29
11.11
36.51
7.64
54.55
5.65
Inception-v3
BaRT
61.38
27.51
29.31
17.46
20.11
26.46
8.77
34.32
9.12
BaRT
30.79
29.63
31,20
23.28
22.21
24.87
6,45
35 65.65
8.44
DuftPure
0.15
NA
NA
NA
NA
NA
NA
NA
NA
Ours
4
80.96
76.83
78.74
73.50
65.00
77.83
76,50
80.10
79.89
JPE
4
82.54
2.65
22.75
1.50
3.17
2.65
4.23
81.48
67.72
JPEG
50
80.95
19.05
51.85
8.99
10.05
17.99
28.57
81.48
66.14
DefT.S
BakT
5
74.07
28.57
34,92
19.05
19.58
26.98
29.57
49.74
53,97
BaRT
10
63.49
35,45
37.57
26.46
29.10
34.39
31.22
52.91
50.26
DiffPure
0.15
73.63
61.55
67.55
47.61
45,30
60.70
63.21
58.32
$7.77
Ours
82.42
76.67
78.89
69.37
62.49
71.48
76.24
81.23
80.07
JPEG
75
77.25
8.47
31.75
6.35
25.93
8.47
10.58
71.96
59.79
JPEG
BaRT
# 5
68.78
31.22
38.10
25.40
36.51
34.39
26.98
50.79
50.26
VIT-B/16
BaRT
54.50
33.33
36.51
28.04
30.69
31.10
30.16
47.62
42.33
DiffPure
0.15
NA
NA
NA
NA
NA
NA
NA
NA
NA
Ours
80.48
74.66
77.01
70.95
63.90
75.19
77.30
79.83
77.30
Table 2: Robust accuracy (%) of various defenses against adaptive PGD, and AutoAttack (AA)
(e 4/255, 40 steps) across four models. The best results each column are highlighted in bold.
ResNettv2
Inception-v3
DefT.S PGD
ViT-B/I6
Defense
Adaptive
Clean
PGD
AA
Clean
PGD
AA
Clean
AA
Clean
PGD
AA
JPEG BaRT
BPDA FOT
44.97
II
0
76.72
0
0
80.95
0
74.07
BPDA BOT
50.79
6.0
0
61.38
11.23
944
74.07
5.2
3.1
68.78
7,31
4.67
16.32
2,4
7.3
AFR+AT
EOT
56.85
20.53
13.24
NA
NA
NA
NA
NA
NA
NA
NA
NA
ANF+AT
EOT
61.67
25.12
24.63
NA
NA
NA
NA
NA
DiflPure
FOT
67.79
36.43
40.93
NA
NA
73.63
37.55
43.18
NA
NA
DiffPure
BPDA + BOT
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
53.78
47.97
Ours
BPDA BOT
84,66
60.19
58.73
80,96
53.68
51.11
82.42
57.22
55.43
80.48
64.17
61,23
backpropagate through the diffusion sampler (Nie et al. 2022 In contrast, DRIFT maintains both clean accuracy and strong robustness across convolutional and transformer-based models. For exam- ple, Res DRIFT achieves 60.19% robust accuracy against BPDA+EOT PGD and 58%7 against BPDA+EOT AutoAttack, outperforming all baselines by large margin. On Inception-v3 and DeiT-S, DRIFT sustains over 50% robust accuracy against both PGD and AutoAttack, while JPEG and BaRT collapse below 10%. On ViT-B/16, DRIFT achieves 64.17% and 61.23% robust accuracy under PGD and AutoAttack respectively, again substantially higher than all competitors.
6.3 DRIFT vs. RANDOMIZED SMOOTHING BASELINES
Randomized smoothing (RS) reports certified top-1 accuracy for an E ball of radius r. i.e., attack- agnostic guarantees that the prediction invariant any perturbation ||ō]|2 S (Cohen al. 2019 Salman et al. 2019). In contrast, DRIFT is empirical defense evaluated with adaptive white- box attacks (&, PGD-EOT); the numbers below are empirical robust accuracies under & attacks at the same radii Table B juxtaposes the standard RS basclines with DRIFT on ImageNet-1K for 50 and ViT-B/I6. On ResNet-50, DRIFT exceeds SmoothAdy RS by +9.1 to +1.55 points grows from 0.5 to 3.0. On ViT-B/16. DRIFT outperforms CAF from r=0.5 to 3.0 with gains between +0.2 and +13.4 points. We stress that RS values are certificates, whereas DRIFT values empirical and should not be interpreted as certified guarantees.
6.4 ABLATION STUDIES

---

Published as a conference paper at ICLR 2026
To better understand the contribution of each component in DRIFT, we conduct ablation ex- periments by selectively including exclud- ing loss terms during training. Table 4 reports robust accuracy under both non-adaptive and adaptive PGD attacks for four different models. We begin with baseline that combines stan- dard cross-entropy loss Lee with adversarial training Lade- While this setup provides rea- sonable non-adaptive robustness (e.g., 75.66% on ResNet-v2), collapses almost completely under adaptive attacks (below 10% across all models). Introducing Jacobian-Space Separa- tion (CJs) substantially improves adaptive ro- bustmess, boosting performance to 39. 39.80% on ResNet-v2 and similar gains across other chitectures. Logit-VJP Separation (LLVJP) proves even more effective, further elevating
Table 3: Robust Accuracy (%) at & radius r on ImageNet-IK. ResNet-50 uses SmoothAdv ran- domized smoothing (Salman et al. 2019, Cohen
2019); ViT-B/I6 uses Certitying Adapters et al.
(CAF) Deng et al. 2024). DRIFT rows are em- pirical cates).
Lg robbustnss the same radii (not certifi-
Model
Method
0,5
1.5
3.0
ResNew:0
SmouthAdy
36.00
45.00
38.00
28.00
20.00
RosNes-50
DRIFT
65.13
55.34
50.78
45.66
27.45
V/T-B/16
CAF (RSls
71.80
53.60
45.80
34.20
21.20
71.96
51.23
30.51
Notes. RS entries (SmoothAdw/CAF) are certified
accuracies; DRIFT entries are empirical accuracies under &2 PGD--OT attacks at the same radii. Certified and empirical numbers should not be compared as if equivalent guarantees.
adaptive robustness to 47. .6%1 on ResNet-v2 and consistently outperforming LJS across models. Finally, combining all three components (LCE. LJs. LLV and Lodv) yields the strongest de- fense. This full configuration achieves the highest adaptive robustness across all architectures, with ResNet-v2 at 53. 78%, Inception-ves v3 at 50.40%, DeïT-S at 48. 15%, and V/T-B/16 at 56. 74%. Impor- tantly, this robustness comes at no cost to non-adaptive performance, which remains on par with or slightly better than the baselines.
Table 4: Robust accuracy (%) of different loss component configurations against adaptive PGD
across four models.
ResNei-v2
Inception-v3 Non-ariativee
DeiT-S
VIT-B/16
Loss Componeuts
Non-adaptive
Adaptive
Adaptive
Non-ndaptive
Adaptive
Non-ndaptive
Adaptive
Log + Euto fos Eis fule Cos + ELyIp + Lude
75.66
3.70
77.25
2.65
79.55
9.52
76.12
8.47
77.21
39.80
78.64
38.54
77.90
36.71
75.34
40.12
76.43
47.61
77.53
45.11
78.65
40.87
76.50
49.73
Rog Egyup + E.s Cuds
76.
53.78
78.83
50.40
78.67
48.15
74.66
56.74
6.5 GRADIENT-NORM SANITY & DINTEE-DIFERENCE CHECK
To rule out gradient obfuscation, low the diagnostic in Athalye et al. (2018): Tramer et al. (2020): for each (z. y) we
fol-
Table 5: D1 diagnostic on ViT-B/16 for Im- ageNet (subset). Gradients well-behaved; the directional-derivatie mismatch By is small across step sizes with tight tails.
(defense ON) and compare the directional derivative v V.C against finiteedifferenc slope. We use BPDA (identity surrogate), EOT over the defense's stochasticity, common randomness (CRN) across paired evaluations, and centered differences,
measure
Input-graddeent morms V. 22
median
pis
p95
1.6677
0,4392
5.756
with 7) E (10-2, 2, 10-3, 10-4) and 10 random unit Lę di- rections per sample. Table [] summarizes the
2n
Directional mismatch Ay
v.C.
(izerw
17
median
meon
p45
p95
1 10-2
0.00562
0.01 0.01884 184
0.02055
0,02300
results (medians/means and 5/95 percentiles across the evaluated subset). Gradients are neither vanishing nor exploding (median 1.67, 5-95% 0.44-5.72). The directional mismatch remains in the 10 10-3 10-3_10-2 range with tight tails (p95 < 4.12 10- 10-2), indicating informative, non-masked gradients under our defense.
10-3
0.00773
0.00106
0.02133
1x 10
0,00477
0.01570
0.00133
0.04114
Setup. Identity BPDA surrogate;
expectation transtsss (EOT) with common randomness: centered finite differences: 10 random unit L2 directions per sample. Lower A, is better.
6.6 LOSS-LANDSCAPE SMOOTHNESS
8

---

Published as a conference paper at ICLR 2026
To assess whether defense induces masking artifacts, we visualize the loss surface around imput along two random, orthonormal di-
Loss landscape 'samole O1
rections (u, v) in input space, plotting f(x + [-T,-]²,
0,09872
bv, (a, grid with = 3/255. For stochastic compo- nents, we evaluate the expected loss via EOT- 128 and use common randomness (CRN) so every grid point shares the same random filter sequence. This follows best-practics diagnos- ties for ruling out gradient obfuscation (Atha- lye et al. 2018 Tramer et al., 2020). Fig- ure 2isows a smooth, near-planar surface with and mild anisotropy (slope larger along one axis), with no checkerboard ing the colorbar, the total loss variation across the square small (AL R 1x1x01=5x00" consistent with informative (oon-vanshhnng but well-behaved gradients. The landscape around is smooth and free of quantization barriers or ceddomness-inducce, platcaus, indicating that our defense does not rely on gradient obfuscation per this diagnostic. Further analysis and discussion are provided in appendix A.3
41 x41
c.0
C.000
0.09559
-0.010-
-0010
-0.005
0.000
0.005
0.030
monotone shading syellow->purppe
plateau artifacts. Read-
Figure 2: Loss-landscape smoothness. The surface smooth and nearly planar over [-3/255, 3/255]² with a coherent slope and no staircase/plateau artifacts.
6.7 RUNTIME EFFICIENCY: DRIFT vs. PURIFICATION DEFENSES
Table 6 reports per-image inference cost on Ima- geNet (ResNet-50). DiffPure Nie et al. (2022) re- quires 5.52s, 11.06 s, or 17.07 s per image at timesteps t E {0.05, 0.10, 0.15), with 7B. GPU mem- ory. In contrast, DRIFT takes only 0.0004 (0.4 (0. and 0.03 GB. This corresponds to speedups of roughly 1.4 10º, 2.8 x 1 and 4.3 x 10d over DiffPure at 1" 0.05, 0.100, 0.15, respectively, while using about ~ 2 less memory. In sum, DRIFT delivers adap- tive robustness with latency and memory footprint compatible with real-time and resource-constrained settings, whereas diffusion-barsd based purification is orders of magnitude costlier.
Table 6: Inference latency and memory
comparison between DiftPure and DRIFT
on ImageNet (ResNet-50).
Timestep
Latency
Memory
Defense
(t")
(s)
(GB)
DiffPure
0.05
5.52
~70
DiffPure
0.10
11.06
~00
DiffPure
0.15
17.07
770
DRIFT
N/A
0.0004
0.03
7 CONCLUSION
We introduced DRIFT, a lightweight and architcttur-agnostii defense framework designed to break gradient consensus, a central vulnerability of transformation-based defenses. Unlike prior pre- processing or smoothing methods that preserve coherent gradients and thus remain exploitable under adaptive attacks, DRIFT leverages ensembles of differentiable and learnable filters trained max- imize gradient divergence while preserving clean accuracy. Our theoretical analysis established a formal link between gradient consensus and adversarial transferability, showing that reducing align- ment among filters directly limits the success of transferable attacks. Building on this insight, we proposed a principled training strategy that combines cross-entropy, Jacobian separation, logit- VJP separation, and adversarial robustness objectives. Extensive experiments on ImageNet-scale CNN and ViT architectures demonstrated that DRIFT consistently outperforms state-of-the-art prepro- cessing, adversarial training, and diffusion-based purification defenses. Notably, DRIFT preserved baseline performance under non-adaptive adaptive attacks, achieved strong robustness against semi-adaptis and adaptive attacks (including BPDA and EoT), and scaled effectively to both convolutional and transformer backbones. These results underscore DRIFT's practicality as a defense that is efficient, modular, and deployable without retraining or modifying the base classifier.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported in parts by the NYUAD Center for Cyber Security (CCS), funded by Tamkeen under the NYUAD Research Institute Award G1104.
REFERENCES
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square at- tack: a query-efficient black- black-box&a adversarial attack via random search. In European conference computer vision, pp. 484-504 Springer, 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense se- curity: Circumventing defenses to adversarial examples. In International Conference on Machine Learning (ICML), 2018.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. international conference on machine learning, pp. 1310-1320. PMLR, 2019.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. In International conference on machine learning, pp. 2196-2205. PMLR, 2020a.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference machine learning, pp. 2206- 2216. PMLR, 2020b.
Jieren Deng, Hanbin Hong, Aaron Palmer, Xin Zhou, Jinbo Bi, Kaleel Mahmood, Yuan Hong, and Derek Aguiar. Certifying adapters: Enabling and enhancing the certification of classifier adversarial robustness. arXiv preprint arXiv:2405.16003 2024.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9185-911811111 99939,993.. 2018.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x words: Transformers for image recognition at scale. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. BeLLhttps://openzeeee id=YicbFdNTTy
net forum?
Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M. Roy. A study of the effect of jpg compression adversarial images, 2016. URL https:t//txxxx...././// wwwrxiiv.orggbbo hatp///b///// /1los///68.... 00853
Amira Guesmi, Ihsen Alouani, Mouna Baklouti, Tarek Frikha, and Mohamed Abid. Sit: Stochastic input transformation to defend against adversarial attacks deep neural networks. IEEE Design & Test, 39(3)x63-72, 2021.
Chuan Guo, Mayank Rana, Moustapha Cissé, and Laurens van der Maaten. Countering adversarial images using input transformations. In 6th International Conference on Learning Representa- tions, ICLR 2018, Vancouver, BC, Canada, April 30- May 3, 2018, Conference Track Proceed- ings. OvenReview.net, 2018. URL BtLLhtps::// http::::////peeeviiweee net/ httpppawwweiieeeeeeeoooreeeaaaoa
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. Communications of the ACM, 60(6):84-90, 2017.
Jovita Lukasik, Paul Gavrikov, Janis Keuper, and Margret Keuper. Improving native robustness with filter frequency regularization. Transactions on Machine Learning Research, 2023:1-36, 2023.
10

---

Published as a conference paper at ICLR 2026
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018.
Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Anima Anandkumar. Diffusion models for adversarial purification. In International Conference on Machine Learning (ICML). 2022.
Edward Raff, Jared Sylvester, Steven Forsyth, and Mark McLean. Barrage of random transforms for adversarially robust defense. In Proceedings of the IEEE/CVE conference computer vision and pattern recognition, pp. 6528-6537, 2019.
Hadi Salman, Jerry Li, Ilya Razenshteyn, Pengchuan Zhang, Huan Zhang, Sebastien Bubeck, and Greg Yang. Provably robust deep learning via adversarially trained smoothed classifiers. Ad- vances in neural information processing systems, 32, 2019.
Hondamunige Prasanna Silva, Lorenzo Seidenari, and Alberto Del Bimbo. Diffdefense: Defending against adversarial attacks via diffusion models. In International Conference on Image Analysis and Processing, pp. 430 442. Springer, 2023.
Janani Suresh, Nancy Nayak, and Sheetal Kalyani. First line of defense: Arobust first layer mitigates adversarial attacks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 7176-7183, 2025.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethink- ing the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818-2826, 2016.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pp. 10347-10357. PMLR, 2021.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. Advances in neural information processing systems, 33:1633- 1645, 2020.
Xiaosen Wang and Kun He. Enhancing the transferability of adversarial attacks through variance tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1924-1933, 2021.
Dongxian Wu, Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip connections mat- ter: On the transferability of adversarial examples generated with resnets. In 8th International Conference Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL httpp::////. http::s:////peeeveeieeee w

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1
MORE DETAILS OF EXPERIMENTAL SETTINGS
A.1.1 IMPLEMENTATION DETAILS OF OUR METHOD
Training Algorithm. The DRIFT filters are trained jointly with frozen base model M using composite loss. Each batch update includes: (i) preserving clean accuracy via cross-entropy; (ii) enforcing gradient diversity through Jacobian separation; (iii) enforcing divergence in logit-space VJPs, including against the identity path; and (iv) adversarial training with base-PGD adversaries. Warmup schedules ensure stability by gradually activating each component. The procedure is sum- marized in Algorithm T
Algorithm 1: Training DRIFT Filters
Input: Frozen base model M; trainable filters JF {fi] dataloader weights
a, SJs. PLVJP, ?; warmups Wja Whvjp Wadvi PGD budget steps T, step size
Output: Trained filters
Define identity path [a)( =
for epoch = 1..... do
for batch (zz, 3)
do
// (i) Clean CE across filters
LCE + R E, (F()((()))))
+ a. LcEi
// (ii) Jacobian-space
separation (after warmup)
if epocch > w/s and K 2 then
Lis +- avg E,, [oos (J), (z) v)]:
C+ B1s LJs:
// (iii) Logit-WJP separation (after warmup) if epoch >> Wlvjp then
LIVIP < pairwise terms vs-identity terms with random logits ws;
C L+ 8INIP LINIP;
// (iv) Base-PGD adversary (after warmup)
12
if epoch Wadv then
δs + PGD(x, y M,e, T); Ladv max; (E(M(f((z ()))))) C+
14
// (v) Update
Backpropagate f; sanitize NaNs/Infs; clip grads; optimizer step;
A.1.2 IMPLEMENTATION DETAILS OF ADVERSARIAL ATTACKS
PGD: We implement fox and f variants of Projected Gradient Descent (PGD) Madry et al. (0018)
For &oo PGD we set e 4/255, 40 steps with step size e/10. For &g PGD we set = 1, 40 steps with step size e/10.
MIM: The Momentum Iterative Method (MI-FGSM) Dong et al. (2018) introduces a momentum
term stabilize the gradient direction. We use = 4/255, 40 steps, and momentum decay 1.0. SGM: The Skip Gradient Method (SGM) Wu et al. (2020) applies gradient smoothing to mitigate gradient shattering in residual connections. We set 4/255, 40 steps, momentum decay = 1.0,
and X = 0.01.
VMI: The Variance-Minimizze Iterative Method (VMI-FGSM) Wang He (021) reduces gradi- ent variance across steps. We set 4/255, 40 steps, momentum decay 1.0, and neighborhood samples N 5.
AA: AutoAttack (AA) Croce & Hein (0220) is ensemble of parameter-free attacks designed for reliable robustness evaluation. We run the standard configuration with c = 4/255, 40 steps.
FAB: The Fast Adaptive Boundary (FAB) Attack Croce & Hein 2020a minimizes perturbation
12

---

Published as a conference paper at ICLR 2026
norm while crossing the decision boundary. We set 4/255 and allow up to 40 steps.
Square: The Square Attack Andriushchenko et al. (2020) is a gradient-free black-bor attack using random square- quare--haped updates. We set e 4/255 and a query budget of 5000.
BPDA EOT: Following Athalye et al. 2018), we evaluate adaptive white-box attacks using Back- ward Pass Differentiable Approximation (BPDA) combined with Expectation Transformation (EOT), BPDA approximates the backward pass of non-differentiable components with the identity mapping, while EOT averages gradients across multiple stochastic forward passes. We use 40 PGD steps, = 4/255, and EOT samples per step. This setting provides the attacker full access to the defense while accounting for stochasticity.
A.1.3 IMPLEMENTATION DETAILS OF ADVERSARIAL DEFENSES
JPEG Compression. Following Dziugaite et al. (2016), we evaluate JPEG-based defenses with compression quality factors of 75% and 50%.
BaRT. For BaRT Raff et al. (2019), we apply k = 5 and k = 10 randomized transformations sequentially at inference time. DiffPure. We use DiffPure Nie et al. 2022) with the authors' recommended hyperparameters for ImageNet-scale evaluation.
Adversarial Training (AT). Standard adversarial training (AT) Madry et al. (2018) is included as baseline for robustness.
ANF. For ANF[Surreh et al. (2025), we configure the first layer with a convolutional kernel size of 15 x 15, 256 filters, and a max pooling kernel size of 5 x 5. FFR. We evaluate Filter Frequency Regularization
et al. (2023) as frequency-based regularization defense. Since ANF and FFR alone do not yield sufficient robustness on ImageNet, follow prior work and combine them with adversarial training using 1-step PGD attack at c c= 4/255.
A.2 MORE EXPERIMENTAL RESULTS
Cross-Filter Transferability. To assess the transferability of adversarial examples across differ- ent trained filters, we employ Projected Gradient Descent (PGD) with e 4/255 and 40 attack steps. Table 7 shows the robust accuracy (RA) when adversarial samples generated using one fil- ter (source/attacker) are evaluated on another filter (target/victim). The results demonstrate that robust accuracy remains high across all off- off-diagonal entries, indicating that adversarial perturba- tions crafted on one filter do not easily transfer to others. This strongly suggests that our Jacobian- and logit-space separation objectives effectively induce gradient divergence among filters, thereby reducing the consensus subspace exploited by transferable attacks. other words, although each filter alone is capable of mitigating adversarial perturbations, the ensemble diversity ensures that perturbations optimized against one filter fail to generalize to others. This property is critical for our defense strategy, it directly targets the root cause of adversarial transferability: gradient align- ment. The consistently high RA values across ViT-16, ResNet-v2, Dei'T-S, and Inception-v3 models confirm the effectiveness of our design breaking gradient consensus across diverse architectures. Additional analysis and ablation studies are provided in Appendix A.3
Table Cross-filter transferability reported as robust accuracy (RA %) under PGD (E 4/255, 40 steps). Rows: source/attacker filter; Columns: target/victim filter.
ViT-16
ResNel-12
DeiT.S
Inception-v3
Sre//Tgt
f2
f1
f1
76.24
77.30
73.00
60.32
58.73
61.38
61.90
61.38
56.05
58.83
47.17
43.67
73.60
70.95
71.00
57.67
59.26
59.79
58.20
66.67
44.22
64.17
62.67
42.50
72.01
59.79
57.67
56.56
55.03
62.83
14
73.07
76.24
74.66
61.38
62.43
65.61
53.92
43.92
67.20
43.67
46.50
60.17
A.3
MORE ANALYSIS
A.3.1 IMPACT OF NOISE BUDGET
We assess non-adaptive attacks (the attacker is not aware of the defense) using PGD under both fxc and lg norms. As expected, increasing the perturbation budget degrades accuracy monotonically,
13

---

Published as a conference paper at ICLR 2026
but the drop remains gradual across typical evaluation ranges (See Table These results should be interpreted as an upper bound on robustness; see our adaptive evaluations for a stricter assessment.
Table 8: Non-adaptive PGD on ViT-B/16 under to and norms.
to
RA (%)
RA (%)
2/255
76.13
0.5
76.71
4/255
74.66
1.0
74.07
8/255
68.25
1.5
73.01
16/255
62.96
2.0
70.89
20 20/255
58.62
3.0
69.30
Non-adaptive results provide useful signal about baseline robustness but can overestimate true se- curity: adaptive, cefense-aware attacks (e.g., BPDA+EOT) are reported elsewhere in this paper for completeness.
A.3.2 IMPACT OF FOT
To quantify the effect of expectation over transformation (EOT) on stochastic defense, we run PGD-& with K E (5, 10, 20} EOT samples per gradient step (common randomness across paired evaluations; all other hyperparameters fixed). As K increases, the attack better estimates the gradient of the expected loss and therefore becomes stronger. Table 9 reports Top-1 robust accuracy (%)
across
(i) Robust accuracy decreases monotonically with for every e, confirming that EOT makes attack more faithful to the defense's stochasticity.
(ii) The marginal gain from K =0 to K0 is smaller than from K=5 to K=10 at low budgets (e.g., 1.06 points <= ==//55))) indicating partial saturation; however, larger budgets the 10 - 20 gain remains non-negligible (e.g.. -6.34 at e=/255). (iii) Overall drops from EOT-5 to FOT-20 grow with e (from -5.82 1/255 about -19 points 4/255-8/255), showing that higher budgets benefit more from accurate gradient estimation.
A.4 IMPACT OF FILTER ARCHITECTURE
The architectural design of the filters /s crucially influences both their ability to diversify gradients and their efficiency in deployment.
Expressivity vs. Simplicity. Filters must be lightweight to avoid prohibitive overhead during in- ference. At the same time, they need sufficient expressivity to induce distinet gradient directions. For example, a simple residual block with two convolutions and a ReLU nonlincarity
f(x) x Conv Cony
already introduces nontrivial nonlinear transformations, ensuring gradients are not trivially aligned. Deeper or wider filters could increase diversity further, but at the risk of overfitting collapsing to similar functions without additional regularization.
Table 9: PGD-EOT ViT-B/16 (ImageNet). Increasing EOT samples strengthens the attack, lowering robust accuracy. A columns show absolute point drops.
EOT-5
EOT-10
EOT-20
A 55+100 10
A 10 +20
A 5-+20
1/255
81.96
77.20
76.14
-4.76
-1.06
-5.82
2/255
74.55
64.50
60.79
-10.05
-3.71
-13.76
4/255
57.62
44.39
38.57
-13.23
-5.82
-19.05
8/255
37.51
24.81
18.47
-12.70
-6.34
-19.04
14

---

Published as a conference paper at ICLR 2026
Gradient Geometry. Architectures with strong local sensitivity (e.g.. small convolutional kernels) tend to decorrelate gradients across filters more effectively. Conversely, architectures with large receptive fields strong averaging effects may inadvertently align filters, weakening separation. Thus, the choice of kernel size, depth, and activation plays a direct role in the effectiveness Lus and LLV in promoting divergence.
Training Stability. Overly complex filters can destabilize joint optimization: with many pa- rameters, filters may collapse toward learning identity-like transformations, reducing their utiliiy. Lightweight architectures constrain the search space, making separation losses more effective in driving gradient divergence.
Computation and Deployment. Inference cost scales linearly with the complexity of each filter. Since DRIFT samples only one filter per forward pass, lightweight architectures preserve the real- time feasibility of the defense. In contrast, heavier filters would diminish the main advantage of DRIFT-efficient deployment without modifying the base model.
Practical Guidance. We find that residual-style shallow convolutional filters strike the best bal- ance: they preserve input dimensionality (ensuring model compatibility), introduce sufficient non- linearity for gradient diversification, and remain efficient both training and inference. More elab- orate architectures (e.g., multi-layer CNNs or attention blocks) may be explored, but lightweight residual filters already achieve strong robustness without sacrificing clean accuracy.
To evaluate how the choice of filter architecture influences DRIFT, we compare several lightweight designs while keeping the base model (ViT-B/16) and training setup fixed. Each filter preserves input dimensionality to maintain compatibility with the backbone.
Architectures tested.
SingleConv: A single 3 convolution with ReLU,
ResBlock (ours): Two 3 x 3 convolutions with ReLU and a residual skip connection.
DeepConv: A stack of four 3 x 3 convolutions with ReLU activations.
MLPFilter: A shallow two-layer MLP applied patchwise to image embeddings.
Table 10: Effect of filter architecture on clean and robust accuracy (%) with ViT-B/16 under adaptive PGD (c 4/255, 40 steps).
Filter Architecture Clean Accuracy
Robust Accuracy
SingleConv
77.3
42.5
DeepConv
75.8
47.9
MLPFilter
74.9
44.2
ResBlock (ours)
80.5
56.7
Results show that the residual block (our chosen design) achieves the best balance: it maintains clean accuracy close to the undefended baseline while significantly improving robust accuracy. Sin- gleCony lacks sufficient expressivity and yields poor robustness. DeepConv provides moderate robustness gains but at the cost of lower clean accuracy and higher training instability. Patchwise ML.s decorrelate gradients somewhat but underperform compared to convolutional filters. Overall, lightweight residual filters offer the most effective and efficient choice for DRIFT.
A.5 IMPACT OF THE NUMBER OF FILTERS
The number of filters K plays critical role in shaping the defense's effectiveness.
Gradient Diversity. Each filter induces a distinct gradient geometry. Increasing expands the ensemble of gradient subspaces, which makes it harder for an adversary to find perturbations that transfer across all filters. In theory. the pairwise consensus I'fa. ff) is kept small, the expected transfer success of perturbations decreases roughly O(1 K), since adversaries must overfit one
15

---

Published as a conference paper at ICLR 2026
filter time. Thus, larger K strengthens robustness by increasing gradient divergence opportuni- ties.
Robustness-Accuras Trade-off. While more filters improve adversarial robustness, they may also introduce redundancy or training instability if K is too large. Empirically, small ensembles (K' 2-4) already capture strong divergence effects without hurting clean accuracy. Beyond certain point, gains diminish; additional filters may converge to similar behaviors unless carefully regularized by LJS and JP.
Training and Inference Cost.
The cost of training grows quadratically with K for pairwise losses:
K
Cost
) (Po+ Pw).
Thus, increasing K both increases backprop passes (for each probe) and the number of pairwise comparisons. Inference, however, remains efficient: only one filter is sampled and applied per forward pass, runtime cost grows only linearly with K when switching filters randomly. This makes DRIFT scalable in deployment even with moderate K.
Practical Guidance. We find K = 3-5 filters offers the best balance: sufficient gradient diver- sity to suppress transferabilityy while keeping training cost manageable. Larger ensembles can be explored if resources allow, but diminishing returns beyond K 6 suggest focusing instead on improving separation losses.
A.6 PROBE COUNT: VARIANCE-COMEN TRADE-OFF
Estimators. (fr,f)) let
DRIFT uses Hutchinson-style probing estimate pairwise alignment. For two filters
61; fu, f)) = cos? (x) (x)
du (w fz) = cos² (V w) vd((ff())),w)
with unit probe in the filter output space, and w unit probe in the logit space. With P. iid. probes (p and Par i.i.d. probes ) the Monte Carlo estimators used in LJs and LIVer are
P. Pup (z: fi.
1 P Se, (z: fi, fz).
1
(is(z;fuff)
Σ
Po
P g=1
Concentration. Each summand is bounded: &o E [0,1] and the [0, 1]. Hence, by Hoeffding's inequality, for any > 0,
Pr [ - Elo.11 2c) I 2exp( -2,,
2 - 2 e 2P
Consequently, the mean-squared eπrо (MSE) scales as O(1/P,) and O(1/Pw), respectively. In practice, this means doubling the number of probes reduces the estimator's standard deviation by roughly 1/v2.
Bias. The estimators are unbiased for the probe-averaged consensus, i.c., J[ Elo.] and E|fivP]] = B[ where the expectation is over the probe distributions (Rademacher or Gaussian, normalized). Any residual bias relative to full Iacobian/gradien alignment comes from using ran- dom projections instead of exhaustively scanning all directions; this bias decreases as probe count grows.
Compute cost. Each probe requires a VJP/gradient evaluation. Thus the per-batch cost scales linearly in probes:
Cost per batch R (P. (K) + Pw. (k) + Pw. K).
corresponding to JS pairs, LVJP pairs, and vi-dent-y terms. Larger K or probe counts im- prove statistical stability but increase backprop passes.
16

---

Published as a conference paper at ICLR 2026
Practical guidance. We find simple schedule balances stability and throughput:
1. Warmup (epochs 1-w): P. and =2 (fast, lets filters stabilize).
2. Main training: P.=5 and = (good variance-compute trade-off).
3. High-fidelity refinement (last 10-20% epochs, optional); -,-8-10, P., -8-10 if com- pute permits.
Empirically, Pe: Pw E [5, 10] yield stable separation signals and stronger cross-filter non- transferability, while larger values show diminishing returns relative to their linear compute cost.
A.7 PROOF OF LEMMA 3.4 AND THEOREM 3.5
We provide full derivations for Lemma 3.4 and Theorem 3.5. Recall that for filter f, we define ()()) & ((M()(2)),
(,(a) (z))(x).
Assumptions. We assume the following standard regularity conditions for all filters fj and inputs
A1 (Gradient boundedness). There exists
such that [19) S
A2 (L-smoothness), The loss L-smooth in æ, i.e.,
[V) V=(s('')ll S L 2/112.
Equivalently, for any 8,
L (x (()+)))))())) 1012
A3 (Margin-loss link). For correctly classified x, misclassification requires a loss increase of at least margin n(x)) > O. For inputs under consideration, assume <(z) 2 Kmin
0.
Gradient consensus. For filters f. and fj. define the gradient consensus at x as
(o.(z) P(fi, (x) (1o, E 1-1,,1]
We consider perturbations aligned with filter fe
(x(x) (x(x) = (=)
[]((()]1 s.
A.7.1 PROOF LEMMA 3.4
Lemma 3.4. Under AI-A2, for any # and any (0, Emax))
(z(x 5x(z)) (x(x) s C=GT(fi
for some constant C = 1 + LEmax depending only on G, L, and e mmax. 2G
Proof. Using L-smoothness (A2),
(,(x 5,) (((()) (1,()), 8,) 11..12
Linear term. Since 8, E e 11:1121
(g(x),,((z)) = (z) gx(x)) al
(x)]2
17

---

Published as a conference paper at ICLR 2026
Applying (19, (2)())]22 < G (AI),
(g)(x))(((2) S EGT fj:)).
(2)
Remainder term. Because | 8² S Emaxe,
L
L
- 11.112
E EmaxE.
(3)
2
2
Substituting (2) and (3) into (1),
t,(x + 6.) - -z(x)) < (Gi +
2 Eaax..
Define
Le C=
2G
giving
- C.a
A.7.2 FROM LOSS INCREASE TO TRANSFERABILITY
Define the misclassification indicator
mis (z, 6,) = (M + +.)) # #y).
From A3, misclassification requires
(
- -1)-)))((())) 2 k(x).
Using Lemma 3.4,
mis (1, 0.) ( 2 x(x))-
Assuming x(x) 2 Kmins
(x 8,) & I fzi)] fzi)) 2
Kmin
CsG
Taking expectation over
p,(2, 8,) A E, (1 8.)1 (I.
(2
where C' absorbs Kmin
A.7.3 PROOF OF THEOREM 3.5
Theorem 3.5. Let (, 8.) be the transfer probability for filter pair (i. Then under Al-A3,
Bizi(1,(x, 6.)] S C&G ; [[T(] r);=)]]=
Thus, reducing the average cradientssonsensuss directly reduces expected transferabilityy
Proof. For each fixed (i,j), the bound above gives
(r 5z) s C&G (IT
Taking expectation over the uniform draw of pairs + j and using linearity, (x 6,)] < C&G Big 1. (ft. fxi))]
(2)
C&G E, E, و [[] fz:))]
(3)
Since C' is independent of (i,j) and x, this yields
E (p, (2, 6,) < CeG (Tfe,
This completes the proofs of Lemma 3.4 and Theorem 3.5.
18

---

Published as a conference paper at ICLR 2026
Filter Convl Kernel
Convl Spectrum
Conv2 Kerne
Conv2 Spectrum
0.02
0.0400
0.02
0.05
0.0375
0.04
0.0350
0.01
0.0325
0.00
0.03
Filter Convl Kernel
Spectrum
Conv2
Conv2 Spectrum
0.01
0.02
0.05
0.00
0.01
0.04
0..11
0.03
0.02
0.01
0.02
Filter Convl Kereel
Conv1 Spectrum
Conv2 Kernl
Conv2 Spectrum
0.03
0.09
0.02
0.07
0.00
0.04
0.06
-0.02
-0.01
0.02
0.05
-0.02
-0.04
0.04
Filter
Convl Kernel
Convl Spectrum
Conv2 Kernl
Conv2 Spectrum
0.060
0.02
0.08
0.03
0.00
0.02
0.050
0.01
0.045
-0.02
0.06
0.035
-0.01
Figure 3: Learned DRIFT filter kernels (spatial and frequency views). Each corresponds one filter in the ensemble. Left: spatial kernels from the Conv1 and Conv2 layers. Right: their corre- sponding Fourier magnitude spectra. The filters exhibit complementary spatial-frequency behaviors (e.g.,
smoothing, edge-emphasizing high- high-frequency responses, and anisotropic patterns), confirming that the ensemble learns diverse transformation regimes rather than collapsing to similar operators.
A.8 FILTERS INTERPRETABILITY
Figure visualizes the learned DRIFT filters. Each FilterB contains two convolutional layers, whose spatial kernels and frequency spectra exhibit distinct structural patterns. The first-layer kernlss (Conv 1) emphasize low- and mid-frequency components, while the second-layer kernels (Conv2) introduce sharper, high-frequency structures. This diversity confirms that different filters special- ize complementary spatial frequuecy transformations, thereby decorrelating gradient directions across the ensemble.
Figure 4 iillustrates the visual transformations produced by each the four learned DRIFT filters various ImageNet samples. Despite preserving the overall semantic content, each filter intro-
19

---

Published as a conference paper at ICLR 2026
Fitor
Fiber
Riai
Filur
Input
Fiter
Fiter
Riber
Filer 4
Figure 4: Qualitative transtormations induced by DRIFT filters. Each column shows the output of a different filter applied to diverse ImageNet samples. Despite preserving semantic content, the filters produce noticeably different low- and high-frequency alterations, including edge enhance- ment, color-tone tone shifting, texture smoothing, and contrast modulation. These complementary trans- formations contribute to DRIFT's ability decorrelate gradients across filters while maintaining perceptual fidelity.
duces distinct textural and chromatic puriations-e.g.. Filter enhances local contrast and sharpens edges, producing stronger high-frequency details. Filter 2 modulates mid-frequency color and tex- ture components, introducing characteristic color-tinting and texture amplification. Filter applies mild anisotropic smoothing with subtle directional bias, reducing fine details in non-uniform way. Filter performs global brightness and tone lifting while preserving or slightly enhancing local structural details. This diversity of responses confirms that the filters operate in complemen- tary transformation regimes, generating decorrelated input gradients while maintaining perceptual fidelity- essential factor behind DRIFT's robust, non-obfuscated defense behavior.
20


Reviews:
[{"review_id": "SrNZfYeyai", "reviewer": "Reviewer_BXyw", "summary": "In this paper, the authors propose a defense method based on learnable filters. To mitigate the issue of gradient consistency, which makes models vulnerable to transfer attacks, the method reduces the transferability of adversarial examples across different filters while enhancing filter diversity for defense. Additionally, lightweight residual convolutional blocks are introduced within each filter to ensure that they can learn meaningful transformations. Experiments conducted on multiple CNN and ViT architectures using the ImageNet benchmark demonstrate that the proposed learnable filter ensemble method achieves strong defensive performance compared with baseline approaches.", "questions": "1. Could the authors provide more details on how the inequalities presented in Lemma 3.4 and Theorem 3.5 are derived? \n\n2. Could the authors offer additional theoretical justification to explain why the proposed Jacobian loss and logit separation loss outperform other adversarial training methods?\n\n3. Could the authors provide more details on how each loss component contributes to the overall filter training process and how the optimal values for the associated hyperparameters are determined?\n\n4. The authors claim that the proposed method can function as a plug-and-play module. Could the authors provide comparative experiments demonstrating how this defense method integrates with and performs across different model architectures?\n\n5. Could the authors provide more details on how different perturbation sizes affect the performance of the proposed method on CNN and ViT models?\n\n6. Could the authors provide additional experimental results on runtime efficiency, including comparisons with more baseline methods?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 1, "strengths": "1. The authors propose using a Jacobian separation loss to reduce transferability across filters, along with a logit-space separation loss to enhance filter diversity.\n\n2. To improve filter performance and enable meaningful feature transformations, the authors design residual modules for each filter.\n\n3. Experimental results on CNN and ViT models demonstrate that the proposed learnable random filter ensemble defense outperforms traditional defense methods.", "weaknesses": "1. Theoretical details are insufficient, as the derivations of the inequalities in Lemma 3.4 and Theorem 3.5 are not provided.\n\n2. The total loss comprises four components. However, it remains unclear how the contributions of each loss are balanced during filter training and how the optimal values of the associated hyperparameters are determined.\n\n3. The overall training procedure of the DRIFT filters is not clearly explained. For instance, Algorithm 1 includes a warm-up stage, separates the optimization of individual losses, and maximizes $L_{adv}$ over the index $i$. However, the paper lacks sufficient details regarding the training process and implementation.\n\n4. The experimental comparison is not comprehensive. The paper lacks evaluations against more recent defense methods and does not consider evaluations against the latest attack techniques. Furthermore, some mentioned methods (e.g., ANF, FFR) are not compared in Table 1, which limits the completeness of the comparative analysis.\n\n5. In Section 6.2, the choice of perturbation size (4/255) is not explained, and no experiments are provided to compare different perturbation magnitudes. \n\n6. The paper lacks ablation studies on the hyperparameters of the proposed loss function to analyze their impact on the experimental results.\n\n7. In Section 6.7, Table 6 presents efficiency comparisons only with the DiffPure method and lacks comparisons with other baseline methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "okpXSJlkjD", "reviewer": "Reviewer_w2HJ", "summary": "This paper proposes a new stochastic adversarial defense method called DRIFT. It identifies gradient consensus, i.e., the alignment of gradients across stochastic transformations, as the root cause of adversarial transferability. DRIFT combats this by learning a small ensemble of lightweight, differentiable preprocessing filters trained to maximize gradient divergence while maintaining clean accuracy. Theoretical analysis links low gradient consensus to reduced transferability. The training loss combines cross-entropy, Jacobian separation, logit-VJP separation, and adversarial robustness terms. Experiments on ImageNet with CNNs and Vision Transformers show substantial robustness gains over adversarial training, diffusion-based purification, and randomized transformations, under both adaptive (BPDA, EOT) and non-adaptive attacks, with negligible computational overhead.", "questions": "1. How does robustness scale with the number of filters? Is there diminishing return or optimal ensemble size beyond n = 4?\n2. Could DRIFT generalize to non-image domains (e.g., NLP, tabular, or time-series data)?\n3. How sensitive are results to the loss weights (β_JS, β_LVJP, λ)?\n4. If the base model is already adversarially trained, does DRIFT still provide gains, or is its effect redundant?\n5. Can the authors show qualitative examples of learned filters to interpret what kind of transformations disrupt gradient consensus?\n6. Could DRIFT be combined with randomized smoothing to yield both empirical and certified robustness?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. This paper reframes adversarial robustness around gradient divergence rather than obfuscation or purification. This is a fresh and intellectually elegant perspective.\n2. This paper provides a simple but formal link between gradient alignment and attack transferability, grounded in the geometry of loss gradients.\n3. This method applies across CNNs and Vision Transformers without modifying the backbone.\n4. This paper is evaluated under strong adaptive attacks (BPDA + EOT, AutoAttack) and validated with sanity checks (finite-difference tests, loss landscape visualization).\n5. This paper achieves 10⁴× faster inference than diffusion-based defenses, with minimal memory overhead.\n6. Presentation and structure of this paper are clear.", "weaknesses": "1. The analytical part (Lemma 3.4–Theorem 3.5) is intuitive but lacks full proofs or empirical validation of constants; thus, it serves more as motivation than as a formal guarantee.\n2. No certified guarantees (unlike randomized smoothing), so the results, though strong, remain empirical.\n3. Filters are trained on a subset of ImageNet validation data—unclear if robustness generalizes across distribution shifts or unseen domains.\n4. The method assumes a high-quality pretrained base; how it performs on weaker or adversarially trained backbones is unclear.\n5. While the method ensures gradient divergence, it does not analyze how filters achieve this (e.g., frequency-domain or spatial patterns).\n6. Effects of ensemble size and loss weighting are reported briefly in appendices but could be expanded in the main text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SvoaQuRkW0", "reviewer": "Reviewer_Fyh6", "summary": "This paper introduces DRIFT (Divergent Response in Filtered Transformations), a new adversarial defense that breaks the tendency of randomized defenses to share aligned gradients exploitable by adaptive attacks. DRIFT inserts a small ensemble of learnable, differentiable filters before a frozen classifier and trains them to maximize gradient divergence in Jacobian and logit space while maintaining clean accuracy. The authors theoretically link gradient consensus to adversarial transferability and design a joint objective combining prediction consistency, Jacobian separation, logit-VJP separation, and adversarial robustness losses. Experiments on ImageNet with CNNs and Vision Transformers show that DRIFT achieves higher robustness under PGD, AutoAttack, BPDA, and EOT, outperforming baselines like JPEG, BaRT, DiffPure, and adversarial training. Sanity checks confirm no gradient masking, and runtime analysis shows orders-of-magnitude efficiency gains. Overall, DRIFT presents a lightweight, theoretically motivated, and empirically strong defense principle based on gradient divergence rather than masking or purification.", "questions": "The paper claims robustness under BPDA and EOT, but the attack strength (number of EOT samples, surrogate choice) is not detailed. Please include robustness curves under stronger settings and alternative BPDA surrogates.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.The paper introduces gradient consensus as a measurable cause of adversarial transferability and proposes gradient divergence as a defense principle. This framing is intuitive, theoretically motivated, and distinct from existing randomization or purification methods.\n\n2.DRIFT’s learnable residual filters provide a lightweight, differentiable, and architecture-agnostic front-end, requiring no retraining of the backbone. The method is computationally efficient (0.4 ms per image), a clear advantage over diffusion-based defenses.\n\n3.The experiments cover both CNNs and Vision Transformers on ImageNet, testing against diverse adaptive attacks. Results demonstrate consistently higher robustness while maintaining clean accuracy.\n\n4.The inclusion of gradient-norm sanity checks, finite-difference validation, and loss-surface visualization shows the authors are aware of gradient masking pitfalls and have rigorously verified true robustness.", "weaknesses": "1.The link between consensus and transferability (Theorem 3.5) is intuitive but lacks formal rigor; constants are unspecified and empirical validation of consensus metrics (ρ) is missing.\n\n2.Ablation results show improvements, but the contribution of each loss component (LJS vs LLVJP) is not deeply analyzed; no visualization or metric demonstrates the intended gradient decorrelation effect.\n\n3.Comparisons to adversarial training and diffusion-based methods appear under-tuned, and evaluation is limited to ImageNet only, lacking smaller, more reproducible datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)