AlignFix: Fixing Adversarial Perturbations by Agreement Checking for Adversarial Robustness against Black-box Attacks
Authors: Ashutosh Kumar Nirala
Jin Tian
Olukorede Fakorede
Modeste Atsague
Venue: Accepted by TMLR ()

Abstract: Motivated by the vulnerability of feed-forward visual pathways to adversarial-like inputs and the overall robustness of biological perception, commonly attributed to top-down feedback processes, we propose a new defense method AlignFix. We exploit the fact that natural and adversarially trained models rely on distinct feature sets for classification. Notably, naturally trained models, referred to as \textit{weakM}, retain commendable accuracy against adversarial examples generated using adversarially trained models referred to as \textit{strongM}, and vice-versa. Further these two models tend to agree more on their prediction if input is nudged towards correct class prediction. Leveraging this, AlignFix initially perturbs the input toward the class predicted by a naturally trained model, using a joint loss from both \textit{weakM} and \textit{strongM}. If this retains or leads to agreement, the prediction is accepted, otherwise the original \textit{strongM} output is used. This mechanism is highly effective against leading SQA (Score-based Query Attacks) as well as decision-based and transfer-based black-box attacks. We demonstrate its effectiveness through comprehensive experiments across various datasets (CIFAR and ImageNet) and architectures (ResNet and ViT).

race between Published in Transactions on Machine Learning Research (08/2025)
AlignFix: Fixing Adversarial Perturbations by Agreement Checking for Adversarial Robustness against Black-box At- tacks
Ashutosh Kumar Nirala
oknirala Giastate edu
Department of Computer Science Jowa State Universiit
Jin Tian
fin.tion tion Carioaaim ac.ae
Professor of Machine Learning Mohamed bin Zuyed University of Artificial Intelligence
Olukorede Fakorede
fakorede Giastate. edu
Department Computer Science Towa State University
Modeste Atsague
woastate@aatate..coo edu
Department of Computer Science Iowa State Universily
Reviewed on OpenReview: https: /ttps://opeeee openrevee...ee.... net/ forum? id= Xg fssm
Abstract
Motivated by the vulnerability of leed-forward visual pathways to adversarial like inputs and the overall robustness of biological perception, commonly attributed top-down feedback processes, propose new defense method AlignFix.. We exploit the fact that natural and adversarially trained models rely on distinct feature sets for classification. Notably, naturally trained models, referred to as weakM, retain commendable accuracy against adversarial examples generated using adversarially trained models referred to as strongM, and vice- versa. Further these two models tend to agree more on their prediction input is nudged towards correct class prediction. Leveraging this, AlignFix initially perturbs the input toward the class predicted by a naturally trained model, using a joint loss from both weakM and strongM. If this retains or leads to agreement, the prediction is accepted, otherwise the original strongM output is used. This mechanism is highly effective against. leading SQA (Score-based Query Attacks) as well as decision brased and transfer-based black- box attacks. We demonstrate its offectivencss through comprehensive experiments across various datascts (CIFAR and ImageNett) and architectures (ResNet and ViT).
1 Introduction
Since the advent of adversarial attacks Szegedy et al. 2001 the field has
adversarial defenses and attacks. Defenses based adversarial training Madry et al. (018)) have withstood the test of time. However, robust accuracy still needs improvement for reliable deployment. In realistic scenarios, allackers lack complete access lo models, making black-box defense a practical choice and must be prioritized. However often these defenses only defend against Score based Query attacks (SQA) (like AAAChen et al. (2022), RND Qin al.] (2021)), while leaving transfer and decision-baset attack surface open. Our work focuses on these practical challenges, proposing defense that works well against all these black-box attacks.

---

Published in Transactions on Machine Learning Research (08/2025)
We argue that simple feed-forward networks struggle with adversarial robustness due to the absence of corrective feedback mechanism, a key component of biological perception Hawkins & Sandra (2004). Notably, Elsayed et al. (2018) found that under rapid, time-limited conditions (+s 74 ms), where humans likely cannot engage these mechanisms, adversarial images also mislead human perception. Motivated by this, we propose incorporating a mechanism inspired by lop-down feedback in biological perception enhance the robustness of trained models.
Nat'
SAT
TRADES
MART
Nat'
00.00
71.27
72.50
75.97
SAT
82.74
51.61
62.22
64.19
TRADES
82.67
62.53
52.94
66.23
MART
78.42
59.01
61.03
54.87
Table 1: Transfer accuracy of adversarics generated by different models. (ResNet-18, CIFAR-10, PGD-100 attack). Columns show models used for crafting the attack.
WeakM prediction
WeakM
jointM nudge ilp
VX
+5
Nudge to WeakM prediction
Yes:
olp:
StrongM
Agree?
StrongM(X)
No
StrongM
o/p: StrongM(x)
Figure AlignFix Architecture: WeakM refers to naturally trained model, Strong refers to an adver- sarially trained model, and joinLM refers to when the models in parallel. Note that the input may be either clean adversarially perturbed: e.g., via transfer attack iterative black box attack using this (AlignFix) setup. AlignFix then computes Z by mudging x toward the WeakM prediction using jointM.
A. key challenge is to define suitable feedback signal. Prior work Zhang & Zhu 2019 showed that ad- versarially trained models rely shape, while naturally trained models rely on texture, indicating they atlend to distinct and often ort hogonal features. This difference makes it difficult for adversarial examples to transfer between them. Table [ shows that adversarial examples crafted on a robust model (strong. are more likely to be correctly classified by a naturally trained model (weakM), and the reverse also holds. Based on this, we use the disagreement between these models as an implicit corrective signal. We adjust the input by madging it towards the naturally trained model's prediction using joint loss from both models. If
2

---

Published in Transactions on Machine Learning Research (08/2025)
this keeps restores agreement, accept the updated prediction, else fall back to the robust model's original output. Figure illustrates the architecture, and the full algorithm is described in Section 3.3
To our knowledge, this is the first work to leverage a naturally trained model to strengthen the robustness of an adversarially trained model. Our contributions are as follows:
We propose Ali, a simple method which incorporates corrective perturbation mechanism to impart robustness under realistic black-box threats.
We experimentally demonstrate that AlignFix improves robustness against rcalistic black box at- lacks, including: score-based, decision-based, transfer, and adaplive allacks.
Please note, while Fix is inspired by the top-down feedback and error correction principles observed in biological perception, it does not implement true feedback, and any direct comparison in mechanism would inappropriate.
2
Background and Related Work
2.1
Preliminaries
We consider K-class classifier f parameterized by 0, which maps input 2, X' to its class label ys. The model outputs logits f=(x;,0) 0) for each class c, and the predicted label is Upred; arg max fo(x;, 0)
We refer to naturally trained models as weakM and adversarially trained models as strong Their pre- dictions are denoted YweakM, and YstrongM, respectively. During defense, we perturb the input using both models jointly by summing their cross-cntropy losses; we denote this configuration as jointM.
In adversarial settings, the input x; is perturbed to at induce misclassification. Perturbations are con- strained to B. [z.]] x || zıllp < e, where Il-IIp is the lp norm and is the perturbation budget. The projection operator II clips x' to keep it within B.[x]).
We focus on the &o-mormm as it is the standard in adversarial robustness research, offering a well-established benchmark with imperceptible and efficiently computable perturbations, Following convention, we set e = 8/255 for CIFAR-10 and = 4/255 for ImageNet.
2.2 Adversarial attacks and defenses
This subsection summarizes adversarial attacks and defenses most relevant to our work. For broader survey, see Akhtar et al. (2021).
Adversarial attacks: Common white-box attacks include FGSM Goodfellow et al. (001)) PGD and targeted-PGD Kurakin et al. (2018); Madry et al. (2018), and their variants such as AutoPGD (APGD) Croce & Hein (2020b), FAB Croce & Hein (2020a), and C&W Carlini & Wagner (2017). These attacks use the gradient of the loss with respect to the input x; to iteratively estimate a perturbation direction that maximizes loss in local neighborhood:
II (af + aaa-sgg(ee sgn(( w)))
(1)
B. B.l
Variants differ in their choice of loss function, step size, and update rules. These are considered the strongest attacks, as the adversary has full access to the model parameters.
In real-world scenarios, attackers typically lack full model access. they can query class confidence scores, they use Score-basod Query Attacks (SQA) iteratively craft adversaries. The Square attack Andrüushchenko et al. (2020) leading example: it perturbs random square regions by #2c and retains changes that reduce confidence in the true class. Other SQA methods include Bandit Tyas et al. (2018b).
3

---

Published in Transactions on Machine Learning Research (08/2025)
SimBA Guo et al. (0199) ZOO Chen al. 2017 SignHunter Al-Dujaili & O' O'Reilly 20019 and NES
Iyas
et al.
2018a)
In decision-bassee based (hard-label) attacks, only the predicted class is available. Notable examples include SPSA
Uesato et al. (0018). HopSkip.Jmm Chen et al. (2020) RayS Chen & Gu (2200), and others Ma et al. (2021
Shukla al. (2021] : Cheng et al. (2018): Brendel et al. (2018).
In transfer attacks, adversaries craft inputs using white-box attacks on a surrogate model. Due to the
transferability of adversarial examples Szegedy et al. (2014), such attacks often succeed when the surrogate shares architecture or training data. When combined with decision-based attacks, they greatly reduce query counts Sitawarin et al. (2024).
A Adversarial defense: Adversarial Training (AT), which trains models on adversarial examples, remains
the most effective defense. Key methods include SAT
Madry al. (2018) and TRADES Zhang al. (2019).
with several extensions such as MART Wang et al.
2019), GAIRAT Zhang ct al. (020)) HE Pang ct al.
20200: Fakorede et al. 2023), MATT Liu et. al. 202 and AWP Wu et al. 2020 Yu et. al. (0022).
Most black-box defenses neglect decision-based and transfer attacks, leading to false sense of robustness. Such defenses can often be bypassed by training a surrogate model. The need to guard against transfer attacks is highlighted by Szegedy et al. (2014); Sitawarin et al. (2024).
2.3 Related work
AlignFix (ours) is an adaptive defense, modifying the input at inference to correct, potential adversarial perturbations. While several methods also perform input nudging, their goals differ. Wu et al. (2021) maximize the total cross-cntropy, which Croce et al.] (2022) show can hurt accuracy near decision boundarics.
Shi al. (221]) use self-supervised signals, whereas AlignFix leverages the disagreement between strongM
adversarially trained) and weakM (naturally trained), incorporating guidance from weakM. Tao et al.
) and Li et al. 2023) apply mudging during training and are not adaptive defenses.
RND Qin et al. (2021) adds Gaussian noise to resist SQA attacks. AAA Chen et al. (2022), closest to ours, adaptively alters logits to mislead SQA attacks but remains vulnerable to transfer attacks. AlignFix, by
contrast, increases the effective decision boundary of strongM, providing robustness beyond just SQA. We
treat AAA as our key baseline.
3 Methodology
3.1 Motivation from Biological Perception
Despite extensive research, neural networks still lack the robustness of biological perception. Adversarially trained models extract more human aligned features Zhang & Zhu (2019), but they remain vulnerable to real world transformations such as rotation and translation Engstrom et al. (2019). This raises a natural question:
Can feed-forward neural networks be adversarially robust, or are they intrinsically vulnerable?
Elsayed et al. (2018) showed that adversarial images can fool humans when presented briefly (71ms), but not when given more time (2s). This suggests that the human brain corrects perception using top-down and lateral feedback. Guo et al. (2022) compared neural activity in primates with representations in ResNet-50. Surprisingly, adversarially trained ResNet-50 was more stable to allacks than the primate visual system under black-box perturbations. Yet, humans (and likely primates) are not easily fooled, implying some form of error correction present in the brain.
3.2 Key Insight from Model Disagreement
While making a prediction, if an oracle could identify the correct class, we could nudge the input toward the correct class to undo adversarial perturbations, We propose using the disagreement between strongM (adversarially trained) and weakM (naturally trained) models proxy for such oracle.
4

---

Published in Transactions on Machine Learning Research (08/2025)
Algorithm AlignPix
Inputs: Fy; strongM, weakM, s size
Output: Prodiction logits for the input 2;
1: logitW <- (aak
2: Obtain ã; by nudging 2; toward arg max(logitW) using Equation with step size size for step
3: logitS t (2.)
arg max max(log)))) arg max(logitW) then return logitS
end if
return (I))
As shown in Table !1 adversarial examples transfer poorly between these models. Attacks most suc- cossful in the white-box setting. When attacked using adversarics generated from naturally trained models, adversarially trained models show high accuracy. Conversely, naturally trained models maintain the highest accuracy even when attacked using adversarially trained models. This is likely because the two models rely on distinct features texture for naturally trained models and shape for adversarially trained ones.
Zhang & Zhu (019 further showed that naturally trained models can still classify shuffled image tiles, while adversarially trained models fail. This was quantified in their Figure 7 (borrowed in Figure f in Appendix C where AT-CNN accuracy drops sharply under patch-shuffling (which destroys shape but preserves local tex- tures), whereas standard CNNs remain largely unaffected. Their saliency map analysis (Figure 2, borrowed in Figure 5 further shows that AT-CNNG focus on object contours, while standard CNNss attend to dispersed texture regions. In absence of adversarial training, the model does not have incentive to learn robust features Tsipras et al. (2018). These findings reinforce that adversarially trained models are shape-biased and robust. texture perturbations. We find that combining the two models yields stronger robustness.
We assume that when the models agree, the prediction is likely correct. Since both models trained for accuracy, simultaneous errors are rare. We experimentally validated this assumption for CIFAR-10 using the ResNet-18 model. We found that when an adversary example is both crafted and mudged using joint.M, then nudging towards the correct class increases the agreement between the two models' predictions from 34.98% to 49.20%, while they are nudged towards a random class, the agreement decreases to 16.99%.
3.3 AlignFix
We present AlignFix Algorithm D
First, we compute the logits logitW returned by weakM on the input 2; (Step 1). Then, we nudge the input one step toward the predicted class arg max (logitW) using Equation (2)
(Step 2). This step uses joint loss from both weakM and strongM:
2,
II (x, - s ssssssiee size sign (Vz ((1)
(2)
B. [x:]
where
Lee (strong (x:ronggM(x:....ee v/) + lce (weakM (z:), v.)
(3)
v arg k maxx(l(og)tWW))
(4)
Here, II denotes the projection operator that ensures ž; stays within the allowed perturbation budget e. Next, we compute logitS from strongM using the nudged input ā, (Step 3). the prediction arg max(logits) matches arg max(logitW)), we return logitS as output (Step 5). Otherwise, we discard the nudging and return the prediction from strongM the original input x; (Step 7).

---

Published in Transactions on Machine Learning Research (08/2025)
(a)
StrongM decision boundary
Py
max Py
weakM decision boundary under strongM attack
E-boundary
(b)
Py
max Py
(c)
Py
max Py
(c')
Py max Py bt
Figure 2: Mustration of the adaptive defense mechanism in AlignFix. Here, y denotes the correct class and y denotes the incorrect class with the highest logit value. The logit for the correct class, Pys is shown in green, while the maximum logit among incorrect classes, which may vary across steps, is represented by marp, Ay and is shown in red. Please note both the logits are logits from AlignFix, which always use strongM to output final logit. See text for further explanation.
3.3.1 Robustness through AlignFix
Figure 2 illustrates how AlignFix defends against adversarial perturbations by leveraging model disagree- ment. A clean input xy point (a) is correctly classified by both models. An attacker perturbs it toward misclassification, reaching point (b), but as long both models agree, AlignFix mudges the input toward the agreed class and returns the prediction. When the models begin to disagroe, two cases arisc: weakM will make the correct prediction (case A) or not (case B). In case A, shown at point (c), weakM remains correct while strongM fooled. AlignFix mudges the imput in the direction favored by weakM, using both models jointly. This leads to point (c'), where if the models now agree, AlignFix returns the corrected prediction from strongM. This effectively expands the decision boundary of the robust model, allowing recovery from misclassification. If not, it falls back to the original output of strongM. For case B, when weakM makes incorrect prediction, we rely robustness of strongM. Since it is hard to fool strongM, the mudging done in the incorrect direction would, often, not be enough for strongM to cause misclassification, especially under realistic black-box scenario.
This mechanism increases the effective decision boundary of strongM, making black-box attacks less effective. Figure 3 shows this effect on the Square attack. While the attack fools strongM and weakM individually, AlignFix Fix prevents failure by letting one model correct the other's error. This frustrates the attack loop and maintains accuracy. Similar robustness is observed against SPSA and RayS. AlignFix also performs well under transfer attacks due to the inclusion of strongM, which learns robust features.
3.4 Design Choices
Since strongM and weakM/ rely different features, option was to nudge the input only when their predictions disagree. This would expose only strongM to the attacker until needed. However, unless the input is corrected continuuusly, this approach reduces AliguFix's effectiveness on SQA attacks by nearly 10 percent. Therefore, we always apply nudging and discard only when it fails to restore agreement.
Logits are always returned from strongM, koeping weakM'ss confidence hidden. Because weakM is casier fool, relying on strongM when they disagree improves robustness. We found that using weakM for output drops Square attack accuracy from 83.5 percent to 71.8 percent on CIFAR-10.

---

Published in Transactions on Machine Learning Research (08/2025)
100
strongM Correct Count weakM Correct Count
54
50
500
1000
1500
2000
2500
Figure 3: AlignFix mitigates the Square attack by combining strongM and weakM. While the attack indi- vidually fools both models over 2500 iterations, their combination in AlignFix, prevents simultaneous error preventing faihure
To nudge the input, two parameters are required: (a) sleps, the number nudging steps, and (b) 8 size, the size of each step. The product n steps x 8 size determines the total mudging magnitude. We found that multiple small steps could be approximated by a single large step, so we fix n steps = 1. Since strongM is trained with a perturbation budget of 0.031 and weakM with c 0, the appropriate nudging lies roughly in between, around c/2. Based experiments Ablation Study), we fix size 0.02. As shown in Table g in Appendix, this value trades off transferability and black-box robustness. For ImageNet, where 4/255, use size 0.01, which yields consistent performance.
CIFAR-10 with ResNet-18 (Appendix Section B
4 Experiments
4.1 Setup
We evaluated AlignFix on CIFAR-10 and ImageNet datasets
! For fine-tuning and ablation, we used ResNet-
18. Following our baseline AAA-linear Chen et al. 20222 we reported results using WideResNet-288, for
CIFAR-10 and WedeResNet-50 for TmageNet. Appendix provides model sources and training details.
For Square attacks, we used the official implementation of AutoAttack outo-attache and set Pixit 0.05, consistent with AAA-linear. SPSA perturbation size 0.001, learning rate 0.01, and 128 samples per gradient estimate. For RayS, we used the official code (https://github cow commuccamm/RayySS with 1K and 10K queries.
http:///ithuu...oo/freeee for 100 iterations with
4.2
Results
In this section, present results for black-box attacks on CIFAR-10 and ImageNet.
4.2.1
SQA attacks
Table shows the results for black-box SQA attacks on WideResNet-28. models. Attack parameters are given in Appendix El We evaluated the first 1K samples from CIFAR-10. RND results taken from Chen al. (2022).
AlignFix-S.a using the SAT model Madry et al. (0188) as strongM, outperforms all baselines including AAA-Linear on the Square attack. It also performs better on NES, while SignHunter and Bandit slightly favor AAA-Linear. SimBA results are similar for both, despite higher natural accuracy of A-nnLin
"Code available in supplement and https://github... http://ghhhbb.ccm////ssraa//////

---

Published in Transactions on Machine Learning Research (08/2025)
Defense
Nat'
Accuracy on SQA
Attack (# queries = 100/2500)
Methodology
Acc'
Square
SignHunter
SimBA
NES
Bandit
Undelended
94.78
39.7/00.2
42.3/00.0
73.5/35.6
68.8/05.0
19.9/01.3
RND
91.05
60.8/49.1
61.0/47.8
76.4/64.3
86.2/68 66.2/68.2
70.4/41.6
AAA-Linear
94.84
83.4/79.8
84.2/83.0
86.4/84.5
84.6/71.0
86.7/82.8
SAT
85.83
76.9/60.5
74.9/56.6
84.1/80.4
83.3/75.4
78.7/66.2
AlignFix-SAT
90.30
85.7/81.3
80.5/79.4
86.0/83.6
87.3/73.2
85.0/81.9
TRADES
86.40
77.1/61.2
74.9/57.0
86.2/82.6
85.4/74.8
80.3/66.2
91.65
87.4/85.8
81.0/79.0
86.6/85.8
87.9/74.7
85.7/83.0
AWP
85.36
75.9/62.7
74.0/60.0
84.1/80.4
83.4/75.2
79.1/68.6
AWi
90.00
86.9/85.0
79.7/77.7
86.4/84.9
87.0/75.1
85.1/82.5
AWP E
88.25
81.3/67.8
79.5/63.1
87.2/81.4
86.9/79.9
83.4/72.5
Alig E
91.80
87.8/87.5
83.8/82.5
88.0/85.8
88.4/77.5
86.0/83.7
WANG23
92.44
86.5/75.5
85.0/71.6
92.1/89.1
91.5/84.8
87.8/79.7
AlignFix-WANN WANG223
94.40
91.4/90.9
87.2/85.6
91.0/89.5
92.1/81.6
$9.8/88.0
Table 2: AlignFix performance compared baselines on SQA attacks for the CIFAR-10 dataset with a perturbation budget of foc (queries 100/2500). WideResseet-28-L1 used for all models.
To test generality, we used other robust models as strongM: TRADES Zhang et al. (2019), AWP, AWP E Wu et al. (2020). and WANG23 Wang et al. (0023). With these, AlignE consistently achieves the highest accuracy across all attacks. For SimBA, we used the SimBA-DCT variant from https://gitan com/ cg563/simple-De waw_lle-baabbox--ttteteee degenerate case where onc-pixel changes fail to fool robust models.
scaling perturbations to the allowed maximum. This variant avoids the
Attack
Undefended
SAT
RND
A-A-Linear
Align Fix (Ours)
ACC(%)
78.48
68.46
75.32
78.48
72.35
Square
55.40/10.88 40.990
61.90/54.40
58.67/50.54
64.35/63.. 663.6
67.05/ 64.95
SignHunter
62.25/17.30
62.65/58.25
59.36/52.98
/71..25 71.75
67. 17.25/64.88
SimBA
70.65/57.35
66.40/644.00
66.36/63.27
70.80/66. 66..0
72.75 72.75/69.99 69.90
NES
76.15/59.35
67. 67.15/64..6
71.33/66.00 06.05
76.60/ /7.2
70.80 260/66.25
Bandit
62.60/27.65
64.70/59.45
65.15/61.38
69.70/69.0 69.20
69.10/67.95
Table 3: AlignFix performance compared baselines on SQA attacks for ImageNet dataset, with a pertur- 4
bation budget of: foo 255
(#query 100/2500). WideResNee-50 used for all models.
Table presents ImageNet results. AAA-Linear evaluated Square attack 1K randomly selected samples (one per class), correctly classified by the naturally trained model. They then scaled the accuracy by 78.48%. In this work, we evaluate and report results (including re-evaluation of our baseline AAA) on the first 2K samples from the validation set, sclected by sorting image filenames alphabetically. This yields consistent yet random- like sampling, as class labels are randomly distributed with respect to filename order. While this unfiltered setting slightly favors AAA, AlignFix outperforms AAA-Linear for Square and SimBA attacks.
Effect of Pinit in Square Attack: AAA-Linear used panil 0.05 for CIFAR-10 and 0.3 for ImageNet, matching the original setup for naturally trained models. Later works use 0.8 to better attack robust models. We found that this change severely reduces AAA-Lincar accuracy, while AlignFix remains stable (Figure
SurrogM
Natural
SAT
AAA
AliguFix
(ResNet-18)
(WideResNet-22
AT
73.17
64.85
73.17
69.33
Natural
16.91
85.03
16.91
79.47
jointM
15.36
78.92
15.36
72.95
Table 4: Transfer attack accuracies CIFAR-10, using ResNet-18 architecture surrogate Model. We note that, AAA-Linear accuracy drops considerably.
8

---

Published in Transactions on Machine Learning Research (08/2025)
Squaer attack accuracy (96)
50-
10
ECAC CIFAR-10
AMA CIFAR-10
D D_init
Figure 4: Square attack accuracy of AAA-Linear for different values of p _nint (fraction of pixels changed every iteration.) for CIFAR-10
Surrogate M
Natural
SAT
AAA
AlignFix
(WRN-50)
(WideResNet-50)
AT
67.05
40.65
67.05
48.95
Natural
00.01
67.15
00.01
61.90
jointM
00.00
59.10
00.00
55.70
Table 5: Transfer accuracy of adversaries generated for ImageNet. We used the same component models that are used in the defense.
4.2.2 Transfer and Decision-based Attacks
Transfer attacks Most black-box defenses overlook the threat posed by transfer attacks, even though a determined adversary casily train surrogate model and launch effective attacks without direct access to the target. For CIFAR-10, we used ResNet-18 model to generate PGD-20 adversaries and tested them WideResNet-28.. 10 models, L.e, the surrogate is much smaller network.
As shown in Table ! AlignFix achioves the highest worst-casc robustness, especially when adversarics are crafted using naturally trained surrogates where other defenses fail. For ImageNet, despite using the same models (due to high computation cost.) as AlignFix for crafting the attack, AlignFix retains significantly higher robustness compared AAA (Table 5).
Models
RayS (IK/10K queries)
SPSA
Undefended
22.30/00.10
00.00
AT
71.40/59.90
62.40
AAA-Linear
58.50/55.11 10
70.10
AlignFix (ours)
12.00/66.60
79.00
Table 6: AlignFix performance compared to basclines, using WidcResNNet28--10, on decision- based attacks for CIFAR-10, with the Eo perturbation of: 255
Decision-based attacks We evaluated AlignFix on SPSA and RayS attacks for CIFAR-10 (Table G While both AlignFix and AAA show good robustness against SPSA, RayS proves more effective against AAA due its strategy of starting from a nigh-perturbation misclassified point. In contrast, AlignFix's dynamic decision boundary offers consistently higher robustness in both cases.
4.2.3 Adaptive Attacks
AlignFix an adaptive defense, and as noted by Croce et al. (0022). is imperative to evaluate it using adaptive attacks. The principle of designing adaptive attacks to exploit the inner weakness of given defense. Since our method relies on the two component models, which use different of features, attacks that attempt. to remove both features are likely to bring its accuracy down. Under our setting, consider

---

Published in Transactions on Machine Learning Research (08/2025)
the case where the attacker knows the overall AliguFix architecture (Figure 1 but lacks access to the internal strongM and weakM models weights. In realistic black-box setting, the attacker can query the deployed AlignFix model and can also train surrogate models to approximate its behavior. For CIFAR-10, we used ResNet-18 models trained with standard and adversarial training as surrogates.
Algorithm Adaptive Attack for AlignFix Defense
Inputs:
(r,y): Input and label pair
surrSSrong surrogW eakM: Surrogate models
AlignFia: Deployed Fli model
s_size, e: Parameters for AlignFix defense and perturbation budget
pgd. ilrs, pgd size: PGD atlack iterations and step size
pgd.Atk (input, label, model, pert_bdgt,, pgd iirs, pgd 8 size, do do_t): t): PGD attack function; do f indi- cates if the attack is targeted
Output: x' such that AlignFiz(2") # y or FAILURE if not found
1: 24 5 pgddAkk(,, StrrStrongM,, size, pgd itrs s size, False)
2: Y/ 1-+-aa arg max
do t True
if Vt = y then
do + False
6: end if
7: x' tr x
8: jointM + Combine surrStrongM and surrogWeaks cakM
for ilr = to pgd ilrs do
10: x' pgdAAt((2' Vvs jointM, e, pgd s size, do t)
11: if arg arg (Aax(AligFFia((() # then
12: return x'
13: end if
14: end for
15: return FAILURE
The goal of an adaptive attack to exploit structural weaknesses in the defense. For AlignFix, two such cases arise: (a) both strongM and weakM agree on an incorrect prediction after mudging, or (b) the two models disagree, and strongM alone makes an error. Case (a) particularly damaging, as AlignFFx returns the incorrect shared prodiction. In case (b), AlignFix discards mudging and falls back on strongM, which, if wrong, also fails the defense. To larget both cases, we devised two-slage PGD-based strategy (Algorithm 2
Step 1: Find vulnerable class using strongM (Lines 1 1-3): We first run an untargeted PGD attack on the surrogate strong model surrStrongM with expanded budget of c c s_ size (Line 1). This yields an adversarial example Tz. We extract the predicted label yt from surrStoongM on (Line 2). If still matches the true label y (Line 4), we skip targeting and fall back to untargeted PGD later by setting do to False (Lines 5 6).
Step Launch a targeted PGD attack using jointM (Lines 7-14): We initialize x' <- x (Line 7) and define jointM by combining the surrogate strong and weak models (Line 8). Then, for each PGD iteration (Line 9), we update x' by running a targeted (or untargeted, depending on do t) PGD attack jointM, aiming for class Ye (Line 10).
Step 3: Query the actual AlignFix model (Lines 11-13): After each update, we check whether AlignlFix misclassifies x' (Line 11). If it does, we return x' as successful adversarial example (Line 12). If not, the loop continues.
Step Fallback (Line 15): If the attack fails after all iterations, we return FAILURE (Line 15). This probes both failure modes of AlignFix: (a) when both models agree incorrectly, and (b) when fallback to strongM fails.
10

---

Published in Transactions on Machine Learning Research (08/2025)
Surrogate Model Accuracy
AlignFix
strong.M weakM AlignFix 50.59
Accuracy
00.00
40.00
60.91
Table 7: AlignFix Accuracy on Adaptive Attack.
To construct effective adaptive attack, utilized surrogate models trained analogous datasets. For CIFAR-10, we trained ResNet-18 using standard and Madry's Madry et al. (2018) method models. The initial budget is e + s __size, where s_size is the nudging used by AlignFix as defense (0.02 for CIFAR-10). As shown in Table 7 AlignFix maintains high accuracy even under such informed attacks. Notably, even the surrogate AlignFix model, under white-box attack, retains 40% hannaacc-hhhhlighhiinn the robustness imparted by strongM.
surrogate
4.2.4 Results on ViT architecture
Models
Nat'
Square
RayS
SPSA
(ViT)
(100/2.5k)
(1k/10k)
Natural
91.8
42.5/00.1
18.7/00.3
06.6
SAT
76.4
65.3/52.7
59.5/51.2
64.8
AlignFix-SAT
80.1
75.8/74.4
61.8/58. 15.8...
76.7
TRADES
80.6
69.7/56.4
63.7/53.8
69.6
Aligulix Tx-ARDEES
84.8
79.9 79.9/77.9
65.7 65.7/61.5
80.3
Table 8: AlignFix CIFAR-10 accuracy with ViT architecture.
To evaluate AliguFix's generality across archilectures, trained Vision Transformer (ViT) models CIFAR-10 using the code from Mo et al. (2022). As shown in Table 8 AlignFix consistently improves the robustness of strongM for both SAT and TRADES, confirming its applicability beyond convolutional networks.
5 Limitations
AlignFix is designed for realistic scenarios black-box settings, including decision-based and transfer allacks. The AlignFix defense, by always outputting the strongM logit, theoretically constrains the total perturbation to the sum of the adversarial attack's perturbation (c) and the defense's corrective nudge (step_size). This may provide degree of white-box robustness, however comprehensive empirical evaluation against white- box attacks is outside the scope of this paper. Moreover, like other adversarial robustness methods (e.g.. AAA, AT), this work focuses specifically on adversarial robustness under too threats. Adversarial training on its own docs not nocessarily extend to all types of robustness, like robustness to other transformations such as rotation as shown by Engstrom et al. (2019). Handling them requires strongM and weakM to be trained for spatial robustness. This is an orthogonal challenge that can be incorporated into AlignFix by substituting spatially robust base models.
Increased Inference Time:
AliguFix incurs 5x overhead due to one backward and two forward passes per input. A simple ResNet-18 model takes s 1.6 seconds to classify the entire CIFAR-10 test set, with batch size of 500 using an RTX-2080 Ti graphics card, whereas AlignFix takes & 8.3 seconds. This is a tradeoff for strong robustness under black-bor box threats, including decision-baser and transfer attacks where prior defenses fail.
As discussed in Section 3.1 biological perception relies on corrective feedback Hawkins & Sandra (2004). Guo et al. (2022): ; Elsayed et al. (2018) attributed the role of feedback for robustness in biological perception. While adversarially robust food-forward models are idcal, current evidence suggests this may be 'fundamentally limited"'. Despite added cost, AlignFix remains practical e.g., classifying 10K CIFAR-10 samples took 8.3s RTX 2080 Ti. This real-time performance is
11

---

Published in Transactions on Machine Learning Research (08/2025)
acceptable in safety-critical settings like medical triage, surveillance, parts of autonomous driving like street sign detection, or offline robust curation.
Mitigation stralegies include gradient approximations, model distillation, or hardware acceleration.
Broader Impact Concerns: We acknowledge that, while AlignFix is developed as a defense, insights into model vulnerabilities can inform stronger attacks, raising dual-use risks. Moreover, the increased compu- tational cost due to multiple model evaluations may contribute to higher energy consumption, potentially limiting accessibility and raising environmental concerns. These Tradeoffs underscore the importance de- veloping energy efficient computational approaches, such as spiking neural networks Roy al. (2019), as well broadly applicable and interpretable defenses.
6 Conclusion and Discussion
Despite progress in adversarial robustness, current models remain vulncrable to practical black-box attacks. Inspired by the stability of biological perception, we introduced AlignFix, defense that combines naturally and adversarially trained models to perform correction via prodiction disagroement. AlignFix improves robustness across diverse attack types without requiring access to model internals, making it suitable for realistic deployment scenarios.
This work highlights the value of incorporating corrective feedback principles into model design and points loward future systems that may integrate memory or structured world knowledge to further enhance robust- ness.
References
Naveed Akhtar, Ajmal Mian, Navid Kardan, and Mubarak Shah. Advances adversarial attacks and defenses in computer vision: A survey. IEEE Access, 9:155161 155196, 2021.
Abdullah Al-Dujaili and Una-May O'Reilly. Sign bits are all you need for black-box attacks. In International Conference Learning Represenlations, 2019.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion,, and Matthias Hein. Square attack: a query-efficient black- box adversarial attack via random search. In European Conference on Computer Vision, pp. 484-501. Springer, 2020.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In International Conference on Learning Representations, 2018.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39-57. Teee, 2017.
Jianbo Chen, Michael Jordan, and Martin J Wainwright. Hopskipjumpattack: query-efficient decision- based attack. In 2020 deee symposium on security and privacy (sp), pp. 1277 1294. IEEE, 2020.
Jinghui Chen and Quanquan Gu. Rays: A ray scarching method for hard-label adversarial attack. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1739 1747, 2020.
Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop on artificial intelligence and security, pp. 15-26, 2017.
Sizhe Chen, Zhehao Huang, Qinghua Tao, Yingwen Wu, Cihang Xie, and Xiaolin Huang. Adversarial attack on attackers: Post-process mitigate black-box score-based query attacks. Advances in Neural Information Processing Systems, 35:14929- 35:14999-14443, 2022.
12

---

Published in Transactions on Machine Learning Research (08/2025)
Minhao Cheng, Thong Le, Pin-Yu Chen, Huan Zhang, JinFeng Yi, and Cho-Jui Hsich. Query-efficient hard-label black-box attack: An optimization-based approach. In International Conference on Learning Hepresentations, 2018.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack. In International Conference on Machine Learnöng, pp. 2196-2205. PMLR, 2020a.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206-2216. PMLR, 2020b.
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Fdoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: standardized adversarial robustness benchmark. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Bench- marks Track, 2021. URL https:://///// https:://ppenreieer http://weeeeooomm////ssssss/0..
Francesco Croce, Sven Gowal, Thomas Brunner, Evan Shelhamer, Malthias Hein, and Taylan Cengil. Eval- uating the adversarial robustness of adaptive test-time defenses. arXiv preprint fttt:2202.171111 2022.
Gamaleldin Elsayed, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alexey Kurakin, Ian Goodfellow, and Jascha Sohl-Dickstein. Adversarial examples that fool both computer vision and time-limited humans. Advances in neural information processing systems, 31, 2018.
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. Exploring the landscape of spatial robustness. In International conference on machine learning, pp. 1802-1811, PMLR, 2019.
Olukorede Fakorede, Ashutosh Nirala, Modeste Atsague, and Jin Tian. Improving adversarial robustness with hypersphere embedding Conference on Acoustics, Speech and Signal Processing (ICASSP), pp- 1-5. TEFF, 2023.
embbedddn and angular- regularizations. In ICASSP 2023-2023 IEEE International
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint Xrv: 14m::412.00 6572, 2014.
Chong Guo, Michael Lee, Guillaume Leclere, Joel Dapello, Yug Rao, Aleksander Madry, and James Dicarlo. Adversarially trained neural representations are already as robust as biological neural representations. In International Conference on Machine Learning, pp. 8072-8081. PMLR, 2022.
Chuan Guo, Jacob Gardner, Yurong You, Andrew Gordon Wilson, and Kilian Weinberger. Simple black-box adversarial attacks. In International Conference on Machine Learning, pp. 2484 2493. PMLR, 2019.
Jeff Hawkins and Blakeslee Sandra. On intelligence. Macmillan, 2004.
Andrew Tlyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In International conference on machine learning, pp. 2137-2146 2146, PMLR, 2018a.
Andrew Ilyas, Logan Engstrom, and Aleksander Madry, Prior convictions: Black-box adversarial attacks with bandits and priors. In International Conference Learning Representations, 2018b.
Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Artificial intelligence safety and security, pp- 99-112. Chapman and Hall 2018.
Qizhang Lá, Yiwen Guo, Wangmeng Zuo, and Hao Chen. Squeeze training for adversarial robustness. Tn The Eleventh International Conference on Learning Representations, 2023.
Feng Liu, Bo Han, Tongliang Liu, Chen Gong, Gang Niu, Mingyuan Zhou, Masashi Sugiyama, et al. Proba- bilistic margins for instance reweighting in adversarial training. Advances Neural Information Processing Systems, 34:23258 23269, 2021.
Chen Ma, Xiangyu Guo, Li Chen, Jun-Hai Yong, and Yisen Wang. Finding optimal tangent points for reducing distortions of hard-label attacks. Advances in Neural Information Processing Systems, 34:19288- 19300, 2021.
13

---

Published in Transactions on Machine Learning Research (08/2025)
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Represenlations. 2018.
Yichuan Mo, Dongxian Wu, Yifci Wang, Yiwen Guo, and Yisen Wang. When adversarial training meets vision transformers: Recipes from training to architecture. In Advances in Neural Information Processing Systems, 2022.
Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Jun Zhu, and Hang Su. Boosting adversarial training with hypersphere embedding. Advances in Neural Information Processing Systems, 33:7779 7792, 2020.
Zeyu Qin, Yanbo Fan, Hongyuan Zha, and Baoyuan Wu. Random noise defense against query-based black- box allacks. Advances in Newral Information Processing Systems, 34:7650-7766 2021.
Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda. Towards spike- based machine intelligence with neuromorphic computing. Nature, (7774)))66007 617, 2019.
Changhao Shi, Chester Holtz, and Gal Mishne. Online adversarial purification based on self-supervised learning. In International Conference on Learning Representations, 2021.
Satya Narayan Shukla, Anit Kumar Sahu, Devin Willmott, and Zico Kolter. Simple and efficient hard label black-box adversarial attacks in low query budget regimes. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 1461 1469, 2021.
Chawin Sitawarin, Jacwon Chang, David Huang, Wesson Altoyan, and David Wagner. Defonding against transfer attacks from public models. In The Twelfth International Conference on Learning Representations, 2024.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv: 03825. 2017.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural worksrks In 2nd International Conference on Learning Representations, ICLR 2014, 2014.
Lue Tao, Lei Feng, Jinfeng Yi, and Songcan Chen. With false friends like these, who can notice mistakes? In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 8458 1458-8466, 2022.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. arXiv preprint arXtv:1805.12155 2018.
Jonathan Uesato, Brendan O'donoghue, Pushmeet Kohli, and Aaron Oord. Adversarial risk and the dangers of evaluating against weak attacks. In International Conference on Machine Learning, pp. 5025 5034. PMLR, 2018.
Yisen Wang, Difan Zou, Jinseng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Represen- tations, 2019.
Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. In International Conference on Machine Leurning, pp. PMLR, 2023.
Boxi Wu, Heng Pan, Li Shen, Jindong Gu, Shuai Zhao, Zhifeng Li, Deng Cai, Xiaofei He, and Wei Liu. Attacking adversarial attacks as a defense. ar preprint urXie:2106. 04938, 2021.
Dongxian Wu, Shu- Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. Advances in Neural Information Processing Systems, 33:2958 2969, 2020.
14

---

Published in Transactions on Machine Learning Research (08/2025)
Chaojian Yu, Bo Han, Mingming Gong, Li Shen, Shiming Ge, Bo Du, and Tongliang Liu. Robust weight perturbation for adversarial training. arXiv preprint arXin:2205 1 1486, 2022.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theo- retically principled trade-off between robustness and accuracy. In International conference machine learning, pp. 7472-7482. PMLR, 2019.
Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. Geometry- aware instance-reweightted adversarial training. orXiv preprint arXiv:2010.0110 2020.
Tianyuan Zhang and Zhanxing Zhu. Interpreting adversarially trained convolutional neural networks. In International Conference on Machine Learning, pp. 7502 7502-7511. PMLR, 2019.
15

---

Published in Transactions on Machine Learning Research (08/2025)
A Model agreement and correct prediction
We work with the assumption that when the two models agree, their agreed prediction tends to be correct. While this is intuitive, as the models are trained to make the correct prediction, we further analyze how and when this assumption holds.
Our assumption is based on the intuition that the chance of StrongM and WeakM predicting the same wrong class low. Formally, let the accuracy of StrongM be Pa and WeakM Pe: their predictions be Sp and Wp. and y be the correct prediction. Assume their predictions are independent and equally likely to predict any of the wrong labels, then
(1 Pp) Ps)
P(Sp=1)ys
(5)
(C (C-1)
(1 Pw)
P(W, y''''v
(6)
(C -
where is the number of classes. We have:
P(correct (ggre))
Picorrect, agree)
(7)
sanuoorreellgggee
Plincorrecl, agree)
P(Sp = y. W = y) En'ty P(Sp 1/' W 15)
(8)
(C 1) Ps Pw
(9)
ps) 11-pa (1-Pw)
Therefore, the assumption holds when Ps: Pw 30% for binary classification and could hold even when Ps: Pw is small for multi-class classification.
Further, as noted in the main paper, we experimentally verified this. For CIFAR-10 using the ResNet- 18 model, we found that when an adversary is both crafted and nudged using both strongM and weakM in parallel (i.e., joinlM), then nudging towards the correct class increases the agreement bet between the two models' predictions from 34.98% to 49. 49.20%, while they are nudged towards random class, the agreement decreases to 16.99%.
B Ablation Study
We conducted ablations for CIFAR-10 by varying the nudging step size s size, the number of nudging steps n step, the weighting between strongM and weakM in jointM, and by testing different strongM models.
Value for s size
Accuracy
0.015
0.018
0.020
0.022
0.025
Natural
88.60
88.70
88.70
89.10
89.40
Square (1k iterations)
81.30
81.60
81.40
82.80
82.00
RayS (1k iterations)
72.00
72.30
72.00
72.00
71.30
Transfer (using jointM)
69.60
68.00
67.30
66.70
65.30
Table 9: Effect on AlignFix accuracy for different values of s size, under various attacks. The results are for the first 1000 samples of CIFAR-10.
Increasing size improves robustness to SQA attacks but reduces performance transfer attacks, seen in Table g. For stet using more steps slightly boosts natural accuracy but reduces SQA accuracy and increases runtime (Table 10 making one-step nudging a practical choice. To construct joinIM, we experimented with different weights on strongM and weakM losses. Table that setting a = 0.5 yields good balance across natural accuracy, SQA, and transfer attacks. Finally, while adaptive nudging methods like AutoPGD could improve performance by tailoring perturbations per sample, their high computational cost makes simpler one-step nudging a more practical choice.
16

---

Published in Transactions on Machine Learning Research (08/2025)
Tr. steps
Size of
Natural Accuracy
Square Attack Accuracy
Square Attack
each step
Time (sec)
1
0.020
87.8
82.4
509.6
2
0.011
89.3
79.6
814.4
3
0.007
89.3
78.6
1123.1
Table 10: Efffec on Fix accuracy for different values of n steps, where total perturbation by nudging was clipped at 0.02. The results are for the first 1000 samples of CIFAR-10.
alpha
Natural Accuracy
Square Attack
Transfer Attack Ace' using surrog'
Accuracy
strongM
weakM
Joint.
0.0
91.8
80.3
70.84
77.88
70.76
0.1
91.4
82.7
69.88
78.94
71.49
0.2
91.0
84.0
69.48
78.94
71.80
0.3
91.1
84.3
69.51
79.31
71.75
0.4
90.9
84.8
69.38
79.51
71.79
0.5
90.9
84.5
69.31
79.45
72.20
0.6
90.4
85.1
68.78
79.49
72.12
0.7
90.3
85.5
68.72
79.66
72.41
0.8
90.3
85.2
68.56
79.77
72.50
0.9
89.8
84.4
68.78
80.11
72.44
1.0
86.6
81.7
65.62
84.44
77.11
Table 11:
Effeec AlignFix accuracy on CIFAR-10 for different values of a as used to define joint... AlignFix uses WideRessett228-10 models, while surrogate models use ResNet-18 architecture.
C AT-CNN shape biased
We have leveraged the observation that adversarially trained models, which we refer to as strongM, are shape biased, while naturally trained models, referred to as weakM, are texture biased. This observation is grounded in the findings of ZZhang & Zhu (2019). who conducted systematic qualitative and quantitative comparisons between adversarially trained CNNs (AT-CNNs) and normally trained CNNs. Below is some of their key observation:
Qualitatively, Figure their paper (borrowed in Figure based on SmoothGrad salience maps, shows that AT-CNNs consistently highlight. object contours across clean, saturated, and stylized images, whereas standard CNNs produce noisy, texture-focused maps.
Quantitatively:
- Their, Table 2 shows that on stylized images (which distort textures but preserve shape), AT- CNNs outperform standard CNNs by a wide margin, confirming texture invariance.
-Their Figure 7 (borrowed in Figure 6) demonstrraes a steep drop in AT-CNN accuracy under patch-shuffling (which preserves local textures but destroys shape), while standard CNNs remain largely unaffected, confirming AT-CNNs reliance on shape.
D Model source and training details used for defense
We provide the source of the WideResNet models in Table 12 For ResNet-18 and for cases where the corresponding model is not present on RobustBench (ie., strongAM for CIFAR-10), trained the model locally. We used Madry's et al. Madry et al. (2018) method to train all the adversarially robust models for CIFAR-10, which are used as M.rongM. In line with the settings used in the literature Wang et al. (2019); Liu et al. (2021), all the base models (i.e., those included in Table 4 as well) have been trained for 120 epochs using mini-batch gradient descent with an initial learning rate of 0.01 (0.1 for NidcResNet), which was decayed by factor of 10 at epoch 75, 90 and 100. The values for other hyper-parameten are weight decay: 0.0035 (0.0007 for WideResNet), momentum: 0.9, and batch size: 128.
17

---

Published in Transactions on Machine Learning Research (08/2025)
(a) Images from Caltech-256
(b) Images from Tiny ImageNet
Sensitivity maps based on SmoothGrad Smilkov et al. (0017) of three models on images under sa saluration, and stylizing, From top to bottom, Original, Saturation 1024 and Stylizing. For each group of images, from left to right, original image, sensitivity maps of standard CNN, underfitting CNN and PGD-L AT-CNN.
Figure 5: Figure 2 from Zhang & Zhu 2019:
0.052
0.91
0,150
5.730
0.550
0.541
0.008
use
NO FOD
(a) Original Image
(b) Patch-Shuffle
(c) Sach--Shuffl-
(d) Patch-Shuffle
Figure 6: Figure from Zhang & Zhu 2019 Visualization of patch-shufiling transformation. The first row
shows probability of 'cake' assigned by different models.
Dataset
strongM/weakkM
Model Architecture WideResNNe-28-110
Model Source
CIFAR-10
weakM
Standard
M-SAT
Trained locally
MTrADEM-TRADES
Trained locally
Mtrong.M-AP
Wn2020Adversarrial
strong.M-AP E
Wu2020AAversariaa extra*
strong
Wang2023Better WRN-28 28-10*
ImageNet
weakM
WideResNet-50
From PyTorch: wide resnet50
M-SAT
Salman2020Do 50 2*
Table 12: Source for different WideResNet models. indicates that the models are obtained from Robust- Bench Croce et al. (2021) and the corresponding source column contains the Model-ID
18

---

Published in Transactions on Machine Learning Research (08/2025)
E
Parameters used for SQA attacks
Method
Hyperparameterr
CIFAR-10
ImageNet
Square
(fraction of pixels changed every iteration)
0.05
0.3
SignHunter
(finite difference probe)
8(0, 253])
0.1 1))
SimBA
d (dimensionality of 2D frequency space)
32
32
order (order of coordinate selection)
random
random
(slep size per iteration)
NES
ó (finite difference probe)
2.55
0.1
" (image bp learning rate)
2
0.002
q (# finite difference estimations / step)
20
100
Bandit
ó (finite difference probe)
2.55
0.1
" (image lp learning rate)
2.55
0.01
7 (ouline convex oplimization learning rate)
0.1
0.01
Tile size (dala-dependent prior)
20
50
Ç (bandit exploration)
0.1
0.1
Table 13: Hyper-parameters as used for SQA attacks
We used the same parameters as used by AAA defense for most of the attacks. We adapted the code from BlackBoxBench (https://github comm/atlvr..l/Blaacbbeeeenn
except for SimBA, where we use SimBA- DCT as simple SimBA is unable to attack adpative attacks like ours and AAA. Further we provide the JSON files that have the values of parameters we used for the atlacks. The details of the parameters have been compiled in Table 13 For the square attack, for which we used code provided by auto attack, we used p init 0.05 for CIFAR-10 and 0.3 for ImageNet.
19


Reviews:
[{"review_id": "ThKIzrQxD7", "reviewer": "Reviewer_MXpr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- The work addresses a critical and timely issue within its research domain.\n- The motivations and objectives are clearly articulated and well justified.\n- ECAC offers a simple yet highly effective defense against a broad range of black-box attacks, including score-based, decision-based, and transfer-based methods.\n\n\nWeaknesses:\n- The paper is difficult to follow at the beginning, as it starts with method comparisons without proper context or citations (see Table 1, Page 1). It would be clearer to either move the table after the relevant explanation or bring forward the accompanying discussion to better support and clarify the table content.\n- The explanation of Figure 2 lacks clarity. It is not clear whether P_y​ refers to the predictions of StrongM, WeakM, or the ECAC method. Additionally, the meaning of the colors used in the figure is ambiguous. Does blue indicate correct classifications and pink incorrect ones, or vice versa? Please provide a more detailed explanation to clarify the figure, including what each symbol and color represents, and how it relates to the methods discussed.\n- See the requested changes for more details.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a novel adaptive approach to defend classification networks against adversarial attacks. The focus is primarily on black-box attacks, a challenging setting where many existing defense strategies fail. ECAC uses the fact that naturally and adversarially trained models rely on different feature representations. While adversarially trained models are robust by design, naturally trained models often resist attacks targeting their adversarially trained counterparts. ECAC uses both models jointly, adjusting inputs toward the naturally trained model's prediction only when their outputs agree, ensuring reliable error correction without sacrificing robustness. Experimental findings show that the proposed method performs competitively.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- I would appreciate clarification regarding Figure 1. As I understand it, the ECAC method utilizes the output of either the weak or strong model when both predict the same class. In cases of disagreement, the final prediction is taken from the strong model evaluated on the clean, unperturbed input. If this interpretation is correct, it raises a concern: relying on the clean image to improve robustness does not constitute an actual defense mechanism. As long as the clean input x is used instead of the adversarially perturbed input x′, the system is not exposed to an attack in the first place.\nAlternatively, if x in this context represents the perturbed input, could you clarify whether ECAC actively modifies or corrects this perturbed input before passing it to the strong model? This distinction is critical, as the current presentation introduces ambiguity.\nThis confusion is further exacerbated by the inconsistent use of notation: in Sections 2.1 and 2.2, the adversarially perturbed image is clearly denoted as x′. If, in Figure 1 or elsewhere, x′ is also being used to represent inputs perturbed for defense purposes, I would strongly recommend adopting distinct symbols to differentiate between adversarial perturbations and any modifications applied as part of the defense. Clear and consistent notation would greatly improve the interpretability of the method.\n\n- The details of the validation experiments are not provided in the paper. The only reference appears on Page 7, where it is stated that the first 2,000 samples are used as the validation set. A comprehensive explanation of the hyperparameter tuning process and validation experiments, at least for one representative experimental setup, is necessary to clarify the methodological details. Additionally, it is unclear whether these 2,000 validation samples were selected randomly. Please clarify this point.\n\n- Regarding Table 3, it is mentioned that \"AAA-Linear evaluated Square Attack on 1,000 randomly selected samples (one per class) that were correctly classified by the naturally trained model.\" However, the test setting for ECAC is not clearly described. Were the same 1,000 samples used to evaluate ECAC, or were the test images for ECAC selected independently? Please provide clarification.\n\n- Table 3 reports the defense performance on the large-scale ImageNet dataset. On this dataset, AAA-Linear outperforms ECAC for several black-box attacks, including SignHunter, NES, and Bandit, raising concerns regarding the relative effectiveness of the proposed method. Including additional results, such as evaluations on other datasets like CIFAR-100, would offer a more comprehensive perspective and facilitate a fairer assessment of ECAC's performance.\n\n- Revise the paper to correct typographical and notation errors, such as the use of s_size in Section 3.4.", "broader_impact_concerns": "ECAC improves robustness against practical black-box attacks but increases inference time, which may limit accessibility in resource-constrained settings. It also does not cover all types of robustness, leaving residual risks in safety-critical applications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "eDW6Mbf3pD", "reviewer": "Reviewer_sqwq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nThis paper is based on experimental observations, and hence designs an algorithm that utilizes an observed phenomenon (the disagreement between strongM and weakM) to nudge the input and hence corrects possible errors (perturbations). The observed insight is very interesting, and the proposed defense is validated on a variety of attack models and adversarial training models.\n\nWeaknesses:\n\nThe concept \"error correction\" does not quite accurately describe the method, as the algorithm seems not to correct any error, but more to perturb and gather more information. In this sense, the title and abstract part are misleading and less informative. The process of the proposed method and how human brains correct the information is not quite related, and the introduction part somehow implies the method is inspired by biological experiments on how human or primate brains \"correct errors in signals\". See summary of contribution. This is quite misleading.\n\nOn the other hand, I wonder simply claiming that strongM is just some type of adversarially trained model and definitely includes more structure features would be a bit limited. Hence, the paper needs to illustrate more on the key observation. That is, how many adversarially trained models did you use to verify the observation? Does it just exist in some defense types but not the others?\n\nAlthough the paper already includes lots of adversarially training models, it only presents part of the results. For example, Section 4.2.1 only shows the comparison with AAA, while other comparisons are hard to find.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a new method towards adversarial perturbations in neural networks, which utilizes the disagreement between a naturally trained network (weakM) and an adversarially trained network (strongM) to perform an error correction for the decision. Specifically, it observes that, when an attack is performed by utilizing both weakM and strongM, then perturbing the example towards the correct class would increase the disagreement between weakM and strongM, and vice versa (observation elaborated in Section 3.2). This information is utilize to nudge the input and reduce the possible errors in making decision. That is, the input will be perturbed towards the class that weakM predicts, and if strongM agrees the class on the perturbed example, the class will be returned; otherwise, the algorithm would trust the decision of strongM. Experiments on multiple attack models validates the effectiveness of the proposed defense against adversarial examples.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "While the paper claims solving an adversarial attack problem by inspirations from biological observation of human brain (which makes mistake within a very short period of time), the key model observation and true technical contribution seem much more limited than that. To this end, the paper is a bit overclaimed.\n\nThe title and abstract (also part of the introduction) need to be rephrased. Especially the abstract includes very limited information about the proposed method, the truly concerned problem, or any insightful observation.\n\nWhile it seems to claim that the defense is mainly for black-box attacks, the later part shows the defense model is designed for all kinds of attacks. This needs to be clarified.\n\nSection 4 needs to include more quantifiable comparisons of the proposed method and other baseline methods. It would be better if it elaborates on the choice of the baseline (and how it relates to the key observation).", "broader_impact_concerns": "The reviewer does not have extra concern for this part for now.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "uQoa7eSjT8", "reviewer": "Reviewer_Bh7h", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n(1) Novelty and Biological Inspiration: The paper presents a novel defense method, ECAC, inspired by error correction in human visual perception, which is a unique and intriguing approach to adversarial robustness. (2) Focus on Realistic Black-box Attacks: ECAC is specifically designed to mitigate realistic black-box threats, which is a crucial area in adversarial machine learning given that attackers often lack full model access. (3) Effectiveness Across Diverse Attack Types: The experimental results demonstrate ECAC's strong performance against a wide range of black-box attacks, including Score-based Query Attacks (SQA), decision-based, and transfer-based attacks. (4) Validation for Disagreement: The paper provides a clear and experimentally validated rationale for using model disagreement (WeakM and StrongM) as an error signal, highlighting that adversarial examples transfer poorly between these models. (5) The ability to provide comprehensive experiments evaluation across various datasets (CIFAR and ImageNet) and provide detailed results in multiple tables.\n\nWeaknesses:\n\n(1) Increased Inference Time: A significant limitation acknowledged by the authors is the substantial increase in inference time (approximately five times higher) due to the extra forward and backward passes required by ECAC. The authors state this might be \"acceptable in safety-critical contexts\", but a more in-depth discussion on potential strategies to mitigate this overhead, or specific scenarios where this tradeoff is truly acceptable, would be beneficial.(2) Limited Scope of Robustness: While effective against black-box attacks, the paper notes that ECAC \"does not necessarily extend to all types of robustness. This limitation could be further explored, perhaps with a discussion of why these specific transformations are not handled and if there are plans for future work to address them.\n(3) Adaptive Attack Algorithm Details: While Algorithm 2 is provided, a more explicit breakdown and explanation of its steps, particularly for readers less familiar with adaptive attack methodologies, could improve clarity.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a novel defense mechanism called Error Correction by Agreement Checking (ECAC), which draws inspiration from human visual perception to enhance the robustness of neural networks against black-box adversarial attacks. The core idea behind ECAC is to leverage the inherent disagreement between naturally trained models (WeakM) and adversarially trained models (StrongM) as a signal for potential errors. When a disagreement occurs, ECAC nudges the input towards the prediction of the WeakM model, effectively correcting adversarial perturbations and expanding the decision boundary of the robust model. The new knowledge presented is that this biologically inspired error correction mechanism significantly improves adversarial robustness across diverse attack types, including score-based, decision-based, and transfer-based black-box attacks, even when the adversary has full access to the model. The authors demonstrate that ECAC consistently outperforms existing baselines on CIFAR-10 and ImageNet datasets and maintains its effectiveness across different architectures like Vision Transformers (ViT). Furthermore, the paper highlights the value of integrating error correction principles into model design, suggesting a new direction for developing more robust AI systems.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Critical for Acceptance:\n(1) In-depth Discussion and Mitigation Strategies for Increased Inference Time: The five-fold increase in inference time is a significant practical drawback. While acknowledged as a \"practical tradeoff,\" for acceptance, the authors need to:  (A) Quantify Impact More Precisely, \nPropose Concrete Mitigation Strategies and elaborate on specific safety-critical contexts where this increased latency is acceptable. \n(2) While Algorithm 2 is provided, its explanation could be improved for broader accessibility.", "broader_impact_concerns": "(1) Potential for Misuse in Adversarial Attacks (Offensive Capabilities): While ECAC is designed as a defense mechanism, research into adversarial robustness inherently provides insights into how models can be fooled. The knowledge gained from understanding model vulnerabilities and developing defenses could, in theory, be leveraged by malicious actors to develop more sophisticated and effective adversarial attacks. A Broader Impact Statement should acknowledge this dual-use potential and discuss the responsibility of researchers to mitigate such risks. (2) The paper explicitly states that ECAC incurs a \"higher inference time, roughly five times higher, due to the extra forward and backward pass\". This significant increase in computational resources could lead to environmental impact and accessibility and equity concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)