SeRI: Gradient-Free Sensitive Region Identification in Decision-Based Black-Box Attacks
Authors: Feiyang Wang
Xingquan Zuo
Hai Huang
Gang Chen
Hangwei Qian
Venue: ICLR 2026 Poster (2026)
Keywords: Machine learning
AI safety
Decision-based adversarial attacks
Sensitive region
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: This paper presents a novel Sensitive Region Identification approach, SeRI, that efficiently enhance adversarial perturbations based on sensitive region identification in decision-based attacks.

Abstract: Deep neural networks (DNNs) are highly vulnerable to adversarial attacks, where small, carefully crafted perturbations are added to input images to cause misclassification. These perturbations are particularly effective when concentrated in sensitive regions of an image that strongly influence the model’s prediction. However, in decision-based black-box settings, where only the top-1 predicted label is observable and query budgets are strictly limited, identifying sensitive regions becomes extremely challenging. This issue is critical because without accurate region information, decision-based attacks cannot refine adversarial examples effectively, limiting both their efficiency and accuracy.
We propose Sensitive Region Identification, SeRI, the first decision-based method that assigns a continuous sensitivity score to each image pixel. It enables fine-grained region discovery and substantially improves the efficiency of adversarial attacks, all without access to gradients, confidence scores, or surrogate models.
SeRI progressively partitions the image into finer sub-regions and refines a continuous sensitivity score to capture their true importance. At each iteration, it generates two perturbation variants of the selected region by scaling its magnitude up or down, and compares their decision boundaries to derive an accurate, continuous characterization of pixel sensitivity.
SeRI further divides selected region into smaller sub-regions, recursively refining the search for sensitive areas. This recursive refinement process enables more precise sensitivity estimation through fine-grained analysis, distinguishing SeRI from prior binary or one-shot region selection approaches. Experiments on two benchmark datasets show that SeRI significantly enhances state-of-the-art decision-based attacks in both targeted and non-targeted attack scenarios. Additionally, SeRI generates precise heatmaps that identify sensitive image regions. The code is available at https://github.com/BUPTAIOC/SeRI.

Published as a conference paper at ICLR 2026
SERI: GRADIENT-FREE SENSITIVE REGION IDENTIFI- CATION IN DECISION-BASED BLACK-BOX ATTACKS
Feiyang Wang's Xingquan Zuo Hai Huang' .2, 2 Gang Chen?, Hangwei Qian*
Beijing University of Posts and Telccommunications, China
2 Key Laboratory of Trustworthy Distributed Computing and Services, China
Victoria University of Wellington, New Zealand
*CFAR and IHPC, Agency for Science, Technology and Research (A (A*STAR), Singapore i "Corresponding author: zucxq@bupt. edu. cn
ABSTRACT
Deep neural networks (DNNs) are highly vulnerable adversarial attacks, where small, carefully crafted perturbations are added to input images to cause misclas- sification. These perturbations are particularly effective when concentrated in sen- sitive regions of an image. However, in decision-based black-box settings, where only the top-1 predicted label is observable and query budgets are strictly limited, identifying sensitive regions becomes extremely challenging, This issue is critical because without accurate region information, decision-based attacks cannot refine adversarial examples effectively. limiting both their efficiency and accuracy. We propose Sensitive Region Identification (SeRI), the first decision- based method that assigns a continuous sensitivity score to each image pixel. It enables fine- grained region discovery and substantially improves the efficiency of adversarial attacks, all without access to gradients, confidence scores, surrogate models. SeRI progressively partitions the image into finer sub-regions and refines a con- tinuous sensitivity
capture their true importance. At each iteration, it generates two perturbation variants of the selected region by scaling its magnitude up down, and compares their decision boundaries to derive an accurate, con- tinuous characterization of pixel sensitivity. SeRI further divides selected region into smaller sub-regions, recursively refining the search for sensitive areas. This recursive refinement process enables more precise sensitivity estimation through fine-graine gree-grriedd analysis, distinguishing ScRI from prior binary or one-shot region selection approaches. Experiments on two benchmark datasets show that SeRI significantly enhances state-of-the-art decision-based attacks in both targeted and non-targeted attack scenarios. Moreover, SeRI produces precise heatmaps of sen- sitive image regions, providing strong validation of the attack process. The code http://hhh..cco//////A/A///////
is available
INTRODUCTION
Deep neural networks (DNNs) have achieved superiority in tasks such as image classification Brun- ner al. (2099. However, they remain highly susceptible carefully crafted adversarial examples generated by adversarial attacks Dong et al. (2020); Chen & Gu (2020); Chen et al. (2020a). Since these vulnerabilities undermine the reliabliiy and security machine learning systems, adversarial robustness has emerged as a central research focus. Li et al. (2021).
Adversarial attacks are typically grouped into white-box attacks Goodfellow et al. (015)
; Carlini
& Wagner )): Madry et al. (2018), gray-box attacks (or soft-label attacks, score- score-based attacks, confidence-based attacks) Chen et al. (0017; īlyas et al. (2019), and black-box attacks Brendel ct al. (2018); 8); Cheng et al. (2019) ; Chen & Gu (2020), based how much information the attacker can obtain about the target models. White- and gray-box attacks require full or partial access to the target model, such as its network architectures, parameters, or confidence scores, which is rarely available in real-world scenarios Chen Gul (2020). Consequently, black-box attacks have drawn significant interest in the research community Cai al. (2022).

---

Published as a conference paper at ICLR 2026
Initialize
Iteratively refine perturbation within selected sub-region
Output
Updatte
The first adjosted perturlition
Split current regios
Sensttive regiee bratmap
Gousian
suberegiond
HSIA
Enhance sub-region pertorbatien
OPT Rays
Compaan pare the Decision Houndary of Three
Split current region
selbrrpionn
advernarial examppl
Per Pertrbatioons tions
adjusted
Superior
perturbation
percurbation
perturbation
Redace sab-region perturbattis
Update
Figure 1: Perturbation optimization process of SeRL. The process starts by generating an initial adversarial example using Gaussian noise or classical decision- based attack such HSJA. The perturbation split into four sub-regions, and the one with the highest f2-norm selected. Two new perturbations are created by enhancing or reducing the perturbation in that sub-region. SeRI compares their decision boundaries and keeps the one with the lowest value. In the next iteration, the selected sub-region is further subdivided for fine-gratd fine-grained optimization. A heatmap highlighting sensitive regions is then generated from the refined perturbation.
Black-box attacks fall into main categories: transfer-based Wang (2024); Sun al. 2024 Park et al. 2024) and decision-based [Cen & Gu (2020); Chen et al. (2020a); Reza et al.
(2025) Transfer-based attacks train a surrogate model using the target model's data and then generate adversarial examples via white-box attack techniques. However, poor per-
2023 Wang
turbation transferability often limits their attack success rate Reza al. (2023). Decision-based attacks Brendel et al. 20188; Chen et al. al. (2020a): Chen & Gu (2020); Reza et al. (2023); Wang et al. (2025) aim to mislead a target DNN model by introducing minimal perturbations while operating under limited query budget Li al. (2021). Unlike white-box. gray-box,
transfer-based attacks, they do not rely on the target model's 's training data, architecture, or output confidence scores. In- stead, they operate purely the model's top-1 predicted label, which is often available in real-world systems Dong et al. (2019); Brunner et (2019).
The role of sensitive regions in adversarial attacks. It is well known that image typically consists of both sensitive regions (i.e., salient objects like an eagle's head in Figure (b)) and non- sensitive
sensitive regions (i.e., irrelevant backgrounds in Figure Fig Focusing perturbations regions, which are areas that contribute mostly to the model's prediction, has been shown to sig- nificantly improve the success rate of adversarial attacks. However, due to the limited information access and strict query constrains in black-box attack settings, identifying sensitive regions becomes significant challenge Shi et al. 2022): Lin et al. (2023). Existing black-bo attacks generally adopt two primary strategies to exploit sensitive regions.
The first strategy gives rise to the transfer-based region-aware approaches Chen al. (2020b); Dong et al. (2020); Lovisotto et al. (2022); Lin et al. (2023). These methods generate sensitive region heatmaps from surrogate model using white-box interpretability techniques, and then use the heatmaps guide perturbation generation architectures, such as Vision Transformers Dosovitskiy (2020) and ResNets He et al. (2016), often focus distinct features regions Shi et al. (2022). As result, the heatmaps generated from the surrogate model may fail to capture the fecision-critica regions the target model, leading to suboptimal attack performance.
the target model. However, models with different
The second strategy relies on decision-based region sensitivity estimation, where sensitivity is inferred from the model's top-1 prediction. PAR Shi al. (2022) is the primary method that
2

---

Published as a conference paper at ICLR 2026
emplifies this approach with patch-wise removal strategy: it deletes a perturbation block, queries the model, and labels the region as sensitive or not based on the hard-label prediction. To reduce query cost, PAR further uses binary decision process that either keeps removes the perturbation region, without any fine-grained adjustment. This approach restricts exploration of the solution space, leading to lower attack success rates. In reality, different pixels respond to perturbations with varying levels of sensitivity. This means perturbations should be weighted in proportion to each pixel's sensitivity, rather than applied in all-or-nothing manner where an entire region is either kept or removed.
We introduce a new definition of region sensitivity grounded in the perturbation decision boundary (see Section 4 for details). Unlike prior binary formulations, this new definition enables a continuous and fine- fine-grained quantification sensitivity, providing a principled foundation for adaptive pertur- bation refinement in decision-based attacks. Different from PAR that treats each region either sensitive or insensitive PAR, our approach introduces an innovative continuous sensitivity formu- lation that assigns perturbation weight score to each pixel. It enables fite-grained control over perturbation strength across regions, leading to targeted and effective perturbation optimization.
Building this definition, we pro- pose Sensitive Region Identification (SeRI), approach that is efficient in both computation and queries, adaptively optimize perturbations down to the level of individual pix- els. SeRI simple implement in decision-based attack settings and can be seamlessly integrated as plug-in perturbation optimizer to re- fine the perturbations generated by various baseline attackers like CGBA Reza et al. (023). This "Attacker ScRI" framework significantly en- hances baseline attacker with mini- mal query-budget.
Input image and perturbation
I-5, Q-13,
0-50, 4 -0.515
I-118, O-412,
I-504, Q-2001,
4. -0.586
4 -0.457
4 -0,413
(a) Non-targeted attack: class 'Lacewing' is misclassified as 'Walking Stick'
Input image and perturbation
1-1. I=1.Q=7,
I-2, Q-10,
I-9, Q=30,
1-492, Q=2000.
1. -11.611
h -1.686
4 4-1.622
f -1.067
An overview of our approach is il- lustrated in Figure [ Specifically, we generate two perturbation vari- ants by either increasing decreas- ing the perturbation strength within the selected region. These two vari- ants, along with the original pertur- bation, are then compared based on their decision boundaries. For this comparison, we adopt the Approxi- mation Decision Boundary Approach (ADBA) Wang et al.) (2025). which operates effectively with minimal query budget. The current pertur- bation is subsequently updated with the variant that yields the smallest es- timated decision boundary. Exper- imental results on two datasets and three models confirm its effectiveness and generalizability. The sensitivity-region heatmaps and the corresponding optimized perturbations produced by SeRI are shown in Figure SeRT's precise thermal grading not only accelerates per- turbation optimization by focusing on the most influential regions, but also produces heatmaps that align with human visual perception, thereby enhancing the interpretability of the attack process.
(b) Non-targeted attack: class 'Bald Eagle" is misclassified as 'Vulture'.
Ioput image and
1-148, Q-400,
Q-2000,
-8,762
-8.695
-8.276
-6.917
(c) Targeted class kaple' misclassfiedd
Figure 2: Heatmaps generated by SeRL, showing how the heatmaps, perturbations, and their &n-norss evolve over it- erations (I) and queries (Q).
3

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Decision-based attacks represent one of the most challenging setting in adversarial robustness, as only the model's top-1 predicted label is observable and no gradient information is available Chen & Gu 2020). Boundary AttackBrendel et al. (2018). Biased Boundary Attack [Brunner et al. (2019) and AHA et al. (202T) perform random walks along the decision boundary to gradually reduce the perturbation strength. Triangle Attack Waag et al. (0022) designs structured fiianglebbased per- turbation in the low-frequency domain to improve efficiency. SurFree Maho et al. (202T) explores multiple carefully chosen perturbation directions simultancously, without relying on gradient esti- mation. OPT[Cheng et al. (2019) and Cenn-OPTCCheenn et al. (2020) reformulate the decision-basee attacks continuous optimization problem solvable via zeroth-order methods. RayS Chen & Gu (220)) and ADBA Wang et al. (2025) use a progressive search strategy that adaptively subdivides perturbation directions to accelerate convergence. Some attacks guide perturbation optimization by estimating the normal vector of the decision boundary at a boundary point. For example, qFool Liu al. (2019) and GeoDA Rahmati al. (2020) improve gradient estimation efficiency by leveraging the fact that decision boundaries near adversarial examples typically have low curvature. QEBA Li al. (2020) reduces query complexity by performing subspace optimization in both the spatial and the frequency domains. HSJA Chen et al. (2020a), 'A Ma et al. (2021). and Bounce Wan et al. (2024) estimates normal vectors to generate adversarial example. Achieving tate-of-the--a performance under the fo-nomm CGBA |Reza et al. (2023) introduces novel semicircular search strategy within a two-dimensional subspace to effectively navigate geometric complexities.
Although recent decision- attacks achieve good performance, they usually ignore which region of the image are more sensitive to the model. Because they generate global perturbations without considering regional differences, the perturbations are often large and easy notice.
Sensitive region- based attacks. Many classical XAI methods such as Grad-CAs CAM Selvaraju et al. (217), Occlusion Zeiler & Fergus (0014), LIME Ribeiro et al. (2016), and SHAP Lundberg & Lee 2017 can highlight important or influential regions of an input image. However, these methods primarily designed to explain the prediction behavior of model rather than to identify regions that are most useful for generating adversarial perturbations. In addition, they rely on gradients, confidence scores, or internal activations compute importance, all of which are unavailable in hard-label decision- attacks. Nonetheless, these interpretability techniques provide a natural motivation for using sensitive regions to guide perturbation generation in adversarial attacks.
Building on this idea, existing work attempts to explicitly use sensitive regions to improve attack quality. These methods can be broadly categorized into three groups: (1) attention-basel methods
Dong et al.] (2020); Chen et al. (2020b). (2) surrogate- methods Lin et al. (2023) and (3)
decision-based methods et al. (2022): Tao et (2023).
Superpixel-guided Attentional Attack (SGA) Dong et al. (2020) perturbs only the regions selected
by attention maps. Attack on Attention (AoA) [Chen et al. (2020b) optimizes an attention-based loss to improve transferability. However, both SGA and AoA need attention maps or activation features, which require white-box access and cannot be used hard-label black-box settings. To avoid these requirements, SRA Lin et al. (2023) uses surrogate model to generate sensitivity maps, but it needs the target dataset to be known. Saliency Attack Dai et al. (0023) uses model-agnostic saliency detection, but the saliency maps may be inaccurate and fail to locate the truly important regions. Decision-based attacks such as HardBeat Tao et al. (2023) search for one vulnerable patch and perturb only that patch. However, when multiple regions jointly affect the classifier, focusing on only one patch becomes incomplete. PAR (Shi et al. (2022) estimates region sensitivity through a patch-wise removal strategy, but its binary keep/remove decision is coarse and cannot reflect con- tinuous importance inside each region. Overall, current methods cannot provide continuous region sensitivity under the strict decision-basse setting.
PROBLEM DEFINITION
Let p E [0, 1] be an image pixel and x = p xC represent a source image with channels C, width W, and height H, respectively, and y(z) denote the true label of x. Let f x + (1...., K) represent a K- -lclass image classification model. Given a source image x which is classified correctly by model f (i.e., f(x) y(x)). The goal of decision-bused black-box altacker is to find an
4

---

Published as a conference paper at ICLR 2026
adversarial example i pxxWxxx P E [0,1], such that f(ø) y(z) for non-targeted attacks, or f(ă) = f (m)))) for targeted attacks, while minimizing the perturbation strength || 1 Here, Tua given target image and (((r)) # y(z), and || ll stands for the 2--norm used measure the perturbation strength. The problem of optimizing the adversarial example é can be formulated as:
arg min j x112 s.t. 1(8)
where I(-) is an indicator function that determines whether the adversarial example ä is in the adversarial regions. For non-targeted attack:
(()) 1 if + v(z),
-1, otherwise.
For targeted attack with targeted image F'argeti
if f()) (drrnga);
((t) -1, otherwise.
Let perturbation be denoted as = CXWXX where v E [--1, An adversarial example then given by H clamp(( (i x)) = clamp(x+d) = clamp(( +|d||2
where clamp(-)
constrains each image pixel to the range [0, 1]. We define the decision boundary for any perturbation
d as g(d) = min(r > 0 [(x + r. Tdllaa 1}. Consequently, the optimization problem in Eq. can be reformulated as:
d
d
arg grgmin((d) s.t. 1(x g(d)
|dd|2
=
(4)
4 PROPOSED APPROACH
The proposed SeRI approach is applied after base attacker (e.g., HSJA), refining its perturbation by identifying sensitive regions through additional queries. With total query budget Q and SeRI query budget fraction P. query allocation is (1 11-P) P) Q for the base attacker and P Q for SeRI. In this paper, we set P 20% for datasets and models, based on the parameter sensitivity analysis presented in Appendix C of the supplementary material. The introduction of base attackers are introduced in Section Here, we focus on our proposed SeRI approach.
4.1 DEFINITION OF PERTURBATION SENSITIVITY
Existing sensitivity definitions be roughly grouped into two families: gradient- based definitions and perturbation-ban based definitions. Gradient-based definitions, such Integrated Gradients (IG) Sundararajan al. (2017), Grad-CAM ( et al. (2017), and their variants, define the sensi- tivity of pixel by the gradient of a target score with respect to the input. Let x [0, 1]xxWx denote the input image, yxx) its true label, and (x(x)()) the logit (or score) for class y(z). A typical pixel-wise sensitivity can be written as
0 (2(())(())
2 (1)
S0,w.h
8 Σ
(5)
Such definitions capture which pixels regions are most influential for the model's internal deci- sion. However, they are not directly usable in decision-based black-box attacks, where gradients with respect to the input are completely unavailable.
Perturbation- definitions estimate sensitivity by actively modifying the input and observing how the model output changes. Representative examples include Occlusion-Basee explanations Zeiler Fergus (2014), SRA Lin al. (2023), and PAR (Shi et al.] (2022). Occlusion and SRA define wise sensitivity via the drop in the predicted confidence of the true class after perturbing a single pixel. Let Py(a)) (I)) denote the predicted probability of class y(x). For each spatial location
(w, h),, they consider a small &oo perturbation of size > 0 on channels at that pixel:
(x) Pa(a) (x: + (...))
Sw.h
(6)
t
5

---

Published as a conference paper at ICLR 2026
where ow.A RCXW>X is zero everywhere except pixel (w, hw, with all C' channels equal to 1. Intuitively, w measures how much the true-class probability decreases when only pixel (wo, h) is slightly perturbed, and thus serves pixel-wise sensitivity score in the black-box setting.
PAR, on the other hand, defines sensitivity at the patch level. Let be an adversarial example the corresponding perturbation. For region (patch) with binary mask M, E {0, 1]cx xXCXW*H H PAR uniformly rescales the perturbation inside b by a factor R E [0, 1]. and the region-wise sensitivity of b is then defined as:
for æ, and d ã
SpAR (b) = min { x E [0,1] 1(x ( d(w)) = =1].
d(hm)) d d d1 R)) Mb,
(7)
where denotes element-wise multiplication and I(.) is an indicator function. smaller value of SpAR (b) means that more perturbation in region b can be removed while keeping the perturbation
adversarial, hence b is less sensitive; a larger value indicates a more sensitive region.
Limitations of existing crrturbatioonbaase sensitivity definitions. Although intuitively appeal- ing, the above perturbation-ban based sensitivity notions have fundamental limitations when applied to continuously optimizing adversarial perturbations in decision-based attacks. While the sensitivity scores in Occlusion, SRA, and PAR are real-valued, they do not provide guidance on how to adjust perturbation strength in smooth, continuous manner.
For Occlusion and SRA, the sensitivity sw. indicates the drop in confidence when pixel (w, h) is perturbed, but it does not specify how the perturbation strength at each pixel should be rescaled relative to others to make the global adversarial perturbation more effective, e.g. to reduce the decision-boundary. For PAR, (p) gives the maximal compression ratio that preserves attack success for each patch b, but this value still cannot serve as continuous scaling factor for perturba- tion refinement. Reducing perturbation strength in patch exactly its PAR threshold produces an adversarial example that is only barely successful; any further reduction in other regions may cause immediate attack failure. This makes smooth, iterative refinement essentially infeasible.
In summary, these sensitivity definitions characterize local relationships between perturbation changes and model outputs (scores or labels), but they do not explicitly capture how local adjust- ments influence the global effectiveness of the perturbation. For decision-basee -asee black-box attacks, we argue that a more appropriate notion of perturbation sensitivity should be defined in terms of its impact on the overall adversarial effectiveness, as quantified by the decision-boundar g(d).
Our decision seeriii--oundary-baasee function I(-), an original image x, and an initial perturbation d that successfully deceives the model (i.e.. 1(x + d) = 1), we define the model's adversarial perturbation sensitivity as a tensor E RCx XH where each element se,,h 2 0 represents the sensitivity weight of pixel (c, w, h). In setting, we seek a transformed perturbation S d (element- wise product) that minimizes the decision boundary g(S d). To ensure that the overall perturbation strength remains unchanged, we enforce an &n-ormm constraint ||S d||2 ||d]|2. Accordingly, the task of optimizing perturbations based on sensitivity becomes
sensitivity definition. Given a target DNN model f. its indicator
the decision-
arg min g(S d), subject to ||S ddl2 ||d]l2.
SeRI perturbation refinement
Eq. defines a continuous optimization problem over a high-dimensional space S E RRCW which makes the problem especially challenging to solve. To manage this complexity, we adopt an iterative region-splitting strategy. We start from the initial region 6D = {l:C, 1:W, 1:H]} and maintain set of non-overlapping blocks B iteration i, with 8D {b*). At each iteration, we adjust the perturbation only within a single block and then subdivide this block into smaller sub- regions. In practice, each selected block is split into four equal sub-regions; Appendix B.T shows that the efficiency of SeRI is largely insensitive to this choice.
Given the current perturbation d' and block set B we select one region to optimize based on its local norm::: we set b" = arg maxse ||d] llz, where dig denotes the restriction of d' to region b. This heuristic prioritizes regions with larger local perturbation, which have greater potential for perturbation reduction. After updating d' within b*. we replace 6" in Bi by its four sub-regions to obtain Bi+1, enabling progressively finer control.

---

Published as a conference paper at ICLR 2026
Once region d" is selected, assess its sensitivity by measuring how local rescaling of the pertur- bation b* changes the overall decision boundary. Let Mr. E {0, 1 CxcxxxX be the binary mask of 6", where entries inside 6* are 1 and 0 elsewhere. We construct three candidate perturbations:
dg d',
1|0
di
(d' + (ie - 1)d ④ My-),
| + (bs 1) d' @ Ma-12
||d/|12
d/ = | + 11 d' @ Mb-112
(d' + (bc - 1)d' My.),
where 0 < h < < & and @ denotes element-wise multiplication. To keep the overall perturbation strength unchanged, all candidates normalized to have the same 22-norm d'. We then choose g(d)) and
j = arg min (E(0,1,2)
and update d' = dzz..
(10)
=f="1,, the perturbation in da is reduced; j 2, it is enhanced; j no change is applied, indicating that b* already close to locally optimal.
This adaptive update strategy guarantees a monotonic decrease of the Decision-boundayy distance each iteration and progressively steers the perturbation of each pixel toward its optimal sensitivity level under the &2 constraint. Theoretical justifications of this monotonic improvement and conver- gence to a stationary perturbation are provided in Appendix A summarized in Algorithm Tof Appendix B
The complete procedure of SeRI is
Comparing decision boundaries of candidate perturbations
In Eq. I0 ScRI updates the current perturbation by selecting the best candidate among do. d1, and d2 based on their ceerision-boundare This requires a query-efficient procedure to compare the decision boundaries. We adopt the Approximation Decision Boundary Approach (ADBA) (aag et (025). which is specifically designed for low-cost decision -oundary comparisons (see Appendix B.2 for details). result, SeRI is highly query-efficient. For example, as shown in Figure a), optimizing single perturbation over 504 SeRI iterations consumes only 2001 queries in total, i.e., about four queries per iteration (2001/504 Rs 3.97).
The effectiveness of ADBA relies several standard assumptions, SeRI, which builds upon ADBA, also inherits these assumptions (stated in detail in Appendix [.I]: (I) local Lipschitz continuity of the decision boundary, (2) locally bounded curvature of the decision boundary, and (3) deterministic hard-label outputs of the target model. Assumptions (1) and (2) are mild and are typically satisfied by modern deep networks such as CNNs and Vision Transformers. Assumption (3), however, rules out randomized defenses (e.g., randomized smoothing or random input transformations Xie et al.) (2018); Raff al. (20T9)), and thus SeRI is not intended for such settings. But for other defenses that would not break these three assumptions, such as adversarial training (AT) Zagoruyko (2016)
and Lipschitz- baseed defenses Tsuzuku et al. (2018) Araujo et al. (2023)
EXPERIMENTS
SeRI improves the perturbations generated by base attackers (e.g., HSJA) by identifying and refin- ing sensitive regions through additional queries. To be effective, the combined "Attacker + SeRI" pipeline must offer better overall query efficiency than the base attacker alone. SeRI is also ex- pected to outperform other sensitivity-awar refinement methods in terms of query usage. To verify these questions. SeRI is evaluated on two famous image classification datasets. SeRI is further compared against multiple decision-based and sensitivity-based attack methods in both targeted and non-targeted settings. All experiments were performed using an Intel Xeon Gold 6330 CPU and four NVIDIA GeForce RTX 4090 GPUs with PyTorch 2.3.0, Torchvision 0.18.0, and Python 1.5.
5.1 EXPERIMENT SETTINGS
Competing approaches. We compare the performance of SeRI with four well-known decision- based attacks, including HSJA, Chen al. (2020a), CGBA Reza et al. (2023), RayS Chen & Gu

---

Published as a conference paper at ICLR 2026
Table Average (median) lg perturbation norms for targeted and non-targeted attacks on ImageNet using a VGG model.
Non-targeted attacks
Targeted attacks
Total Query
2,000
5,000
10,000
2,000
5,000
10,000
HSJA
8.18(5.29)
4.43(2.51)
3.39(1.91)
72.3(66.5)
50.6(36.9)
33.7(20.8)
HSIA+PAR
6.65(3.61)
3.88(2.35)
3.08(1.82)
53.0(50.1)
35.4(32.3)
20.4(14.0)
HSJA+SeRI
6.47(3.55)
3.63(2.22)
2.85(1.51)
48.3(46.5)
32.0(29.9)
18.5(12.3)
CGBA
3.91(2.02)
1.9(11.1)
1.19(0.75)
77.4(74.9)
58.8(56.4)
40.2(33.1)
CGBA+PAR
2.81(1.55)
1.46(0.85)
1.03(0.64)
58.3(55.1)
39.4(37.8)
23.1(15.6)
CGBA+SeRI
2.92(1.39)
1.36(0.70)
0.96(0.54)
53.0(50.0)
36.0(33.2)
21.1(13.3)
RayS
5.14(3.50)
3.54(2.27)
2.72(1.76)
RayS+PAR
3,48(2.38)
2.23(1.37)
1.75(1007
RayS+SeRI
3.46(2.29)
2.16(1.27)
1.56(0.92)
ADBA
4.04(2.77) 3.03(1.94)
2.44(1.53)
ADBA+PAR
2.82(1.74)
2.02(1.19)
1.67(0.99)
ADBA+SeRI
2.78(1.71)
1.95(1.07)
1.49(0.87)
Table Average (median) le perturbation norms for targeted and non-targeted attacks on ImageNet using VIT model.
Non-targeted attacks
Targeted attacks
Total Query
2,000
5,000
10,000
2,000
5,000
10,000
HSJA
13.2(9.47)
6.78(4.27)
4.18(2.88)
34.3(30.0)
16.0(15.2)
8.09(7.94)
HSIA+PAR
9.62(6.19)
4.98(3.33)
3.85(2.46)
27.7(24.1)
13.0(9.13)
6.35(5.54)
HSIA+ScRI
9.02(4.40)
4.13(1.95)
3.25(2.21)
26.9(23.1)
12.0(8.81)
6.15(5.27)
CGBA
4.59(3.13)
2.33(1.52)
1.59(1.05)
36.6(31.3)
14.1(10.3)
5.80(4.73)
CGBA+PAR
3.67(2.36)
2.08(1.45)
1.40(0.94)
29.3(25.6)
12.4(8.42)
4.79(4.26)
CGBA+SeRI
3.53(1.99)
1.89(1.29)
1.27(0.85)
29.0(24.0)
11.9(8.99)
4.61(3.91)
RayS
10.7(5.24)
7.20(3.50)
5.15(2.79)
RayS+PAR
7.05(3.75)
4.62(2.61)
3.33(2.02)
RayS+SeRI
7.01(3.69)
4.52(2.00)
3.20(1.55)
ADBA
7.56(5.06)
5.03(3.63)
3.65(2.71)
ADBA+PAR
5.67(3.65)
3,87(2.60)
2.82(1.95)
ADBA+SeRI
5.53(3.62)
3.59(2.10)
2.39(1.26)
ADBA Wang al. (2025). and PAR Shi al. (022). HSJA and CGBA operate under the 2-norm constraint, with CGBA representing the state of the art among such attacks. Additionally, both methods support both targeted and non-targeted attack scenarios. Meanwhile, RayS and ADBA are leading methods with respect to the -oomoomm setting. Building on RayS, ADBA leverages an approximate decision boundary enable efficient perturbation comparisons, significantly reducing query usage. Due to their fe -oom constraint, both methods are limit to non-targeted attacks Chen & Gu (2020); Wang et al. 2025)). Although RayS and ADBA are originally designed for the fo -norm attacks, SeRT can still be applied to refine their perturbations, enabling improved performance under the f2-norm constraint. Similar to SeRI, PAR region- sensitivity perturbation optimizer for the decision- based setting. It can serve noise-initialization module for other decision-based attacks, enhancing their noise-compression capability.
2020
We evaluate SeRI across 12 attack configurations: the four base attackers (HSJA, CGBA, RayS, and ADBA), their respective "Attacker + SeRI" variants, and their "Attacker + PAR" counterparts. All configurations assessed based on attack performance under the fn-morm constraint.
Benchmark datasets and models. we conduct comprchensive experiments on two datasets us- ing three representative models: ImageNet Deng al. (2009) with VGG19 Simonyya & Zisser- man (2015) and the Vision Transformer (ViT) Dosovitskiy (2020) architecture, and CIFAR-
Krizhevsky et al. (2009) with adversarial trained WideRes et model from Wang et al. 2023
Hyperparameter settings. We adopt the recommended hyperparameter settings in Chen et al.
2020a): Chen & Gul (2020); Shi et al. (2022): Reza et al. 2003): Wang et al. (025 Specifically,
8

---

Published as a conference paper at ICLR 2026
for all four competing algorithms and SeRI, the decision boundary search tolerance 10-5 We also follow hyperparameter settings of PAR Shi et al. (2022) and set the initial and minimum patch size to 56 and I, respectively. In SeRI, according to the parameter sensitive analysis in Appendix C, the thresholds k 0.2 and 1.8, and the query budget percentage / is set to 20%.
5.2 EXPERIMENT RESULTS
For each model, randomly select 1000 test images. Tables 1 and 2 present the average and median 2n-norms achieved by all competing approaches under query budgets of 2,000, 5,000, and 10,000, for both non-targeted and targeted attacks. The best values are highlighted in bold.
Main results. As shown in T CGBA+SeRI achieves the best non-targeted attack performance at both 5,000 and 10,000 queries, outperforming others in terms of average and median f2-norms. At 2,000 queries, it achieves the best median performance, while ADBA+SeRI obtains the lowest av- erage l2 norm. For targeted attacks, HSJA+SeRI consistently achieves the best performance across all query budgets. As evidenced in Table CGBA+SeRI consistently outperforms other methods in both non-targeted and targeted attacks at 5,000 and 10,000 queries, achieving the lowest average and median l2 norms. The only exception is at 2,000 queries for targeted attacks, where HSIA+SeRI performs better.
The results show that the "Attacker SeRI" variants consistently deliver the best performance across all attack settings. For any given base attacker (e.g., CGBA), combining it with SeRI outperforms both the attacker alone and its "Attacker PAR" counterpart. Particularly, for the targeted attack results in Table "Attacker + SeRI" achieves superior performance across all query budgets. These results confirm that SeRI's region sensitivity estimation significantly enhances the query efficiency of decision based attacks.
Results adversarially trained WRN model. Similar to the results non-adversarially trained VGG and VIT models, Table 3 shows that CGBA+SeRI achieves the best per- formance at 2,000, 5,000, and 10,000 queries. Additionally, in Table 3 "Attacker + SeRI" reduces the E norm of perturbations by approxi- mately 30% compared to "Attacker PAR". also reduces the fn-norm by 15% compared to "Attacker PAR". as evidenced in [ad These results indicate that SeRI provides greater performance gain on adver- sarially trained WRN models than on non-adversarially trained models. This performance advantage likely due the increased difficulty of attacking adversarially trained WRN models, which demands more precise and adaptive perturbation optimization. PAR's binary decision mechanism is less effective against the stronger defenses adversarially trained WRN models. In contrast, SeRI's fine-grained sensitivity search enables significantly better performance in this challenging setting.
Table 3: Average (median) l perturbation norms for non-
targeted attacks on CIFAR 100 using WideResNet model.
Total Query
2,000
5,000
10,000
HSJA
3.26(2.22)
1.75(1.15)
1.26(0.88)
HSIA+PAR
2.59(..74)
1.52(0.79)
1.18(0.83)
HSTA+SeRI
2.08(1.41)
1.40(0.70)
1.13(0.63)
CGBA
2.20(1.54)
1.48(1.01)
1.19(0.84)
CGBA+PAR
1.71(1.14)
1.22(0.73)
1.07(0.65)
CGBA+SeRI
1.48(0.86)
1.14(0.67)
1.02(0.62)
RayS
3.17(2.22)
2.68(1.84)
2.49(1.69)
RayS+PAR
2.54(1.79)
2.29(1.70)
2.15(1.49)
RayS+SeRI
1.79(1.28)
1.55(1.08)
1.44(1.01)
ADBA
2.93(2.13)
2.62(1.77)
2.46(1.67)
ADBA+PAR
2.41(1.80)
2.18(1.63)
2.09(1.50)
ADBA+SeRI
1.74(1.23)
1.50(1.09)
1.43(1.04)
5.3 HEATMAPS GENERATED BY SERI
Figure shows the heatmaps over sensitivity regions generated by SeRI across different optimiza- tion iterations and queries. More heatmaps generated by SeRI and PAR can be found in Appendix [ In these heatmaps, regions with stronger red intensity indicate areas where SeRI increases perturba- tion strength, while regions with stronger blue intensity mark non-sensitive areas where perturbation strength reduced. Figure (a), the class 'Lacewing' is initially misclassified as "Walking Stick'.
With more iterations, the high-intensity regions in the heatmap increasingly converge and concen- trate on the lacewing itself. This example demonstrates the precise thermal grading capability of
SeRI method, which is superior than competing PAR method (see Appendix D for details). At

---

Published as a conference paper at ICLR 2026
the 504th iteration, the background is completely marked blue low-intensity regions, most of the lacewing area is highlighted in intense red, indicating high sensitivity. The transitional areas, includ- ing the lacewing's 's wings and boundaries with the background, to mnodrrate-intensity regions. This precise thermal grading not only accelerates perturbation opti- mization but also aligns with human perception, making highly interpretable.
marked in orange, corresponding
Similarly, in Figure the final heat regions converge on the bald eagle's head and left wing areas. In the targeted attack shown in Figure (c)), the heatmap concentrates primarily on the Minivan's wheel and door regions in the target image, which enables successful perturbations that misclassify the bald eagle as Minivan. Note that the interpretive patterns suggested by these heatmaps are secondary effects of the optimization process and should not be interpreted as complete explanation of model behavior.
5.4 FAILURE CASES AND WHEN SERI UNDER-PERFONE
While SeRI generally improves sicision-bbseed attack performance, its effectiveness can vary across different conditions. First, SeRI may under-perfore on images with weak or diffuse salient structure, such as cluttered rexture-dominated scenes. In these cases, regional sensitivity becomes less informative, and SeRI provides only modest gains over the base attacker. Second, SeRI requires moderate query budget reliably estimate regional importance. Under very low query budgets (e.g., fewer than 50 queries), the region partition becomes too coarse for meaningful refinement, leading to diminishing improvements. Third, similar all decision-basse attacks, SeRI assumes stable outputs from the target model. Therefore, it is not applicable to randomized or stochastic defenses that introduce noise into model predictions, since such randomness breaks the boundary- consistency assumptions required for reliable sensitivity estimation. Recognizing these limitations helps clarify when SeRT most effective and guides future extensions.
CONCLUSION AND FUTURE WORK
Sensitive regions in images play a crucial role determining adversarial vulnerability. However, un- der strict decision-based black-box settings, it is challenging to identify such regions using only top- model predictions. To address this problem, we introduced a novel definition of region sensitivity based perturbation decision boundaries. This formulation provides a principled and fin-grained way to characterize regional importance and is naturally suited for decision-basee attacks. Building on this definition, proposed SeRI, adaptively allocates perturbations across regions. Extensive experiments demonstrate that SeRT not only improves the attack performance of state-of-the-att decision-brut based methods but also produces heatmaps that reliably highlight sensitive image areas, offering clear interpretability benefits.
sensitivity-guider decision-based attack framework that
While our study focuses on image classification, the core idea of SeRI is general and can be extended to other vision tasks such as object detection and semantic segmentation. For multi-output models, SeRI can be adapted by replacing the current decision-boundary based perturbation comparison with score-based or multi-output perturbation comparisons. This allows SeRI to estimate region sensitivity by examining how perturbations affect task-specific outputs. Exploring these extensions, as well as studying SeRI under stronger or task-specific defenses, represents promising direction for future research.
ACKNOWLEDGMENTS
This research is supported by the China Scholarship Council (CSC, Grant No. 202506470044) and partially supported by the BUPT Excellent Ph.D. Students Foundation (Grant No. CX20241003). This research is supported by A Career Development Fund (Project No. C243512010). The authors thank the anonymous reviewers for their insightful and constructive comments earlier version of this manuscript.
10

---

Published as a conference paper at ICLR 2026
THE USE OF LARGE LANGUAGE MODELS
During the preparation of this manuscript, large language models (LLMs) were used sparingly to enhance grammar, clarity, and readability. All conceptual contributions, technical methods, analy- ses, and experimental results are entirely original and were developed solely by the authors. The authors have carefully verified the accuracy of all claims and take full responsibility for the content of this paper, in alignment with the ICLR Code of Ethics.
ETHICS STATEMENT
This research advances the field of Al security by developing efficient query-based black-box ad- versarial attacks to explore vulnerabilities in deep learning models. As with all security research, there is a potential for dual use, as these techniques could be misused to threaten Al system integrity. However, firmly believe that proactively uncovering and understanding these vulnerabilities is critical step towards building more robust and reliable AI systems. By addressing these weaknesses, this work contributes building more reliable and trustworthy AI, enhancing security across wide spectrum of applications. Furthermore, we emphasize that SeRI is designed solely for research into adversarial robustness, with the goal of advancing defenses and ensuring the safe deployment of machine learning systems.
REPRODUCIBILITY STATEMENT
We have made every effort to ensure the reproducibility of our results. The code is provided via the link in the Abstract well in the supplementary material. In Section 4 further provided the full algorithm pseudocode. In Section 5.1 detailed parameter settings and experiment environment are elucidated.
REFERENCES
Alexandre Araujo, Aaron Havens, Blaise Delattre, Alexandre Allauzen, and Bin Hu. A unified algebraic perspective on lipschitz neural networks. arXiv preprint arXiv:2303.03169, 2023.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision- -ased Adversarial Attacks: Reli- able Attacks Against Black-Box Machine Learning Models, 2018. arXiv:1712.04248 [cs, stat].
Thomas Brunner, Frederik Diehl, Michael Truong Le, and Alois Knoll. Guessing smart: Biased sampling for efficient black-box adversarial attacks. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 4958-4966, 2019.
Zikui Cai, Chengyu Song, Srikanth Krishnamurthy, Amit Noy-Chowdhury, and Salman Asif. Black- box attacks via surrogate ensemble search. Advances in Neural Information Processing Systems, 35:5348-5362. 2022.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy, pp. 39-57, 2017.
Jianbo Chen, Michael Jordan, and Martin J Wainwright. Hopskipjumpattakk: A query-efficient decision- desee attack. In 2020 IEEE Symposium Security and Privacy, pp. 1277-1294, 2020a.
Jinghui Chen and Quanquan Gu. Rays: A ray searching method for hard-label adversarial attack. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & 2020.
Data Mining, pp.
Pin- Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsich. Zoo: Zeroth order opti-
mization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop artificial intelligence and security, pp. 15-26, 2017.
Sizhe Chen, Zhengbao He, Chengjin Sun, Jie Yang, and Xiaolin Huang. Universal adversarial attack on attention and the resulting dataset damagenet. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(4):2188-2197 2020b.

---

Published as a conference paper at ICLR 2026
Minhao Cheng, Huan Zhang, Cho Jui Hsieh, Thong Le, Pin Yu Chen, and Jinfeng Yi. Query- hard-label black-box attack: An optimization based approach. In 7th International Conference Learning Representations, 2019.
Minhao Cheng, Simranjit Singh. Patrick Chen, Pin-Yu Chen, Sijia Liu, and Cho-Jui Hsieh. Sign- OPT: A Query-Efficient Hard-label Adversarial Attack, 2020. 19Xiv:1909.110 10773 [es, stat].
Zeyu Dai, Shengcai Liu, Qing Li, and Ke Tang. Saliency attack: Towards imperceptible black-box adversarial attack. ACM Transactions on Intelligent Systems and Technology, 14(3):1-20, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255, 2009.
Xiaoyi Dong, Jiangfan Han, Dongdong Chen, Jiayang Liu, Huanyu Bian, Zehua Ma. Hongsheng Li, Xiaogang Wang, Weiming Zhang, and Nenghai Yu. Robust superpixcl-guided attentional adversarial attack. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12895-12009 12904, 2020.
Yinpeng Dong, Hang Su, Baoyuan Wu, Zhifeng Li, Wei Liu, Tong Zhang, and Jun Zhu. Effi cient decision based -bx adversarial attacks on face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7714-7722, 2019.
Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.1192) 2020.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
Godfrey Harold Hardy and Edward Maitland Wright. An introduction to the theory of numbers. Oxford university press, 1979.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Andrew Hyas, Logan Engstrom, and Aleksander Madry. Prior convictions: Black-box adversarial attacks with bandits and priors. In International Conference on Learning Representations, 2019.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical Report, University of Toronto, 2009.
Huichen Li, Xiaojun Xu, Xiaolu Zhang, Shuang Yang, and Bo Li. Qeba: Query-efficient boundary- based blackbox attack. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1221-1230 2020.
Jie Li, Rongrong Ji, Peixian Chen, Baochang Zhang, Xiaopeng Hong, Ruixin Zhang, Shaoxin Li, Jilin Li, Feiyue Huang, and Yongjian Wu. Aha! adaptive history-driven attack for decision- based black-box models. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 16168-16177, 2021.
Chenhao Lin, Sicong Han, Jiongli Zhu, Qian Li, Chao Shen, Youwei Zhang, and Xiaohong Guan. Sensitive region-aware black-box adversarial attacks. Information Sciences, 137:118929, 2023. ISSN 00200255.
Yujia Liu, Seyed-Mohsen Moosavi-Dezfoole and Pascal Frossard. A geometry-inspired decision- based attack. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp.
4890-4898, 2019.
Giulio Lovisotto, Nicole Finnie, Mauricio Munoz, Chaithanya Kumar Mummadi, and Jan Hendrik Metzen. Give me your attention: Dot-produc attention considered harmful for adversarial patch robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), pp. 15234-15243 June 2022.
12

---

Published as a conference paper at ICLR 2026
Scott M Lundberg and Su-In Lee. A unified approach interpreting model predictions. Advances in neural information processing systems, 30, 2017.
Chen Ma, Xiangyu Guo, Li Chen, Jun-Hai Yong, and Yisen Wang. Finding optimal tangent points for reducing distortions of hard-label attacks. Advances in Neural Information Processing Sys- tems, 11119288--13000 2021.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
Thibault Maho, Teddy Furon, and Erwan Le Merrer. Surfree: a fast surrogate-fres freee black-box attack. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10430-10439, 2021.
Jeonghwan Park, Paul Miller, and Niall McLaughlin. Hard-label based small query black-box adver- sarial attack. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision ( (WACV), pp. 19886-9955222 2024.
Edward Raff, Jared Sylvester, Steven Forsyth, and Mark McLean. Barrage of random transforms for adversarially robust defense. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6521-6530, 2019.
Ali Rahmati, Seyed-Mohsen Moosavi-Dezfoolf Pascal Frossard, and Huaiyu Dai. Geoda: a geo- metric framework for black-box adversarial attacks. In Proceedings of the IEEF/CVF conference computer vision and pattern recognition, pp. 8446-8455 2020.
Md Farhamdur Reza, Ali Rahmati, Tianfu Wu, and Huaiyu Dai. CGBA: Curvature-aware geometric black-box attack. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 124-133, 2023.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017.
Yucheng Shi, Yahong Han, Yu-an Tan, and Xiaohui Kuang. Decision-basee black -box attack against vision transformers via watch-wise adversarial removal. Advances in Neural Information Process- ing Systems, 15:11121-, 2022.
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recogni- tion. 3rd International Conference on Learning Representations, 2015.
Xuxiang Sun, Gong Cheng, Hongda Li, Chunbo Lang, and Junwei Han. Stdatav2: Accessing efficient black-box stealing for adversarial attacks. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-17, 2024.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pp. 3319-3328, PMLR, 2017.
Guanhong Tao, Shengwei An, Siyuan Cheng, Guangyu Shen, and Xiangyu Zhang. Hard-label black- box universal adversarial patch attack. In 32nd USENIX Security Symposium (USENIX Security 23), pp. 697-714, 2023.
Yusuke Tsuzuku. Issei Sato, and Masashi Sugiyama. Lipschitz-margin training: Scalable certi- fication of perturbation invariance for deep neural networks. Advances in neural information processing systems, 31, 2018.
13

---

Published as a conference paper at ICLR 2026
Jie Wan, Jianhao Fu, Lijin Wang, and Ziqi Yang. Bounceattack: A query-efficient decision- based adversarial attack by bouncing into the wild. In 2024 IEEE Symposium on Security and Privacy (SP), pp. 1270-1286. IEEE, 2024.
Fan Wang, Mingwen Shao, Lingzhuang Meng, and Fukang Liu. Dual stage black-box adversarial attack against vision transformer. International Journal of Machine Learning and Cybernetics, pp. 1-12. 2024.
Feiyang Wang, Xingquan Zuo, Hai Huang, and Gang Chen. Adba: Approximation decision bound- ary approach for black-box adversarial attacks. In Proceedings of the AAAI Conference Artifi- cial Intelligence, pp. 7628-7636, 2025.
Xiaosen Wang, Zeliang Zhang, Kangheng Tong, Dihong Gong, Kun He, Zhifeng Li, and Wei Liu. Triangle attack: A query-efficient decision-based adversarial attack. In European conference computer vision, pp. 156-174, 2022.
Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. In International Conference on Machine Learning, pp. 36246-366666 36263, 2023.
Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International conference on machine learning, pp. 5286-5295. PMLR, 2018.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. In International Conference on Learming Representations, 2018.
Sergey Zagoruyko. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European conference on computer vision, pp. 818-833, Springer, 2014.
14

---

Published as a conference paper at ICLR 2026
A THEORETICAL ANALYSIS OF SERI
In this appendix, we provide a formal theoretical analysis of SeRI to analyze its convergence and op- timality properties. Let g(d) denote the d as defined in eq A smaller value of g(d) implies more effective adversarial perturbation.
corresponding to perturbation direction
A.1 ASSUMPTIONS
To analyze the convergence and optimality properties of SeRI, we introduce the following standard assumptions commonly used pacision-based attack theory.
Assumption (Local Lipschitzness of the decision boundary). all perturbations d1, d2 in neighborhood of the search trajectory:
There exist such that for
(9(d1) g(d2) < Ll|d1 d2l12-
Assumption (Bounded curvature of the decision boundary). Let A = | (x))l denote the local curvature of the classifier's decision boundary. There exists constant 0 such that S C in the neighborhood of interest.
Assumption (Deterministic hard-label output of the target model). The classifier is query- stable: for any input z, repeated queries to the black-box oracle always return the top-1 label, i.., f(2 is deterministic.
In particular, there is randomized defense mechanism such randomized smoothing, random input transformations (c.g., random translation, rotation, crop), or stochastic ensembling that would change the hard label between queries.
Lemma 1 (Stable ADBA comparison). Under Assumption the outcome of ADBA's comparison between any two perturbations di, d2 is deterministic: running ADBA multiple times with the same hyperparameters and query budget always yields the same ordering between g(d1) and g(d2).
Proof. ADBA is a deterministic algorithm once the sequence of oracle outputs fixed. Under Assumption 3, the hard-label oracle deterministic, the entire sequence of labels queried dur- ing ADBA is fixed. Therefore, the returned decision boundary estimates and the induced ordering between g(di) and g(d2) are also deterministic.
These assumptions are mild and standard: Assumption holds for ReLU, VIT. and CNN classifiers almost everywhere; Assumption 2 states that the decision boundary has locally bounded curvature; Assumption follows from the concentration guarantees of ADBA Wang et al. (205)).
Assumption 3 acknowledges that SeRI, like all decision-based black-box attacks, may degrade or even fail under randomized defenses such as random cropping, random resizing, randomized smoothing, stochastic input transformations. This is not limitation specific SeRI: in the decision-based setting, the attacker only has access to a single hard label per query, and it is fun- damentally impossible detect or compensate for stochastic transformations applied inside the black-box box oracle. Consequently, all existing decision-bar based attacks, including HSJA, CGBA, RayS, ADBA, and region-basse baselines such as PAR, are equally unable to guarantee stable compar- isons under such randomized defenses. Our assumption therefore reflects an inherent limitation of the decision-baat -base black-box threat model rather than a deficiency of SeRI itself.
A.2 SERI CAN GENERATE FINE-GRAINED SENSITIVITY HEATMAPS
Here, we demonstrate that our SeRI approach can produce fine-grained sensitivity heatmaps. Sup- pose SeRI performs I iterations to generate such heatmap. In each iteration, it either increases the perturbation strength within a selected region by factor k₁ = 1.8, or decreases by a factor reformulate the problem follows: Given any initial per- turbation pixel p E 1, with p + O, we aim to show that, after / iterations of multiplicative updates that either 1.8 or p <- 0.2 p. the perturbation can be adjusted to approach any target
l2 = 0.2. To analyze this process,
15

---

Published as a conference paper at ICLR 2026
value 8 E [-1,1]. This guarantees that SeRI can converge optimal perturbation configuration consistent with fine-graing one-grained sensitivity patterns.
Perturbation Adjustability of SeRI. Let Po [- {0} be initial perturbation value, and let
p" [-1,1] 1] be an arbitrary target value. Under the multiplicative update rule:
Pi+1 = 1.8 p. (increase),
(11)
0.2 p,
(decrease),
for any 0, there exists finite sequence of such updates such that:
1r 17 <.
(12)
Proof. We prove the theorem in two parts: (1) by showing that the ratio pr / can approximate any positive real number arbitrarily closely using multiplicative updates, and (2) by extending this to match the signed target value 8".
Step 1: Logarithmic Density of the Multiplicative Process. After I updates of comprising in- creases (x 1.8) and n = y decreases (x0.2), the perturbation becomes:
Pí po (1.8)" (0.2)"
(13)
Taking logarithms:
log Pr = log lñol m log 1.8 + log 0.2
(14)
Let log 1.8 and 3 = llog 0.2. Then:
Pr 1
log
ma + n23
(15)
Since a/8 & Q, the set of linear combinations {ma + n3 m. E Z.) is dense in IR (this is standard result from Diophantine approximation Hardy Wright (1979)).
Exponentiating both sides, the set (* (0.2)* m,n E Z} is dense in Thus, for any positive target ratio r = 1 ///ol and any 0, there exist integers m, n such that:
(1.8)) <
(16)
Step 2: Matching Sign and Closeness
Choose 8 e/ e/lpol. Then:
P0 (1.8)" 101 <
(17)
Now, since all multiplicative updates preserve the sign of Po. we ensure that:
If pt 0, use Po > If < 0, use Pu- < 0
This ensures the **sign of ğи matches that of 5***. Hence:
|5
<e <c
(18)
Since and are finite, the total number of operations I = is also finite. The corresponding update sequence can be constructed by applying m increases and n decreases in any order, due to the commutativity of multiplication.
16

---

Published as a conference paper at ICLR 2026
Conclusion. Therefore, for any 0, any initial po E [-1,1]] (0), and any target p 11, finite sequence of updates exists such that:
P5 501 <6 <6
(19)
A.3 MONOTONIC IMPROVEMENT OF SERI UPDATES
In each iteration SeRI constructs three candidate perturbations: {d', de d. where d and d' are obtained by scaling the perturbation on the selected region b' by factors < 1 k (followed by 22-norm match |dd||2) described in Algorithm [
Theorem (Monotonic Decision-Boundary Descent). Under Assumptions 1-3, the iterates of SeRI satisfy
g(d S g(d')
for all iterations i.
Moreover, suppose that the selected region be has nonzero sensitivity, i.e.,
dg(d') ad b*]
+ 0,
and that the scaling factors < < 1 are chosen sufficiently close to so that the corresponding perturbation changes remain local neighborhood of d' where a first-order descent direction exists. If, in addition, the ADBA tolerance is chosen sufficiently small and thus can successfully compare the decision boundary of d', d' and d', then the inequality is strict:
g(d < g(dd).
However, if the scaling factors are chosen too close to 1, then the perturbation updates in each refinement iteration become excessively small, which slows down the optimization process and reduces the overall attack efficiency. Therefore, the scaling factors should strike a balance: they must be close enough to 1 ensure that the updates stay within the local descent region, yet not so close that the algorithm makes negligible progress in each iteration. Based on the empirical results reported in Section C requirements.
we adopt = 0.8 and i 1.2 suitable choices that satisfy both
Proof. By Lemma and Assumption 3, the hard-label oracle deterministic, hence ADBA pro- duces a deterministic approximation ī(d) of the true decision boundary g(d) for each candidate perturbation. The stopping criterion ADBA implies that there exists tolerance > 0 such that lő(d) - g(d) < for all candidates considered in SeRI at iteration t.
Let d' be the current perturbation and let d'+1 denote the candidate selected by SeRI among {d', di, d') using ADBA's estimates. Then we have
g(d't1) S g(d'),
which implies
a < o(d"+1)+ S g(d')
+r +
g(d') 2r.
Thus, up to additive eπror of order т, the update is non-increasing.
We now show strict descent under the additional conditions. Let w² denote the perturbation direction that is nonzero only on the selected region b*, so that the effect of scaling on b' can be parameterized by a scalar a along u'. Consider the one-dimensional function
(a(a) = g(d oru)-
The assumption Og(d)) * 0 equivalent to (0) + By continuity of sof and the local Lipschitz property in Assumption 1, there exists og > 0 and a constant > 0 such that, along the descent direction, we have
(P(()) < (p:(0) for all E an))
17

---

Published as a conference paper at ICLR 2026
The multiplicative scaling region b" induces two perturbations d' and d' which can be written as = aguu
d' d' agu',
for some scalars ogg 0g determined by k and the subsequent normalization. By choosing k < à sufficiently close to 1, we can ensure that both logl and log| lie within (0, oo). i.., the two scaled candidates stay inside the local neighborhood where the first-order descent behaviour of yo: is valid.
Therefore, at least one of the scaled candidates, say dh {d,, d²). satisfies
g(d") pi(a') I p:(0) cla'l S g(d") -d, for some constant d' 0 that depends on and the chosen scaling factors. Choosing / yields g(d'),
g(d I g(dd) 25 s g(d - d'
which gives strict descent.
If no region has nonzero sensitivity, then (0) = 0 for all directions considered, and the candidates coincide up higher-order terms and numerical tolerance. In that case, the above argument shows that the update is non-increasing but not necessarily strictly decreasing.
Theorem establishes that SeRI is greedy descent method for minimizing the decision boundary distance. This formally justifies the empirical observation that SeRI consistently strengthens the attack.
A.4 CONVERGENCE OF THE ITERATIVE PERTURBATION PROCESS
We now show that SeRI converges to fixed point under the above assumptions.
Theorem 2 (Convergence Stationary Point). Assume Theorem 7 holds and that g(-) is lower bounded. Then the sequence {g(d")} converges, and any limit point d' satisfies:
0g(d") = 0 for all subregions b. ad[b]
Proof. Since g(d't1) g(d') and g is non-negative, the sequence is monotonically decreasing and bounded below. Thus converges. If limit point d' had nonzero partial derivative in some region b. the multiplicative update with sufficiently small scaling factor would strictly decrease g, contradicting convergence. Hence all directional derivatives vanish.
This establishes that SeRI converges to stationary perturbation under its multiplicative refine- ment mechanism.
A.5 CONSISTENCY THE REGION-SENSITIVITY ESTIMATION
Recall that SeRI recursively partitions current region into four equal sub-regions (Algorithm 2). producing a quad-tree hierarchy. Let S* denote the true optimal pixel-wise sensitivity map, and Tet Sş denote the diecewise-constane approximation obtained after & refinement iterations.
To formalize the convergence analysis, we first define the notion the cell size of region.
Definition (Cell size). For any region b C R², we define its cell size h(b) as the side length of the smallest axis-aligned square containing b. Equivalently, h(b) proportional to the diameter of up to constant factor:
diam(b) I <2)(
Under the recursive split used by SeRI, every refinement iteration reduces the cell size by half. If hy denotes the initial cell size (corresponding to the entire image domain), then the cell size after refinement levels is
hg 2- ho-
18

---

Published as a conference paper at ICLR 2026
Regularity assumption. We assume that S* is piecewise Hölder continuous of order > O, meaning that the image domain can be partitioned into finitely many subdomains {24} such that for each S24,
Ss (x) - (v)1 S C| - yll"; Yx, y E 24,
for some constant C' > O. This assumption allows S to have discontinuities across object bound- aries semantic edges, while maintaining smoothness within each piece.
Theorem 3 (Consistency of Region Refinement), Lei Sy. be the region-wise sensitivity estimate produced after k levels of quad-tree refinement. If S is piecewise Hölder continuous of order 0, then
| - 1 = 0(2 kor)
Sketch. Consider any region refinement level Since S₁. assigns a constant value b, the approximation error on this region satisfies
sup (x)) - (z)) < Ch = -Ch
=Eb
by the Hölder condition on each smooth piece. Summing over the quad-tree cells and noting that the number of cells grows only polynomially while the per-cell error decays as hg. we obtain
S 1 0 O(2-2ka))
which proves the stated 0(2 ke) convergence rate in the L norm.
This theorem shows that SeRI's region-refinemen strategy is provably consistent: as the quad-tree tree becomes finer, the region-wise sensitivity estimate S₁ converges to the true underlying sensitivity S In Appendix B. we further conduct an empirical comparison among several partitioning strate- gies (1 x 2, 2 2, 3 x 3, and 4 x 4). The results show that the 2 x 2 split provides the best trade-off between refinement granularity and query efficiency, while the other strategics produce comparable but slightly inferior results. This empirical evidence supports the thcoretical analysis by demonstrat- ing that recursive region refinement robust to the exact splitting scheme, and that the 2 x 2 strategy achieves both strong practical performance and favorable theoretical properties.
A.6 ROBUSTNESS UNDER MODEL VARIATIONS
Using Assumption 2, we show that ADBA remains robust even under boundary curvature. Proposition 1 (ADBA Robustness Under Bounded Curvature). Let d1, d2 be two candidate pertur-
bations. Under Assumption
10(d1) ((d2)) < -(112- - d2ll2) + Okk]|d1 d2ll2)
Thus, for perturbations differing only on a single region scaling (as in SeRI), the second-order term negligible, and ADBA's ranking of candidates is stable.
Sketch. A second-order Taylor expansion of the boundary distance along the perturbation direction yields the claim.
This establishes that SeRI is robust to model architectural variations, since curvature only affects second-order terms.
A.7 CONNECTION BETWEEN DECISION-BOUNDAN AND ATTACKABILITY
In this subsection we clarify why minimizing the decision- decision-boundary g(d) is closely aligned with improving the attackability of a model. Recall that for a fixed clean image and perturbation direction d, the decision-boundary distance is defined as
(dd) = min > 0: [(x + r d/[ld])2) = 1}:
that is, the smallest radius at which the perturbed point leaves the original class enters the target class (targeted attack).
attack)
19

---

Published as a conference paper at ICLR 2026
Attackability under an l2 norm budget. Suppose attacker is allowed to any perturbation with lg norm at most e > O. For a fixed direction d, the attack succeeds within this budget if and only if there exists r I such that [(x + r dd/[[]]2) 1, which is equivalent to g(d) E. Hence, for a fixed e we have
1 fattack succeeds within norm c} ([g(d) S Ee.
Consequently, among two perturbation directions d, and d2, if g(d1) < g(d2) then for any fixed norm budget
((g(d)) I < 2 1(0(d2) <}
This shows that smaller decision-boundary is strictly more favorable for achieving successful attacks under the same l2 constraint: any budget for which d2 can succeed, di can also succeed, while the converse is not true.
Attackability under random perturbation magnitudes. The above argument can be generalized to randomized attack procedures. Let be a nen-negative random variable describing the perturba- tion magnitude produced by some attack algorithm along direction d (for example, the final radius found by a boundary search under a fixed query budget). The attack succeeds if R Z g(d). Denot- ing the cumulative distribution function of R by FR(r)= = Pr(R S r), the success probability for direction is
Pr(success d) Pr(R g(d) = Fx(g(d)).
If FR is strictly increasing, then for any two directions d, and d2
g(dg) < g(22)
1 1 F=((() > 1 - FR(g(d2)),
which means that a smaller seccision-boundan distance implies a larger success probability under the same attack procedure and query budget. Therefore, minimizing g(d) is equivalent to maximizing the attack success probability for any fixed distribution of perturbation magnitudes.
Empirical correlation between g(d) and attackability. To complement this theoretical argu- ment, we conduct an additional experiment where, for each test image, we record the estimated decision-boundary distance ((d) produced by ADBA and the corresponding attack outcome (suc- cess or failure) under a fixed norm threshold report the empirical attack success rate in each bin. The results, shown in Figure A.7, display clear monotone trend: bins with smaller ğ(d) exhibit substantially higher attack success rates, while bins with larger g(d) have markedly lower success rates. This empirical evidence supports our the- oretical analysis, and confirms that SeRI's objective of reducing the approximate decision-boundaly distance directly translates into higher attackability under practical query budgets.
We then group samples by quantiles of g(d) and
B PSEUDOCODE OF SERI
In Algorithm| SeRI iteratively enhances reduces the perturbation strength within the selected sub-region to identify whether the selected sub-region is sensitive to perturbation, and then splits the sub-region into four smaller parts. Splitting each region into four parts offers the best trade-off between precision and efficiency. As shown in both our theoretical analysis and the experimental results in Appendix B.T. this strategy achieves superior performance compared to alternative parti- tioning schemes. The initial region by has shape of C' x W x H. Algorithm Spplis bo into four smaller sub-regions, forming the initial sub-region set (see Appendix[B.. for details).
In line 2 of Algorithm D at each iteration, the sub-region 1 with the hightest fy-norm perturba- tion is selected. Then in lines 3-5, SeRI generates two new perturbations, di and d', by respectively reducing and enhancing the perturbation within sub-region b*. The reduction and amplification mul- tipliers are denoted by 1 and i, respectively. In line 5, the perturbations d' and d' are normalized to match the overall perturbation strength of d', ensuring a fair comparison among d' d' and d' as defined in Eq. [ In line 6, we use ADBA WWan et al. (2025) to compare the decision bound- aries of d' and d' to determine the best perturbation (see Algorithm 3 in Appendix B for details).
Specifically, Algorithm takes as input the current best perturbation d. along with its approximate decision boundaries Rmin and Rmmx (lower and upper bounds of the true decision boundary), well a candidate perturbation d'. The algorithm then outputs the superior perturbation d, together
20

---

Published as a conference paper at ICLR 2026
Algorithm Perturbation Optimization via Gradient-Frees Sensitivity Region Identification
Input: Original image z, initial adversarial example &", indicator I(.). query budget Q;
Output: Adversarial sensitive region heatmap 8, optimized adversarial example I;;
Initialization: Iteration number +- 1, initial perturbation d² + z - I, approximation decision
boundary Rmin < 0, and Rmax 1: initial region by -- {1 C, : W, H), sub-region set
Algorithm (bo): perturbation multipliers k + 0.2, and +- 1.8;
1: while remaining query budget >> do
2: br arg max ||d]s] l2
bEB
3:
de dede +
d.d''ddccoppl dd'.copp(,cd"" (rpp() .copy()
4:
d' o), d'16) + dop k- dio
d' d. ||d' d:
|| 1 lla d:
5:
| ll2
6:
dg: mmnn Rmax + A Algorithm (d', Rmiin Rmaax d')
7:
if d, d' then
8:
d++ +
9:
else
10:
ds. Rminy Rmax + Algorithm 3 (d', Rminy Reuxx d)
11:
if d, d³ then
12:
d'+1 t di
13:
else
14:
d'+1 d'
15:
end if
16:
end if
17:
B - (B), JuAlgorrro
it i+1
19:
S 2 d' 11 se,wc di //.. w,w
20:
x-x+ -- Penux d'
21:
end while
return S,
Algorithm 2 Split Perturbation Into Four sub- sub-regions
Input: Initial region by <- [1 C, #2, V1 : y2);
Output: sub-region set of bu:
I: Fmid f- 21 Ymid < #32
2: bi < :C:Ca1: Tmid: Mmid
3: 62 < {1: C, + 1: x2 8/1 ymidd]
4: by <- {1: C, Emid, Umid 1 3/2
5: by < {1: : C, Fmid + 1 22 Manid + 1 1/2
6: return {bi, by, by, ba}
with updated boundary estimates Rmin and Rmax- d outperforms d', as determined in line 7-8, this indicates that perturbations within region b* do not contribute positively the attack's success. Hence di is deemed the best perturbation among d', d' and d', and the updated perturbation di is set to d'. In line 10, if d' performs better than d', is further compared with d' using Algorithm 3. In lines 1 11-15, d++1 1 is updated to match the better perturbation between d' and d', based on their decision boundaries. In line 17, the selected region is removed from the sub-region set and replaced with its four child sub-regions generated by Algorithm 2 In line 19, the heatmap S is up- dated by performing element-wii wise division between and d'. Subsequently in line 20, the current adversarial example ă is updated. In lines 21-22, if the query budget is reached, the algorithm stops and returns the sensitive region heatmap S together with the optimized adversarial example ă as the final output.
21

---

Published as a conference paper at ICLR 2026
Table 4: Average (median) l norms for non-targeted attacks on ImageNet using a VGG model.
Total Query
2,000
5,000
10,000
1x2 split
3.292(1.429)
1.630(0.811)
1.024(0.569)
2x2 split
1.020(0.562) (.587)).771) 3.234(1.385)
3x3 split
3.290(1.410)
1.029(0.569) 1.645(0.800)
1.598(0.776)
4x4 split
3.311(1.444)
1.040(0.570)
B. SPLIT SELECTED REGION INTO FOUR SUB-REGIONS
To capture adversarial perturbation sensitivity at a fine granularity, the current region bo is divided into four sub-regions after each iteration. The detailed procedure is shown in Algorithm 2 In line 1, Imid and Vmid denote the horizontal and vertical midpoints of the current region by. Lines 2-5 generate four sub-regions 61 b2, b3, by based on these midpoints. Line 6 returns the set of newly created sub-regions.
To determine the optimal number of sub-regions per split, we conducted experimental compar- ison of 2x2 split against three alternatives: 1x2 binary split (direction chosen by aspect ratio), 3x3 split, and 4x4 split. The evaluation was performed on 100 images from the ImageNet dataset using a VGG model. We report the average (median) &2 perturbation norms achieved by the "CGBA+SeRI" attack. The results. presented in Table 4 demonstrate that the 2x2 split yields the best performance across 2,000, 5,000, and 10,000 total queries. However, the performance of other split settings is comparable, their norms are only marginally higher (by approximately 3%). This indicates that the overall performance of SeRT is not highly sensitive to the exact number of splits.
B.2 COMPARE TWo PERTURBATIONS USING APPROXTMATION DECISION BOUNDARY
To compare perturbations effectively in decision-based attack setting, we follow the cutting-edge Approximate Decision Boundary Approach (ADBA) recently proposed in WWag et al. (2025). ADBA avoids the need to precisely compute decision boundaries with high query cost. Instead, compares two perturbations by identifying a perturbation strength where one successfully fools the model but the other fails. This indicates that the successful perturbation has smaller decision boundary and is more effective.
The main idea of ADBA is that, it is unnecessary to precisely identify the decision boundaries of two perturbations to compare them. Instead, if we can identify an Approximate Decision Boundary (ADB), such that at this perturbation strength, one perturbation d, successfully fools the model while another direction d2 fails, then we can infer that the decision boundary d1, g(dr), is smaller than that of d2, i.e., g(di) S ADB < g(d2)). This implies that dj outperform d2.
The procedure is summarized in Algorithm 3 In this algorithm, Rmax and Rmin represent the upper and lower bounds of the ADB for dj. Specifically, d1 successfully fools the model perturbation strength Amxx (i.c., [(x + Rmax d1) = 1, indicating g g(d1) & Rmax), but fails to fool the model at perturbation strength Rmin (i.e., Txx Rmiin di
indicating Rmin < g(d))))) ))). In lines 1-3, if perturbation d2 fails to fool the model at ADB R it indicates that its true decision boundary is greater than Rmax. i.c., g(di) S Rmax g(d2). In this case, d2 is less effective than di and Algo- rithm retuur di as the superior perturbation. Otherwise, in lines 4-6, if perturbation d2 successfully fools the model at ADB Rmins it indicates that its true decision boundary is smaller than Rmin. i.e., g(d2) < Rmin < (. This implies that d2 is more effective than d1, and Algorithm 3 returns d2 as the superior perturbation. In lines 7-13. if both di and d2 either succeed or fail the current approximation R. then Rmax or Rmin are updated accordingly to narrow the search interval for R. In lines 14-17, the current value of leads successful attack for one perturbation but not the other, the successful perturbation is returned along with the updated ADBs, Rmin and Rmax- These updated bounds are carried into the next iteration to narrow the search range and reduce the number of queries required in lines 1-6 of Algorithm
Finally, in line 20, if the search interval Rmsx - Rmiin becomes smaller than a search tolerance thresh- old 7, indicating that g(d1) and ((2)) nearly equivalent, it becomes unnecessary to distinguish between them. Hence, the algorithm directly returns d2 along with the current Rmin and Rm Returning di produces similar results in our experiments.
22

---

Published as a conference paper at ICLR 2026
Algorithm 3 Compare Two Perturbations Using Approximation Decision Boudnary
Input:
Current best perturbation d1 with approximation decision boundaries Rmins Rmaxi candidate perturbation d2, and search tolerance = 10-6;
Output: Superior perturbation d, with updated approximation decision boundary Ruins
R
1: if [x + Rmmxx d2) then
2: return d1, Rminn Rmax
3: end if
4: if [xx + Rmin d2) = then
5: return d2, 0, Rmin
6: end if
7:
while Rmax Rmin do
8:
if [(x + R d2) = and [(x + R- d;) = then
9:
R (Rmin + RRmax)/2
10:
Rmax f- R
11:
else if' [x + R d2) = and (xx + R d1) = then
12:
R 4- (Riin + Rmax)
13:
Rmin t R
14:
else if I R d2) = and I(x R d1) 0 then
15:
Rmax 5 R
return d2, Rmin: Rmax
17: else if [xx + R dg) = and 1(x + R d,) = then
Raaxx R
19:
return d1, Rmins Rmax
end if
21:
end while
22: return d2, Rmins Rmux
C PARAMETER SENSITIVITY ANALYSIS
Our SeRI approach summarized in Algorithm introduces several parameters, including the thresh- olds & 0.2 and & 1.8, as well as the SeRI query budget percentage p 20%. These parameter settings are based on preliminary experiments with the ImageNet-VGG19 model. Similar results have been observed across other datasets and models.
Table -orm of perturbation for varying ScRI hyperparameters (k, k) under 5,000 query budget.
1.7
1.8
k = 1.9
k k= 2.0
K 0.05
1.261
1.258
1.257
1.262
& 0.10
1.258
1.251
1.251
1.259
& = 0.15
1.251
1.247
1.260
1.249
À = 0.20
1.243
1.239
1.246
1,255
F = 0.25
1.258
1.256
1.251
1.265
Table 6: f--norm of perturbation for varying SeRI hyperparameter under different query budget.
Total Query
2,000
5,000
10,000
20,000
P 10%
3.475
1.330
0.857
0.700
P 20%
3.112
1.259
0.823
0.670
P 30%
3.304
1.480
0.873
0.747
P = 40%
3.332
1.719
0,994
0.819
P 100%
4.174
2.826
2.237
1.914
To evaluate the impact of different parameter settings, conduct sensitivity analysis. Specifically, vary the threshold and 5 and the percentage P (Table 6 across range of values. For
23

---

Published as a conference paper at ICLR 2026
7
V%CCBA+100%55RRI
20%CCBAA+8%%SeRI
40%CCBA*+0%%SeRI
60%/CCBB+44%555RRI
80%.CBBA+2%%5eRI
100%CGBA+0%SSRI
Average l,, -norm
PAR
3
2
1000 2000 3000 4000 5000
6000 7000 8000 9000 10000
Queries
Figure 3: Decrease of the perturbation &g norm under different P settings of SeRI the ImageNet- VGG model.
each setting, attack performance is assessed using the fz-norm of the final perturbation. The total query budget is set 5,000 for threshold & and &. Regarding P, we evaluate the attack under query budgets of 2,000. 5,000, 10,000, and 20,000. This analysis examines whether the optimal choice of P' depends on the total query budget. A lower f2-norm indicates more effective attack.
We conduct experiments on the ImageNet dataset using VGG19 model for non-targeted attack. This model have been introduced in Subsection 4.1. The results presented in Tables and with the best values highlighted in bold.
Our experimental results demonstrate that the parameter combination (f- 0.2, k 1.8) and P 20% consistently yields the best performance. As shown in Table achieves the lowest & feperturbation of 1.239. Additionally, Table confirms that 20% results in the lowest e-perturration across all four query budget settings.
5 the setting (h: 0.2, 1.8)
24

---

Published as a conference paper at ICLR 2026
We also plot the decrease of the perturbation fy-norm under different P settings of SeRI on the ImageNet- VGG model ().g From the figure, we observe that when the query budget is small (before approximately 1900 queries), the setting P 0% achieves the lowest mo-nmmm However, as the query budget increases, the performance of P = 0% becomes significantly worse. This is expected because P 0% corresponds to initializing SeRI with random perturbation, which allows for a rapid early reduction of the fg-norm but provides no meaningful optimization direction for later refinement.
For P = 20%, 40%, 60%, and 80%, we observe a noticeable "jump" in the f2-norm when SeRI begins operating. indicating that SeRI can effectively refine and further optimize the perturbation produced by the base attacker.
The setting P 100% corresponds using the base attacker alone (CGBA) with no ScRI refine- ment. In this case, no second-stage improvement occurs, and thus the 2--norm does not decrease significantly at 10,000 queries.
D COMPARISON OF HEATMAPS GENERATED BY SERI, PAR, LIME, AND SHAP
4 provides a qualitative comparison of seven input images alongside the corresponding heatmaps and perturbation produced by SeRT and PAR. In this experiment, we adopt CGBA as the base attacker and set the total query budget to 10,000. As illustrated in the figure, the proposed SeRI method yields significantly more informative and concentrated heatmaps than PAR, leading to both improved interpretability and attack quality. The advantages of SeRI can be summarized in two major aspects:
1. Semantically concentrated perturbations. SeRI naturally guides perturbations toward semanti- cally meaningful object regions (e-g.. dog's head or a whale's tail), while PAR often generates artifacts scattered across background areas. These background perturbations are visually disruptive and lack semantic relevance. By effectively suppressing such noise, SeRT achieves substantially reduced perturbation strength and produces adversarial examples that are less perceptible to human observers.
2. Continuous and fine-grained saliency modeling. SeRI provides continuous-valued regional im- portance estimates: highly influential areas are highlighted in dark red, moderately relevant areas appear in orange or yellow, and unimportant background regions represented in blue. This con- tinuous sensitivity landscape more faithfully reflects the underlying structure of the model's decision surface. By contrast, PAR applies binary patch-retention mechanism, either preserving or remov- ing an entire patch, thus failing to capture nuanced differences in regional contributions and often hindering the optimization performance.
Furthermore, the heatmaps generated by SeRI enhance the interpretability of the attack, exhibit- ing strong consistency with those produced by classical explainable AI (XAI) methods. Figure compares the heatmaps generated by SeRI, PAR, LIME, and Kernel SHAP.
LIME model-agnostic local explanation method based on superpixel perturbation and sparse regression. In our implementation, LIME is configured with 1,000 perturbation samples and uses superpixel boundaries (yellow contours) to highlight high-importance regions.
Kernel-based SHAP is the black-box variant of SHAP, requiring only model output scores rather than gradients. It estimates Shapley values by solving locally weighted linear regression. In our experiments, Kernel SHAP is configured with 50 superpixels (SLIC segmentation), 300 sampled coalition evaluations. Regions important to the model are visualized in red, moderately relevant areas in yellow/green, and unimportant areas in blue.
To quantify the agreement between our sensitivity maps and classical XAI methods, we compute the Pearson correlation coefficient (PCC) between the heatmaps produced by SeRI/PAR and those generated by LIME and SHAP. As shown in Table7] SeRI achieves PCC scores of 0.723 with LIME and 0.786 with SHAP values commonly interpreted as indicating strong correlation. In comparison, PAR exhibits noticeably lower correlations (0.610 10 with LIME and 0.637 with SHAP). These results suggest that SeRI captures more consistent and meaningful saliency structure than PAR, and aligns
25

---

Published as a conference paper at ICLR 2026
The heatmaps and perturbations generated by SeRI
The heatmaps and perturbations generated by PAR
The input images
Figure Heatmaps and perturbations generated by SeRI and PAR.
more closely with the explanations provided by established XAI methods, despite relying only hard-label queries.
This quantitative evidence is further supported by the qualitative comparisons in Figure5) For the first-row dog example, both LIME and SHAP assign the highest importance to the dog's head, and SeRI similarly concentrates its heatmap this region. A similar pattern is observed in the
26

---

Published as a conference paper at ICLR 2026
The heatmaps generated by LIME generated by SHAP
Thee heatmaps
The heatmaps
The heatmaps generated by PAR
The Input images
generated by SeRI
Figure Heatmaps generated by LIME, SHAP, SeRI, and PAR.
second-row bird image: LIME and SHAP highlight the bird's torso, wings, and legs, and SeRI again identifies these same regions with high saliency.
Interestingly, the third-row whale-tail example reveals and SHAP partially capture the whale's tail but fail to fully localize it, SeRI successfully places nearly all of its saliency the tail, closely matching the true decision-critical region. This high-
clearer distinction. While LIME
27

---

Published as a conference paper at ICLR 2026
Table 7: Average Pearson Correlation Coefficient (PCC) between the heatmaps produced by
SeRI/PAR and those from LIME and SHAP.
Method PCC w/ LIME
PCC w/ SHAP
SeRI PAR
0.723
0.786
0.610
0.637
lights SeRI's ability more accurately approximate the model-dependent sensitivity landscape than existing black-box XAI baselines.
Overall, both the quantitative PCC analysis and qualitative heatmap comparisons indicate that SeRI not only delivers more semantically accurate and interpretable region sensitivity estimates than PAR, but also maintains strong alignment with classical XAI methods such LIME and SHAP. These results further validate the reliability and effectiveness of SeRI's boundary-driven sensitivity estima- tion framework.
E COMPARE ATTACK SUCCESS RATE (ASR), STRUCTURAL SIMILARITY INDEX (SSIM), AND STANDARD DIVISION OF l2 N2-N (62-STD)
To further verify that SeRI provides advantages beyond fn-n reduction, we conduct additional experiments three complementary metrics: (1) Attack Success Rate (ASR), (2) Structural Simi- larity Index (SSIM), and (3) Standard Deviation of f--norm (G2-STD). These metrics jointly capture attackability, perceptual imperceptibility, and optimization stability.
We evaluate on ImageNet, CIFAR-10O, 100, and MNIST datasets with diverse model architectures: ResNet50, Inception-và v3, VGG19, VIT. WideResNet (WRN), Engstrom Wong & Kolter (0018) 8), and Lipschitz Tsuzuku et al. (2018). For each model, we randomly sample 500 images from the test set. We adopt untargeted attacks with query budget of 10,000 queries. Perturbation thresholds are set to c = 2.5 on ImageNet, c = 1.0 on CIFAR-100, and 3.0 on MNIST. We evaluate two base attackers, CGBA and ADBA, optionally enhanced with PAR SeRI.
Table 8: Attack success rate (ASR) of SeRI measured under a query budget of 10,000.
Imagenet ResNNt50
Imagenet -Inception
Imagenet -GG119
Imagenet -Fngstrom
CIFARI00
CIFARIO0
MNIST
-VIT
-WRN
-Lipschitz
CGBA
49.8%
77.6%
86.4%
28.4%
79.4%
55.6%
76.6%
CGBA+PAR
56.0%
82, 92.8%
87.2%
38.8%
84.6%
58.4%
82.8%
CGBA+SeRI
64.6%
86 96.8%
91.0%
46.8%
87 87.2%
61.8%
88.0%
ADBA
16.0%
58.6%
61.2%
18.0%
36.0%
27.2%
51.2%
ADBA+PAR
52.6%
62.8%
71.0%
41.0%
47.8%
36.6%
72.6%
ADBA+SeRI
57.8%
66.2%
79.4%
49.8%
58.0%
43.8%
87.0%
Table 9: Average Structural Similarity Index (SSIM) of SeRI on the ImageNet dataset, measured under query budget of 10,000.
Imagenet ResNNe550
Imagenet -Inception
Imagenet
Imagenet -Engstrom
CIFARI00
CIFARI00 -WRN
MNIST
-VGG19
-ViT
Lipschitz
CGBA
0.959
0.982
0.994
0.903
0.996
0.964
0.449
CGBA+PAR
0.963
0.985
0.996
0.940
0.998
0.971
0.751
CGBA+SeRI
0,964
0.986
0.996
0.946
0.998
0.973
0.786
ADBA
0,962
0.972
0.982
0.903
0.981
0.910
0.445
ADBA+PAR
0.966
0.977
0.983
0.951
0.985
0.930
0.723
ADBA+SeRI 0.967
0.979
0.984
0,958
0.987
0.933
0.745
The results in Tables 8 9 and 10 collectively highlight the clear and consistent advantages brought by SeRI across all datasets and model architectures. Most notably, SeRI provides substantial improvement in attackability, achieving the highest ASR in every tested setting and outperforming both base attacker and PAR by large margins. This demonstrates that SeRI's sensitivity-guides
28

---

Published as a conference paper at ICLR 2026
Table 10: STD of mo-norm under a query budget of 10,000.
Imagenet
Imagenet Inception
Imagenet -VGG19
Imagenet -Engstrom
CIFARIOO
CIFARIOU
MNIST
ResNet50
-VIT
-WRN
-Liisccitz
CGBA
7.404
4.297
1.318
16.99
0.484
1.001
3.214
CGBA+PAR
5.972
3.661
1.265
4.891
0.432
0.933
1.997
CGBA+SeRI
4.636
2.398
1.124
3.628
0.390
0.853
0,939
ADBA
3.648
3.700
2.294
7.508
1.321
1.902
1.034
ADBA+PAR
3.211
3.505
2.007
4.659
1.165
1.451
ADBA+SeRI
2.789
3.448
1.856
3.653
1.032
1.126
0.738
refinement is highly effective in driving perturbations toward the true decision boundary even under strict query constraints.
At the same time, SeRI preserves exceptional perceptual quality, achieving SSIM scores that match or exceed both the base attackers and PAR, and often approaching values near 1.0, indicating that SeRI generates adversarial examples that remain visually indistinguishable from clean images.
Furthermore, SeRI consistently achieves the lowest 22DSTD, revealing that its continuous region- aware optimization yields far more stable and reliable perturbation magnitudes across images. Im- portantly, these improvements occur simultaneously: higher ASR, better perceptual quality, and greater optimization stability, demonstrating that SeRI offers a comprehensive enhancement rather than offe Overall, the results confirm that SeRI is robust and broadly effective refinement module that significantly strengthens decision-basee based black-box attacks across diverse architectures and evaluation metrics, establishing it as a strong and practical advancement for the field.
29


Reviews:
[{"review_id": "QsbCPLHBHr", "reviewer": "Reviewer_LU14", "summary": "This paper proposes a method for generating sensitivity regions (or heatmaps) in the context of black-box adversarial attacks. In essence, the authors extend the previous PAR (Patch-wise Adversarial Removal) method by transforming it from a discrete, patch-based approach into a continuous sensitivity estimation framework. This allows the model to more precisely identify which regions of an image are most influential to the prediction, thereby improving the efficiency and interpretability of black-box attacks.", "questions": "1. Have you measured attack success rate (ASR) or other metrics that could reflect the effectiveness of SeRI?\n\n2.Can you also compare SeRI with explainable AI (XAI) methods that produce sensitivity or saliency maps?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "The paper is clearly written and easy to follow. The authors provide a well-organized and thorough summary of related work, which helps the reader understand the motivation and context of their approach. Overall, the proposed SeRI framework, when combined with existing black-box attack methods, appears to perform well in terms of L2-norm improvements. The experimental results suggest that SeRI can enhance the efficiency of existing black-box attacks by producing smaller perturbations under the same query budget", "weaknesses": "The main contribution of the paper appears somewhat weak for two reasons.\n\n(1) The idea of constructing sensitivity region maps has already been introduced by the PAR method. Therefore, the novelty of this paper mainly lies in extending PAR from a discrete, patch-based formulation to a continuous sensitivity estimation framework. Given this, the additional components of SeRI—such as the use of ADBA—should provide more original or technically innovative ideas. Moreover, it seems that the core novelty of the work resides in the appendix, which feels somewhat misplaced.\n\n(2) Alternatively, the paper could strengthen its contribution through more comprehensive and convincing experimental evidence demonstrating a general improvement in black-box attacks. Currently, the results primarily focus on improvements in the L2 norm. Including evaluations based on attack success rate (ASR) or other complementary metrics that more clearly demonstrate the practical advantages of SeRI would make the contribution more substantial.\n\n(A list of specific questions will be provided in the Question section.)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "h0izdjlfJA", "reviewer": "Reviewer_tyNQ", "summary": "The paper presents a novel method, SeRI, for identifying sensitive regions in decision-based black-box adversarial attacks. The approach is innovative in its use of a continuous sensitivity score and recursive region partitioning, and it demonstrates strong empirical performance across multiple datasets and models. The work is practically useful and addresses a relevant problem in adversarial machine learning. However, while the technical contribution is clear, the academic rigor and scholarly depth of the paper could be significantly improved to meet the standards of a top-tier conference.", "questions": "None", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "Novelty: The idea of using a continuous sensitivity score in a decision-based setting is novel and represents a meaningful advance over binary region-selection methods like PAR.\n\nEmpirical Validation: Extensive experiments on ImageNet and CIFAR-100 with multiple models and attack configurations demonstrate the effectiveness of SeRI.\n\nPracticality: The method is query-efficient and can be integrated as a plug-in module with existing attacks, which is a valuable feature for real-world scenarios.\n\nInterpretability: The generated heatmaps provide visual explanations of sensitive regions, enhancing the interpretability of the attack process.", "weaknesses": "1. Lack of Theoretical Depth\nWhile the method is motivated intuitively and empirically validated, the theoretical foundation is underdeveloped. The proof in Appendix A, while correct, is limited to a simplified multiplicative update model and does not fully justify the convergence or optimality of the overall algorithm.\n\nThe paper would benefit from a more formal analysis of the convergence properties, sensitivity of the partitioning strategy, or robustness to model variations.\n\n2. Limited Comparison with Related Work\nThe comparison with PAR is thorough, but the paper does not sufficiently situate SeRI within the broader literature on interpretability methods (e.g., Grad-CAM, LIME, SHAP) or other region-based adversarial strategies beyond PAR.\n\nA deeper discussion of how SeRI relates to saliency detection or feature attribution methods would strengthen the scholarly contribution.\n\n3. Methodological Simplicity\nThe core algorithm—iterative partitioning and perturbation scaling—is conceptually straightforward. While this is a strength in terms of usability, it may also be perceived as lacking in algorithmic sophistication compared to some recent adversarial attack methods.\n\nThe paper does not explore alternative sensitivity formulations or adaptive partitioning strategies, which could have added depth.\n\n4. Evaluation Metrics and Generalizability\nThe evaluation is primarily based on l2 norm reduction and attack success rate. Additional metrics such as human perceptual studies, transferability, or robustness to defenses would provide a more comprehensive assessment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HcF6ZIrglu", "reviewer": "Reviewer_YrJv", "summary": "This paper focuses on black-box adversarial attacks where the attacker has access only to the model’s predicted class labels. The goal is to find a perturbation with the minimal norm that leads the model to misclassify. The proposed method, Sensitive Region Identification (SeRI), recursively divides the input image into four square regions and prioritizes further subdivision of regions with larger perturbation norms, thereby achieving smaller overall perturbation norms. SeRI also facilitates the generation of heatmaps that indicate regions where the model is more sensitive to perturbations. The effectiveness of the proposed method is demonstrated through experiments on benchmark datasets.", "questions": "- What are the main reasons SeRI is expected to outperform other saliency estimation methods (excluding PAR)?\n\n- Does SeRI maintain its advantage when applied to clean models trained on CIFAR100 and robust models trained on ImageNet?\n\n- Could the authors provide approximate numbers of queries required for (a) heatmap generation, (b) achieving performance gains over other saliency-based methods, and (c) the point at which the perturbation norm saturates?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The recursive partitioning of perturbation regions enables SeRI to produce more fine-grained heatmaps and achieve smaller perturbation norms compared to existing patch-based sensitive region-based attacks.\n\n- Experimental results on ImageNet and CIFAR100 show that combining SeRI with existing methods generally outperforms combinations with PAR, supporting the superiority of SeRI.\n\n- The paper is overall well-structured and easy to follow, with the exception of a few pseudocode details.", "weaknesses": "- The discussion of prior work on sensitive region-based attacks is limited to only one study. The paper should also discuss and compare related methods such as HeadBeat [1] and Saliency Attack [2], which—despite differences such as the presence or absence of surrogate models—address similar goals.\n\n- Experiments on defense methods are limited. The evaluation does not include ImageNet-based experiments and only considers adversarial training (AT) as a defense. When focusing on $\\ell_2$ -norm attacks, there exist more suitable defense methods based on Lipschitz constants of the network for comparison [3,4], which is expected to be more robust.\n\n- The paper does not analyze how perturbation norms saturate with respect to computational resources. Providing insights into the number of queries required to construct the heatmap, and identifying how many queries make each method effective, would be valuable for readers.\n\n[1] G. Tao et al., Hard-label Black-box Universal Adversarial Patch Attack, 2023  \n[2] Z. Dai et al., Saliency Attack: Towards Imperceptible Black-box Adversarial Attack, 2023  \n[3] Y. Tsuzuku et al., Lipschitz-margin training: Scalable certification of perturbation invariance for deep neural networks, 2018  \n[4] A. Araujo et al., A Unified Algebraic Perspective on Lipschitz Neural Networks, 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gpaihu1fWn", "reviewer": "Reviewer_kXVd", "summary": "This paper proposes Sensitive Region Identification (SeRI) for query-efficient decision-based adversarial attack. \nIt iteratively identifies sensitive regions through recursive splitting, and enhances the adversarial perturbation by scaling sub-regions based on decision boundary of the victim model. \nThe key claim is that the authors redefined the term 'sensitive region' in terms of continuous score, improving query-efficiency and interpretability. \nSeRI is easily plugged into existing adversarial attack methods such as HSJA, CGBA, RayS, and ADBA, consistently improving l2 perturbation norms under restricted query budget.", "questions": "1.  The proof in Appendix A shows that multiplicative scaling can approximate any target magnitude, but does this imply convergence of the overall iterative optimization process under query noise or ADBA comparison errors? If not, what assumptions are required for such convergence to hold? \n2.  How does the proposed decision-boundary–based sensitivity differ conceptually from gradient or Jacobian-based sensitivity in white-box settings? Can it be shown that SeRI approximates those measures under some assumptions? \n3.  Could you provide a more detailed analysis of the query budget ratio $P$? Specifically, how does SeRI perform under extreme cases (base = 100%, SeRI = 0%, and vice versa)? Was 20% empirically optimal across all models or tuned per dataset? \n4.  Have you tried SeRI on non-classification tasks (e.g., segmentation, detection, or long-tailed settings)? If not, do you expect any fundamental obstacles? \n5.  How much wall-clock overhead does SeRI add per query compared to the base attacker? Is the improvement in query count offset by additional computation time? \n6.  Have you measured any quantitative correlation between SeRI’s heatmaps and gradient-based saliency maps?  If not, how do you justify the claim that the generated heatmaps improve interpretability?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Well-motivated problem statement. The paper addresses a legitimate and well-defined challenge in decision-based black-box attacks - identifying sensitive image regions when only top-1 labels are available under strict query budgets.\n2. Clear mathematical formulation. The problem is formally and coherently defined through a continuous sensitivity optimization framework (Eq. 5), with explicit constraints and variables, making the proposed objective easy to follow.\n3. Method is intuitively connected to the problem. The proposed iterative region-splitting and perturbation-scaling mechanism directly reflects the intuition that different image areas contribute unequally to model decisions, leading to a natural algorithmic design.\n4. Mathematical convergence argument. Although limited in scope, the paper provides a formal statement (Appendix A) showing that the multiplicative update process can approximate any target perturbation magnitude; however, this does not prove attack optimality or convergence under noisy query comparisons. \n5. Representative datasets and models. Experiments are conducted on standard and diverse benchmarks — ImageNet (VGG19, ViT) and CIFAR-100 (adversarially trained WideResNet) — which are widely accepted for evaluating adversarial attacks.\n6. Comprehensive set of baselines. The comparison covers multiple strong decision-based attacks (HSJA, CGBA, RayS, ADBA, and PAR), representing the state of the art across both $\\ell_2$ and $\\ell_\\infty$ norms.\n7. Consistent empirical improvements. Across all datasets, models, and query budgets, SeRI outperforms existing methods in terms of lower $\\ell_2$ perturbation norms and better query efficiency, demonstrating reliable performance gains.", "weaknesses": "1. Lack of justification for boundary-based. The paper adopts a decision-boundary-driven optimization scheme but does not clearly argue why boundary comparison is preferable to simpler success/failure-based evaluation, or under what conditions it provides advantages. \n2. No proof of attackability or optimality. While a convergence argument is given for the multiplicative updates, there is no theoretical or empirical evidence that minimizing the approximate decision boundary $g(d)$ actually improves attack success or robustness - i.e., no proof of optimality.\n3. Missing discussion on architecture-dependent performance. The submitted code includes experiments on Inception-v3, where the attack success rate is noticeably lower and convergence is unstable compared to ResNet and ViT backbones. This discrepancy suggests that SeRI’s effectiveness may depend on the smoothness of the decision boundary, yet the paper does not mention or analyze this behavior. An explicit explanation of why SeRI fails or underperforms on Inception-like architectures is needed. \n4. Evaluation under stronger defenses is missing. The only defense tested is an adversarially trained WRN, which is relatively weak. Robustness against stochastic or randomized defenses (e.g., random smoothing, input transformation) should be tested. \n5. Lack of statistical reporting. The paper reports only averages and medians of $\\ell_2$ norms. However, standard deviation or inter-quartile ranges would better reflect query stability and robustness variability. \n6. Missing transferability evaluation. There is no experiment analyzing whether perturbations generated by SeRI transfer to unseen models, which would further validate the claim that SeRI captures model-agnostic sensitive regions. \n7. No discussion of limitations or failure cases. The paper lacks analysis of when SeRI fails or degrades - e.g., noisy decision boundaries, non-salient images, or low-budget attacks - and provides no insight into practical failure modes. \n8. Writing and presentation issues. The paper contains several grammatical and structural errors (e.g., duplicated phrases, inconsistent notation, and citation misuse), which hinder readability and reduce perceived rigor.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)