PriSM: Prior-Guided Search Methods for Query Efficient Black-Box Attacks
Authors: Pavlos Ntais
Thanassis Avgerinos
Venue: Accepted by TMLR ()

Abstract: Deep Neural Networks are vulnerable to adversarial examples in black-box settings, requiring query-efficient attack methods. We propose PriSM (Prior-Guided Search Methods), which systematically exploits two types of transferable surrogate information: decision boundary geometry and loss landscape topography. We demonstrate their utility through complementary attacks: (1) TGEA leverages boundary geometry to initialize evolutionary optimization with surrogate evolved populations, maximizing attack success rates, and (2) SGSA leverages loss topography via multi-scale saliency guidance to direct Square Attack's perturbations, minimizing query costs. Across MNIST, CIFAR-10, and ImageNet, both methods achieve 30-60% query reductions compared to uninformed baselines, while also being competitive with state of the art hybrid attacks. Our evaluation reveals a strategic trade off: SGSA excels in query efficiency through local exploitation, whereas TGEA maximizes success rates via global exploration. Our comprehensive evaluation also demonstrates that different types of surrogate information require matched exploitation strategies, providing practical guidance for query-efficient black-box attacks.

Published in Transactions on Machine Learning Research (03/2026)
PriSM: Prior-Guided Search Methods for Query Efficient Black-Box Attacks
Pavlos Ntais
pntais@di. uoa.gr
Deportment of Informatics and Telecommunications University of Athens
Thanassis Avgerinos
thanassis uoa.
Department Informalics and Telerommmunicaaions University of Athens
Reviewed on OpenReview: https: // openreview. net/ pet//foumm? id= UQs0h2kfhp
Abstract
Deep Neural Networks are vulnerable to adversarial examples in black-box settings, requiring query-efficient attack methods. We propose PriSM (Prior-Guided Search Methods), which systomatically exploits two types transfcrable surrogate information: decision boundary geometry and loss landscape topography. We demonstrate their utility through comple- mentary attacks: (1) TGEA leverages boundary geometry to initialize evolutionary op- timization with surrogate evolved populations, maximizing attack success rates, and (2) SGSA leverages loss topography via multi-scale saliency guidance to direct Square At- tack's (Andriushchenko et al. (220)) perturbations, minimizing query costs. Across MNTST, CIFAR-10, and ImageNet, both methods achieve 30-60% query reductions compared to uninformed baselines, while also being competitive with stale of the art hybrid allacks. Our evaluation reveals a strategic trade off: SGSA excels in query efficiency through local exploitation, whereas TGEA maximizes success rates via global exploration. Our compre- hensive evaluation also demonstrates that different types of surrogate information require matched exploitation strategies, providing practical guidance for query-efficient black-box attacks.
1 Introduction
Deep Neural Networks (DNNs) have achieved remarkable success across critical domains, including
tonomous vehicles, biometric authentication, and medical diagnostics (Krizhevsky et al. 2012 He et al. 2016). However, their susceptibility to adversarial examples subtly perturbed inputs that cause misclas- sification, poses significant security risks (Szegedy et al. 2014 Goodfellow et al. 2015 Carlini & Wagner 2017
In real-world deployments, attackers often operate under black-box constraints, where they lack access model internals and only query the model for predictions (Papernot et al. 2017). This setting presents two key challenges: (1) achieving high attack success rates without gradient information, and (2) minimizing
the number of queries to evade detection and reduce computational costs (Chen et al. 2017 Lyas et al.
2018). Existing approaches: transfer-based, query-based, and hybrid methods face a fundamental Irade-off: transfer attacks require zero queries but suffer from low success rates, while query-basse attacks achieve high success the cost of thousands of queries per sample (Papernot et al.] [0017] Dong et al. 2018 Chen et al. 2017 Andriushchenko al.) 2020 Huang Yu (202)). Opportunities remain to more fully leverage surrogate model information enhance both query efficiency and attack success.

---

Published in Transactions on Machine Learning Research (03/2026)
In this work, we introduce PriSM (Prior- Guided Search Methods), novel framework that strategically incorporates surrogate-derived guidance in different ways in order to enhance black-box adversarial searches more effectively. Our key contributions are:
TGEA: A transfer-guided evolutionary attack with advanced initialization strategies (TASI and SEGI) that warm-start CMA-ES optimization (Hansen 2006), delivering the highest success rates on complex models.
SGSA: A saliency-guided enhancement to the Square Attack (Andriushhhenko et al. 2020 that uses multi-scale gradient maps from surrogates to intelligently inform perturbation placement and sizing, achieving state-of-the-art query efficiency.
Intuitive Analysis: Intuitive just instifications showing that SGSA exploits local loss geometry trans- ferability, while TGEA leverages global decision boundary correlations.
Our results demonstrate that PriSM offers strategic choice between minimizing query costs and maximizing attack success. Both SGSA and TGEA reduce queries by up to 50% over random basclines while achieving equal or greater success rales. This work advances the understanding of prior-guided paradigms in adversarial machine learning, providing practical tools for robustness evaluation.
2 Related Work
Black-box adversarial attacks balance success rates and query costs through three paradigms: transfer- basod, query-basod, and hybrid approaches, with recent advancements incorporating saliency guidance and evolutionary optimization.
Trasefer-based attacks exploit cross-model transferability by generating perturbations surrogate models trained on similar data distributions (Papernot et al. 2017 Liu et al.) (017). Efforts to boost transferability include ummentum integration (Dong et al. 2018), input diversity (Xie et al.] 2019), and ensemble meth- ods Liu et al.) 2017). Although these require no queries to the target, their effectiveness often limited against robust defenses (He et al. 2017).
Query-based attacks iteratively refine perturbations via direct model queries. Score-based methods like SimBA (Guo et al., 2019) and Square Attack (Andriushchenko et al.) 2020
confidence scores, with Decision-based approaches
Square Attack leading benchmarks such BlackboxBench (Zheng et al.
2025
like Boundary Attack (Brendel et al. 2018) rely solely on hard labels but incur higher query demands (Chen
& Gu 2020). Saliency-guided variants prioritize perturbations in salient regions (Dai et al. 2023
Soor et al.
2025
Hybrid attacks combine these paradigms, using surrogate priors to initialize or guide query-based searches.
Examples include TAGA (Huang & Yu 2022). which refines transfer seeds via genetic algorithms,, and
Hybrid Batch Attacks (Suya et al. 2020 which leverage local surrogates for scalable generation. Recent
work enhances transferability through meta-learning (Fu et al. 2022), ensemble surrogates (Cai et al. 2022
and saliency integration (Wang et al. 2024 Huang & Kong (022). Most recently, PBO (Cheng et al. 2024
introduced Bayesian optimization with a learned function prior, achieving state-of-the-art query efficiency on standard models by exploiting loss surface smoothness.
Evolutionary algorithms navigate high dimensional spaces without gradicnts (Su et al. 2019 Alzantot et al. 2019). CMA-ES (Hansen) 2006) adapts search distributions via covariance updates, showing promise in adversarial contexts, though random initializations can lead to slow convergence (Qiu et al. 2021 Kuang
ct al. 2019
Despite these advances, existing hybrids often underutilize surrogate priors for continuous guidance or so- phisticated warm-starts. Our PriSM framework addresses this through SGSA, which embeds multi-scale saliency from surrogates into every Square Attack query docision, and TGEA, which enhances evolutionary hybrids with surrogate-evolved initializations (c.g., SEGI), achieving up 8.24% higher success rates on robust models while maintaining low query costs.
2

---

Published in Transactions on Machine Learning Research (03/2026)
3 Background
3.1 Adversarial Attacks
Let fa : X' + I denote a deep neural network classifier with parameters 0, mapping input a E X to logits over K classes. For a clean input x with true label y, the model's prediction 9 = arg max; fo(2)..
An adversarial example x' x + a perturbed input that causes misclassification while remaining per- ceptually similar to æ. The perturbation is typically constrained by || e, where p {0, 0o) and controls the perturbation budget.
White-box attacks. With full access to model parameters and gradients, white-bor attacks like PGD Madry et. al. 2019) iteratively optimize:
x+ [x sign(V=[(fooz") v)))
(1)
where II, projects onto the E, ball radius e, and L is the attack loss (e.g., cross-entropy).
Black-box attacks. In black- box settings, attackers lack gradient access and can only query the model. These attacks categorized by query type:
Score-based: Access to confidence coores/logits Decision-based: Access only to predicted labels
Transfer-based: Zero queries, relying on adversarial transferability
3.2 Square Attack
Square Attack (Andriushchenko al. 2020) is a scorc-based black- black-box attack that does not rely on local gradient information. It selects localized square-shaped updates at random positions such that the pertur- bation is situated approximately at the boundary of the feasible set. At iteration it randomly samples square region depending on the current square size h(0, The algorithm minimizes the margin-based loss (( = (ē) max fx()). The update rule for the perturbation 8 is defined as:
Project(0' + v4) if L(fo(x Project + +v)). ( otherwise
81+1
(2)
8
where the update ii sampled from discrete distribution {-2c, 26 within the square region, ensuring the perturbation stays at the corners of the ballbal. The Project function ensures the adversarial example remains within the valid e-ball bound: Project(z) min(max(z, - x), 1e
This operation constrains the perturbed input to the intersection {= E Rd || æ1100 10
3.3 CMA-ES for Black-Box Optimization
Covariance Matrix Adaptation Evolution Strategy (CMA-ES) (Hasse 2006 is gradient-free optimizer that adapts a multivariate normal search distribution. At generation g, samples candidate solutions:
M(m(9), (o)))) i=
(4)
where (9) is the mean vector, (1)) is the step size, and ((a) is the covariance matrix.
The mean is updated using the a best individuals:
(9+1) +1)
W.S F:A
(5)
3

---

Published in Transactions on Machine Learning Research (03/2026)
where Zex denotes the i-th best solution and wi are recombination weights.
The covariance matrix adapts to the local loss landscape through rank-one and rank-u updates, enabling efficient search in high-dimensional spaces without gradient information. However, CMA-ES suffers from slow convergence when initialized randomly, particularly for adversarial attack objectives.
3.4 Surrogate Models and Transferability
A surrogate model S& is white-box model accessible to the attacker, typically trained on similar data as the target model ft. Adversarial transferability refers to the phenomenon where adversarial examples crafted on fa often fool ft:
Pr[Je(x + 6.) # > 0 where 5g = arg max L(f, v),
Transfcrability ariscs from models learning similar decision boundarics on similar data distributions (Paper- not et al. 2017). However, transfer success varies significantly with model architecture, training procedure, and robustness (Huang et al.) 2017
Saliency maps.
Gradient- braddent-bbsed saliency maps indicate input region importance:
S(z) ((f
High saliency regions are more sensitive to perturbations. While saliency is model-spocific, relative im- portance patterns often transfer across models (Simonyan & Zisserman 2015), enabling surrogate-guin attacks.
4 Methodology
4.1 Motivation
In this work, we investigate how surrogate model information can be systematically exploited to improve query efficiency in black-box adversarial attacks. While based attacks require zero queries but suffer from low success rates, and query-based attacks achieve high success at the cost of thousands of queries, hybrid approaches that stralegically leverage surrogate priors remain underexplored. To address this gap, we propose two methods that demonstrate the importance of matching surrogate information types to compatible search algorithms through the unifying principle of structural alignment, where the dimensionality and nature of the surrogate prior matches the operational mochanics of the black-box search algorithm. This principle guides our methodological decisions, ensuring that distinet lypes of transferable priors naturally align with specific optimization paradigms. More specifically, focus on two transferable properties that provide information about the attack surface:
Decision Boundary Geometry. Models trained on similar data distributions learn correlated decision boundaries. Adversarial examples lying near surrogate's decision boundary are statistically enriched near the target's boundary (Section 4.2.4. This geometric correlation provides spatial information indicating which regions of input space are promising for adversarial search.
Loss Landscape Topography. Gradient-based saliency maps reveal input regions where the loss function is most sensitive to perturbations. Empirical evidence shows that these sensitivity patterns transfer across architecturally similar models, even when exact gradient vectors differ. This topological information provides directional guidance, indicating which perturbations are most likely to increase loss.
We locus on these properties because they represent fundamentally different aspects of transferability that is directly exploited in query-based attacks. Boundary geometry provides information about the global structure of the adversarial space where misclassification regions exist, which is critical for initializing scarch algorithms effectively. Loss topography provides information about the local sensitivity landscape, which in- put regions most vulnerable to perturbation, which is essential for guiding iterative refinement. Together,
4

---

Published in Transactions on Machine Learning Research (03/2026)
they span the spectrum from global positioning to local gradient information, enabling complementary ex- ploitation strategies for query-efficient attacks.
We incorporate these properties into SOTA query-based attacks through two distinct approaches: TGEA (Section 4.2.4 leverages boundary geometry to warm-start evolutionary optimization with surrogate-cvolved populations, achieving up to 98% attack success rates while reducing queries by 30 60% compared to random initialization. SGSA (Section 4.3.3) leverages loss topography to guide Square Attack's random perturbations toward salient regions, achieving query reductions of 30-50% while maintaining competitive success rates. TGEA and SGSA are operationally independent algorithms, not sequential steps in pipeline. However, they are strategically complementary, targeting different transferable properties at different scarch scales.
Ultimatll,, these methods demonstrate that understanding what transfers enables principled design of how to exploit it. By exploring two distinct types of transferable information, we demonstrate that effective exploitation depends on matching the information structure the search algorithm's operational charac- teristics. Thus, this framework not only achieves substantial query reductions across diverse threat models, but also provides a foundation for future work to identify and exploit additional transferable properties in black-box adversarial settings.
4.2
Guided Evolutionary Attack (TGEA)
4.2.1
Overview
Traditional evolutionary attacks on black-box models have been widely used in the literature (Qiu al. 2021 lyas et al., 2018 Alzantot et al. 2019). They rely on randomly initialized populations, which empirically demonstrate can lead to relatively slower convergence and higher query costs. This is particularly problematic in scenarios where query budgets are limited or the decision boundaries of the target model are complex. To address these limitations, we introduce the Transfer-Guides Evolutionary Attack (TGEA), a framework that leverages information from a surrogate model to initialize the population of an evolutionary algorithm, which acts as powerful global search method. By using transfer-based priors, we demonstrate that TGEA improves the efficiency of the attack process and significantly reduces query complexity.
The core idea of TGEA is to build upon the hybrid attack paradigm by bridging the gap between transfer learning and evolutionary optimization. As ostablished by (Suya et al. (020), this approach uses high-quality adversarial examples generated on a surrogate model to provide a 'warm start' for the attack on the black- box model. However, while (Suya et al. 2020) typically transfers a single adversarial candidate to initialize a local gradient estimator (e.g., NES or AutoZOOM), TASI and SEGI utilize surrogates to construct a diverse population of candidates. By estimating both the mean and the covariance of these candidates, our method warm-starts the CMA-ES search distribution rather than just single trajectory. This captures the structural geometry of the adversarial subspace, effectively defining directed search cone rather than a single point; this population- baasdd alignment smooths out local irregularities, allowing the optimizer to step over non non-tronstieee con-transeerable traps and converge toward the global optimum.
Tsarie tucal
Spiinu
Canvengesr
Figure 1: Schematic illustration of TGEA's initialization strategy. TGEA's scarch cone (green) uses surrogate evolved populations to initialize CMA-ES, enabling convergence the global optimum (gold), unlike single- point transfer attacks (blue) that get trapped in local optima random initialization (brown) that lacks guidance.
5

---

Published in Transactions on Machine Learning Research (03/2026)
4.2.2 Population Initialization Strategies
Transfer-Attack Seeded Initialization (TASI). TASI generates diverse initial candidates by applying multiple types of attacks to the surrogate model, effectively serving as a mechanism to combine multiple powerful attack strategics into a single, unified initialization pool. For clean input 2, we construct a
diverse set of transfer attacks A A21.42..... Ag...., (e.g.. PGD (Madry al. 2019 Square Attack (Andriushchenko et al. 2020), Boundary Attack (Brendel et al., 2018
For each attack A,, generate base adversarial example x!dh A((x, fa) and create q local variants by adding Gaussian noise:
Fi = nuda + NN(0,02),, j=1,...,;
The final population is Po (2a, h== k:j=.... with Pal = N.
Surrogate-Evolved Genetic Initialization (SEGI). SEGI employs a meta-opliniiation process, running a Genetic Algorithm (GA) entirely on the surrogate model to evolve a high-quality initial population. Starting
from a random population of size N, we evolve it for G generations using a surrogate-spoeific fitness function:
F.(6) - as -max fas(1 + 5) io (a + 6) + 5. Deentroid (0) + Raiv (P) jFy
(9)
where fajj denotes the j-th class score from surrogate fa, and Raiv is a diversity penalty:
Raiv = = P
MSE(6,p)
(10)
PEP
The term Decutroid guides the search in the latent space:
Deontroid (0) ||=( + 6) c. l1l min ||=(1 + 6) -cill2
(11)
izy
where z(-) is the penultimate layer embedding and & is the centroid class the latent space (computed on the surrogate). This encourages the GA to maintain population diversity and steer candidates toward incorrect class clusters, preventing premature convergence.
After G generations, select the top N individuals by fitness, add small Gaussian noise to each, and use them to initialize CMA-ES on the target model.
Our fitness functions build upon the frameworks established in (Huang & Yu 2022 and (Qiu et al. 2021 but extend them by assigning adjustable weights to each component term. These weights were empirically calibrated through ilerative experimentation to optimize overall performance. It is important however note that unlike (Huang Yu 2022) which uses standard genetic operators, our TGEA employs CMA-ES optimization.
4.2.3 CMA-ES Refinement
Given the initialized population Po, we apply CMA-ES to optimize the target model's loss. function for the target model is:
The fitness
Fi(6) = a: max fe, (x + - St +)
(12)
We deliberately distinguish the weighting parameters (t) from those used the surrogate stage (a,, 7a) to prioritize exploitation of the target model's specific docision boundary over the exploration-heavy objective used during initialization. View Appendix B1 for ablations. CMA-ES iteratively updates the mean vector ml1), step size o(!), and covariance matrix (1)) described in Hansen 2006).

---

Published in Transactions on Machine Learning Research (03/2026)
4.2.4 Design Rationale and Intuitive Motivation
Decision Boundary Transferability. TGEA leverages the empirically observed phenomenon that models
trained on similar data learn correlated decision boundaries (Papernot al. 2017 Liu al. 2017). We define the near-boundary region with margin threshold т as:
Bj = {x: |margin([())) <+}
(13)
where margin([(z))) = fa(a) - max (2).
Prior work suggests that adversarial examples near the decision boundary of a surrogate model are often also near the boundary similar target models (Tramèr et al. 2017). While the exact enrichment factor varies with architecture similarity, we qualitatively expect:
E BE
x x (E) > E(
(14)
Expected Query Reduction. By initializing CMA-ES with examples from B3, we essentially hypothesize that the search starts closer to By compared to random initialization. Since CMA-ES convergence depends the initial distance to the optimum (Hansen & Ostermeier 2001), transfer-based initialization should reduce the number of queries required.
To validate this hypothesis, we compare queries required by TGEA versus random initialization across our experiments. Our results show that TGEA consistently requires fewer queries than random CMA-ES (Tables 1-4), with reductions ranging from 30-60% depending on the dataset and model architecture.
4.3 Suliency-Guided Square Attack (SGSA)
4.3.1
Motivation
The Square Attack (Andriushchenko et al. 2020) achieves state of the art query efficiency through random square-shaped perturbations that align with CNN inductive biases, specifically exploiting the local spatial correlations learned by convolutional architectures. By perturbing contiguous square regions rather than individual pixels; Square Autack naturally largets the receptive field structures that CNNs are sensitive to, making highly effective in black-box settings. However, despite this architectural alignment, its purely random search strategy treats all spatial locations uniformly, allocating queries without regard their potential impact model predictions. This leads significant query waste irrelevant low-sensitivity image regions such uniform backgrounds, occluded areas, or regions far from decision boundaries that contribute minimally to inducing misclassification.
We empirically demonstrate that this limitation can be addressed by incorporating surrogate-derived saliency maps to provide spatial guidance for the attack. Specifically, gradient- saliency maps computed on an accessible surrogate model identify regions where the loss function exhibits high sensitivity to input pertur- bations. By leveraging the empirical observation that such high-gradient regions often transfer across archi- tecturally similar models, we target Square Attack's random search toward sensitive areas. This approach preserves Square Attack's strengths while intelligently focusing computational resources on perturbations most likely to succeed, thereby further reducing query costs without sacrificing attack success rates.
4.3.2 Hybrid Guidance Map Generation
SGSA constructs dynamic vulnerability heatmap S(x) by combining multi-scale saliency and attention mechanisms from the surrogate model.
Multi-Scale Saliency. We compute the gradient magnitude of the surrogate loss with respect to the input at multiple scales {s1, $2, s3):
grad (x: = [V. v)l2
(15)
Smutes (x)
Upsample(Smooth(S"4
grad (I)))
(16)
S
7

---

Published in Transactions on Machine Learning Research (03/2026)
The multi-scale approach captures different levels of feature abstraction. By incorporating downsampled scales, we effectively apply a low-pass filter to the gradient information, guiding the attack towards structural, low frequency vulnerabilities that are more transferable across architectures than high frequency noise (Ilyas et al. 2018). Specifically, high frequency gradient features (pixel level noise, texture artifacts) are model specilic, while low frequency structural patterns (object boundaries, semantic region importance) represent universal vulnerability patterns shared across architectures. Downsampling suppresses model specific noise while retaining transferable structural signals; subsequent upsampling restores spatial resolution for precise perturbation placement.
Attention Map Integration. We optionally incorporate an attention mechanism A(x) from the surrogate:
(x) = (1 3) (m) + A-x)
(17)
where 3 = 0.3 balances gradient sensitivity and model attention.
Temporal Smoothing. As the adversarial example evolves, we update Shybrid periodically and apply exponential moving average:
s(++1) x- x + +11 (1- Shybeia
(18)
with À 0.9 to maintain stability while adapting to perturbation changes.
2
(a) MNIST
(b) CIFAR-10
(c) ImageNet
Examples of saliency maps generated from surrogate models (left: original, right: map). Brighter regions indicate higher model sensitivity and thus represent more promising areas to attack.
Figure 2:
4.3.3 Guided Perturbation Placement and Sizing
At iteration SGSA selects the square location and size based on s(t)
Probabilistic Location Sampling. We first normalize 5(1) to create valid probability mass function over pixel locations:
$
Ti
(0)
c(t)
(19)
Eu,
Su.
We then sample the top-left corner (r,c) of the square from this distribution:
(r, Categorical(z
(20)
This focuses perturbations high-saliency regions.

---

Published in Transactions on Machine Learning Research (03/2026)
Figure 3: Comparison of sampling strategies for black-box adversarial attacks. Left: Uniform sampling allo- cates equal probability across all spatial locations via uniform distribution, characteristic of random search methods like Square Attack. Right: Saliency-guideds sampling uses a probability distribution concentrated on high-gradient regions identified through surrogate model gradients (warm surface) that correlate with target model vulnerabílities (bluc wireframe), cnabling more efficient perturbation placement.
Adaptive Size Adjustment. The square side length (4) computed via an inverse relationship with local saliency:
h (1) Masse saliency_faccor(rr c)
(21)
where:
saliency_factee
(22)
1 + Osccale s(t)
High saliency leads to smaller, precise perturbations; low saliency to larger, exploratory ones.
Lergas
Galerry
Ieat
Figure 4: SGSA adaptive sizing mechanism. The saliency map shows gradient magnitude from the surrogate model. Large perturbation squares (cyan, dashed) are placed in low-saliency regions for exploration, while small squares (green, dashed) target high-saliency regions for precise attacks.
4.3.4 Fallback Mechanism
To ensure robustness against poor transferabilityy SGSA includes dual fallback triggers:
Stagnation-based: If loss docs not improve for Tstag itcrations, revert to random search for the next. Trand iterations.
Stochastic: With probability Prand, perform random square placement to escape local optima.

---

Published in Transactions on Machine Learning Research (03/2026)
4.3.5 Design Rationale and Empirical Validation
Motivation from White- Box Attacks. The core principle of SGSA, prioritizing perturbations in high- gradient regions, mirrors the fundamental mechanism of gradient-based white-box altacks. Methods like PGD and FGSM explicitly compute:
= sign(Vv_((f(x).))))
(23)
which concentrates perturbation magnitude where gradients are largest, i.c., where the loss is most sensitive to inpul changes. SGSA exlends this principle to the black-box setting by using surrogate gradients to guide random search.
Local Gradient Transfcrability Hypothesis. SGSA leverages multi-scale saliency patterns from surrogale models to guide perturbation placement. Let S.(x), = IV (L)( and St(x); = (f)())))) denote gradient magnitude saliency maps for surrogate and target models, respectively.
We measured the Spearman rank correlation between surrogate (ResNet18) and target model multi-scale saliency maps on 1,000 correctly classified CIFAR-10 samples:
Measured Correlations:
CIFAR-10 standard models: p = 0.68 + 0.16 (InceptionV3), p = 0.63 = 0.18 (VGG16)
CIFAR-10 robust modols: p = 0.41 + 0.20 (WidcRosNet), = 0.39 = 0.19 (WRN-94-16)
Multi-scale averaging yields 55-63% improvement. over single-scale for standard models and 160% for robust models (all p 0.001). The moderate correlations (p 0.39-0.68) demonstrate that saliency patterns transfer sufficiently to guide black-box attacks, with standard models showing stronger correlation than adversarially trained models due to their distinet loss landscapes (Qin et al. 2019).
Query Efficiency Through Guided Search. SGSA prioritizes perturbations in high-saliency regions, which we expect to be more effective for causing misclassification. To understand the potential query savings, consider simplified model where:
High-saliency regions H comprise fraction of the input space
Due to correlation SGSA places perturbations in effective regions with higher probability
Perturbations in H. are À times more effective than in low-saliency regions L
Under these assumptions, we can heuristically approximate the expected query reduction as:
1
E[QsGSA] RC 1 pax(A 1) [[QQsquar]]
(24)
While this is an oversimplffication (it assumes binary effectiveness, independence, and perfect guidance), it provides intuition for why guidance helps. For typical conservative values, a 0.2 (CNN saliency con- centrates on roughly 20% of spatial locations corresponding to object boundaries), p 0.65 (our measured surrogate target correlations from Section 4.3.5), and À = (reflecting the expectation that perturbations in high saliency regions significantly more effective at inducing misclassification than changes to the back- ground) the model predicts approximately 21% query reduction. While our empirical results show greater reductions (30-50% in Tables g and 5), this simplified model provides intuition for the fundamental mech- anism: by concentrating search on regions where surrogate gradicnts transfer, SGSA reduces the expected queries needed to find successful perturbations.
Adaptive Sizing Rationale. Our adaptive sizing rule (0) = - /1 (1 Oscale S!!!) is motivated by the intuition that:
In high-saliency regions (steep local loss), smaller perturbations provide more precise control
In low-saliency regions (flatter loss), larger perturbations enable faster exploration
10

---

Published in Transactions on Machine Learning Research (03/2026)
5
Experimental Evaluation
5.1
Experimental Setup
5.1.1
Datasets and Models
We evaluate our framework on three datasets of incrcasing complexity to test scalability and robustness:
MNIST:
10,000 grayscale 28 x 28 handwritten digit images. We target two models: Model A (CNN
architecture from (Tramèr et al. 2020 and Model B (architecture from (Carlini & Wagner 2017 both trained in-house. The surrogate is a lightweight 2-layer CNN.
CIFAR-10: 10,000 RGB 32 32 images across 10 classes. We evaluate on standard pretrained modcls: InceptionV3 (Szegedy et al. 2016) and VGG16 (Simonyan & Zissorman 2015). To assess performance against defenses, test adversarially trained models: WideResNet (Carmon et al, 2019) and WRN-94-16 (Bartoldson et al. 2024). the latter being the current state-of-the-art on BenchtBench (Croce et al. 2021). The surrogate is ResNet18 (He et al. 2016). Non robust model weights used are from Phan 2021
ImageNet: 1,000 validation images from ImageNet-mini Figotin 2019 (224 224 RGB). Targets
include DenseNet-121 (Huang et al. 2018 and EfficientNet-BI Tan & Le 2019 The surrogate is ResNet18.
5.1.2 Attack Configuration
We benchmark PriSM against state-of-the-art methods categorized in Table D All attacks operate under the Loo norm. We set perturbation budgets E {0.2, 0.3) for MNIST, f0.1, 0.2] for CIFAR-10, and {0.05, 0.1} for ImageNet. The maximum query budget is fixed at Qmax 1000. We sample ~100 correctly classified images per dataset.
Table 1: Summary of evaluated black-box attack methods
Method
Typc
Description
TGEA-TASI (Ours)
Hybrid
Transfer attacks + CMA-ES
TGEA-SEGI (Ours)
Hybrid
GA-evolved surrogate + CMA-FS
SGSA (Ours)
Hybrid
Saliency-Guided Square Attack Randomly initialized CMA-ES
Random CMA-ES
Score-based
Square Attack (Andriushchenko et al. 2020
Score-based
Random square perturbations
SimBA (uu et al.
2019
Score-based
Coordinate wise random search
HSJA CCen et al.
2020
Decision-based
HopSkipJump Attack boundary search
PBO
Cheng et al.
2024
Hybrid
Bayesian optimization with function prior
5.2
Results
5.2.1
MNIST
Tables 2 and summarize performance on MNIST. The simple dataset structure favors local search strate- gies. SGSA achieves the highest Attack Success Rate (ASR) of 99.81-100% while maintaining low query costs (54.37-181.8) queries). PBO demonstrates exceptional query efficiency (as low as 14.15 queries) but occasionally trails SGSA in success rate on Model A. TGEA variants show competitive ASR but higher query costs, as global optimization is less critical for this lower-dimensional manifold.
11

---

Published in Transactions on Machine Learning Research (03/2026)
Table 2: ASR and AQ on MNIST (Coos E = 0.2).
Table 3: ASR and AQ on MNIST (600, E = 0.3).
Model A
Model B
Model A
Model
Method
ASR 1
AQ +
ASR. t
AQ 4
Method
ASR
AQ
ASR 1
AQ
TGEA-TASI
65.76
250.05
82.11
197.74
TGEA-TASI
91.76
130.24
97.13
74.94
TGEA-SEGI
47.15
255.86
60.34
272.78
TGEA-SEGI
83.69
164.57
90.78
125.23
Random
37.77
519.96
51.37
556.02
Random
76.85
398.30
88.39
357.62
Square
87.86
239.38
91.79
264.05
Square
99.93
91.45
100.00
97.73
SGSA
89.89
177.48
93.98
181.80
SGSA
99.81
54.37
100.00
56.68
SimBA
59.32
306.05
69.07
265.60
SimBA
94.14
199.98
81.50
151.06
HSJA
20.23
431.29
26.48
483.21
HSJA
72.09
407.43
76.04
398.74 73
PBO
70.39
25.36
89.05
14.15
PBO
97.44
99.75
99.70
67.30
5.2.2 CIFAR-10
Results for standard CIFAR-10 models are presented in Tables and A strategic trade off emerges: TGEA-SEGI achieves the highest ASR (up to 98.36%) among cvolutionary methods, proving effective for maximizing allack success. PBO excels in query efficiency, achieving the lowest AQ with high ASR.
Table ASR and AQ on CIFAR-10 10 (Loo, e 0.1).
Table 5: ASR and AQ on CIFAR-10 (Loo 0.2).
InceptionV3
VGG16
InceptionV3
VGGI6
Method
ASR 1
AQ 1
ASR. f
AQ 1
Method
ASR
AQ
ASR
AQ
TGEA-TASI
89.69
120.75
88.03
180.28
TGEA-TASI
97.10
53.82
97.20
59.73
TGEA-SEGI
91.43
105.26
89.80
127.82
TGEA-SEGI
98.36
27.95
98.19
34.80
Random
85.27
148.62
83.29
198.42
Random
95.56
78.30
92.61
80.85
Square
88.20
76.64
81.56
91.70
Square
94.39
38.15
91.33
41.95
SGSA
85.64
53.47
82.85
77.62
SGSA
93.04
30.21
91.04
34.05
SimBA
75.89
105.63
76.33
163.42
SimBA
77.92
81.80
77.94
104.60
HSJA
56.39
368.84
40.07
442.09
HSJA
92.67
240.01
77.71
255.88
PBO
88.69
52.86
87.80
47.95
PBO
98.67
16.87
99.43
19.33
Robust Models. Tables 0 and evaluate attacks adversarially trained models. Typically, these models mask gradients, making attacks harder. However, observe a distinct phenomenon where SGSA outper- forms baselines significantly. We hypothesize is because adversarial training creates 'perceptually aligned" gradients, meaning the gradient of a robust surrogate is more interpretable and structurally correlated with the targel. model than that of non robust. model (Tsipras et al., 2019). SGSA possibly leverages this restored correlation break the traditional effccieccy-suceess trade off, achieving the highest ASR (61.24%) the SOTA robust model WRN-94-16.
Table 6: ASR and AQ on robust models (loo, c 0.1).
Table 7: ASR and AQ on robust models (fxc, 0.2).
NeteResNet
WRN-94-16
WideRes Net
WRN-94-16
Method
ASR 1
AQ 1
ASR. 1
AQ 1
Method
ASR 1
AQ
ASR
AQ
TGEA-TASI
40.23
135.09
32.17
124.62
TGEA-TASI
60.82
181.41
55.71
225.07
TGEA-SEGI
37.08
140.98
31.27
178.06
TGEA-SEGI
59.32
178.22
52.33
225.51
Random
38.11
143.05
31.58
189.83
Random
60.52
178.84
54.65
254.03
Square
42.09
115.60
32.92
174.91
Square
65.07
116.10
60.65
151.02
SGSA
49.38
52.78
37.99
120.44
SGSA
68.20
67.94
61.24
123.70
SimBA
41.95
82.14
30.12
195.92
SimBA
51.15
125.47
46.20
293.47
HSJA
38.62
77.46
26.32
73.93
HSJA
48.02
123.61
29.19
54.89
PBO
45.52
22.03
31.58
124.72
PBO
69.16
92.25
62.11
141.93
12

---

Published in Transactions on Machine Learning Research (03/2026)
5.2.3 ImageNet
Tables B
and 9 summarize performance on ImageNet models. SGSA achieves superior query efficioncy on
Dense Net (55.85-75.9) queries) while TGEA-SEGI maximizes ASR on EfficientNel (76.88-89.10%) demon-
strating scalability high dimensional imputs.
Table 8: ASR and AQ on ImageNet (Loo e 0.05).
Table 9: ASR and AQ on ImageNet (Lo,, e - -01)).
DenseNet
Efficient
DenseNet
Elficient
Method ASR 1
AQ 1
ASR. t 72.73
AQ 4
Method
ASR t
AQ
ASR
AQ
TGEA-TASI
75.43
99.05
188.89
TGEA-TASI
88.63
128.01
83.82
158.75
TGEA-SEGI
72.50
100.08
76.88
193.62
TGEA-SEGI
89.47
111.25
89.10
148.42
Random
72.50
98.79
76.88
182.77
Random
89.10
110.78
86.84
130.38
Square
82.01
116.57
69.16
137.35
Square
91.84
79.93
84.71
118.79
SGSA
85.59
75.91
72.87
102.31
SGSA
93.99
55.85
86.99
89.56
SimBA
66.77
53.04
50.55
64.33
SimBA
71.47
79.29
55.06
97.45
HSJA
42.86
77.33
18.10
116.00
HSJA
30.16
98.23
28.65
171.39
PBO
71.43
116.60
61.21
167.55
PBO
93.33
100.67
81.87
167.02
To assess the generalization of our methods beyond convolutional neural networks, we extend our evaluation newer, Vision Transformer architcctures. We perform experiments on two representative models: the standard Vision Transformer (ViT-B/16) (Dosovitskiy al. 2021) and the Data-efficient Image Transformer (DeiT-S) [Touvron et al. 2021). These experiments demonstrate that priors derived from CNN surrogates (spatial saliency and decision boundary geometry) can effectively transfer to the distinct loss landscapes of transformer-basee models.
Table 10: ASR and AQ on ViT (lxc, c 0.05).
Table 11: ASR and AQ on ViT/DeiT (lx,, c = 0.1).
B/1-
DeÏT-S
ViT-B/16
DeñT-S
Method
ASR f
AQ
ASR
AQ +.
Method
ASR. 1
AQ
ASR 1
AQ
TGEA-TASI
64.26
215.30
61.36
202.65
TGEA-TASI
77.15
193.95
79.16
197.41
TGEA-SEGI
55.05
241.17
57.33
275.49
TGEA-SEGI
74.22
208.98
75.10
178.05
Random
53.12
248.62
56.67
267.22
Random
72.11
219.93
73.43
198.04
Square
56.93
131.16
62.99
132.11
Square
69.86
139.33
82.25
124.67
SGSA
61.59
111.10
70.10
105.14
SGSA
75.74
110.40
87.90
104.52
SimBA
32.63
240.42
47.21
243.12
SimBA
43.24
221.42
46.21
243.12
HSJA
24.82
180.42
42.13
201.32
HSJA
47.91
193.32
36.42
202.42
PBO
53.15
193.10
63.25
191.50
PBO
62.61
186.51
72.23
189.13
5.3
Sensitivity Analysis and Component Validation
To demonstrate the robustness our hyperparameters and identify optimal configurations, we conducted systematic sensitivity analysis for both attacks. We present the key findings here, with comprehensive experimental results and additional ablation details available in Appendix B. The results, summarized in Figure t and Figure highlight key trade-offs between hyperparameter choices and attack performance.
5.3.1 TGEA Configuration and Fitness Terms
Figure
analyzes the evolutionary components of TGEA:
Impact of Fitness Terms (Figure 5a): We performed an ablation study on the SEGI fitness function components. The results show that removing the target class penalty Ye. unsurprisiggy, causes the most severe dogradation, particularly for DenseNet (green bar drops significantly). This confirms that explicitly penalizing the true label critical for evolving effective "search concs" that steer the population away from the original class manifold.
13

---

Published in Transactions on Machine Learning Research (03/2026)
We tested CMA-ES population sizes of {6. 16, 26). The data indicates optimal size of 16 across most models. Smaller populations (6) lack sufficient diversity to escape local optima, while larger populations (26) appear to dilute the search pressure, leading to slower convergence and lower ASR, especially for VGG16 (orange linc).
Population Size (Figure d)):
The attack shows stability around a target weight or 1.0. However, increasing the target penalty "/ (Figure 3 yields significant gains for DenseNet, suggesting that complex models might benefit. from more aggressive penalties the true class probability.
Weight Balancing (Figure p-c):
Thupee
Figure 5: Sensitivity Analysis of TGEA Configuration. We evaluate (a) Ablation of SEGI Fitness terms, (b) Target Weight on, (c) Target Penalty ne and (d) CMA-ES Population Size. The analysis reveals that the target penalty term is critical for success (a) and that a population size of 16 provides the optimal balance between exploration and exploitation (d).
5.3.2 SGSA Hyperparameter Sensitivity
Figure 0 provides comprehensive breakdown of how SGSA's components influence success rates:
Saliency Scale (Figure Ga): We notice a positive correlation between the scale factor and ASR, particularly for VGG16 and InceptionV3. A factor of 10 consistently outperforms smaller factors (2 or 5), suggesting that stronger Gaussian smoothing essential for extracting stable, transferable gradients from the surrogate.
Attention Weight (Figure 6b): The attack is relatively robust to 3 values between 0.1 and 0.75, with peak performance typically around 3 0.3. Notably, setting 0 (no attention) leads to a sharp drop in ASR for InceptionV3 and VGG16, validating the necessity of the attention mechanism.
The update frequency controls the trade-off between gradient freshness and computational cost. While & = 50 offers a stable sweel spot, increasing & to 100 (lazier updates) causes a noticeable performance degradation on complex architectures like EfficientNet (red line), indicating that high dimensional landscapes require more frequent gradient refreshos.
Update Frequency & (Figure 6c):
Temporal Decay À (Figure 6d):
Higher decay values tend to yield better results, suggesting that retaining a longer history of gradients helps stabilize the search direction against local noise.
14

---

Published in Transactions on Machine Learning Research (03/2026)
Soalo
do Ingast Attieationn Weight
Welge:: (8)
Irspact Update Froquency 2
Impact Temmpraa Decuy
180
0.2
04 8.5 Temporal
37
09
Upddt Prequaxc
Figure 6: Sensitivity Analysis of SGSA Hyperparameters. We analyze the impact of (a) Saliency Scale Factor, (b) Attention Weight 3, (c) Saliency Update Frequency k, and (d) Temporal Decay across four target architectures. The results validate that stronger smoothing, moderate attention, and high temporal retention generally are more robust.
6 Analysis and Discussion
Our experimental evaluation across MNIST, CIFAR-10, and ImageNet reveals consistent patterns in the performance characteristics of PriSM's two approaches.
6.1 Performance Against Baselines
Substantial Improvements Over Established Baselines. Both SGSA and TGEA demonstrate sig- nificant improvements over established baseline methods across all experimental settings. SGSA achieves 30 50% query reductions compared to Square Attack while maintaining or exceeding success rates. For instance, on InceptionV3 (e = 0.1), SGSA requires only 53.47 queries versus 76.64 for Square Attack, a 30.23% reduction, while maintaining competitive ASR (85.64% vs 88.20%). Similarly, TGEA consistently outperforms Random CMA-ES initialization by 30-60% in query efficiency, validating the fundamental value of surrogale-guided initialization for ionary methods. On CIFAR-10 at 0.2, TGEA-SEGI requires only 27.95 queries compared to Random CMA-ES's 78.30 queries while achieving higher success rates (98.36% 95.56%)
Against other gradient free baselines like SimBA and HSJA, PriSM methods show even more pronounced advantages. On ImageNet models, SGSA reduces queries by 40 50% compared to SimBA while achieving 15 20% higher success rates. These systematic improvements across diverse baselines confirm that exploiting transferable surrogate provides substantial benefits over uninformed search strategies.
Competitive Performance with SOTA PBO. PBO (Cheng al. 2024 represents the current state of the art in query-effieient black-box attacks, leveraging Bayesian optimization with learned function priors. Our evaluation reveals that PriSM achieves competitive superior performance depending the attack scenario, with clear complementary strengths.
On CIFAR-10 standard models, PBO achieves exceptional query efficiency (16.87 vs 52.86 queries) with high success rates, outperforming both SGSA and TGEA variants in this regime. This advantage stems from Bayesian optimization's sophisticated modeling of loss surfaces through Gaussian processes, which is particularly effective on simpler, well-behaved optimization landscapes where the GP function prior can accurately approximate the target model's loss function.
15

---

Published in Transactions on Machine Learning Research (03/2026)
However, adversarially trained models, the performance dynamics shift substantially. On WRN-94-16 (e 0.2), SGSA matches PRO's success rate (61.24% vs 62.11%) while requiring 12.84% fewer queries (123.70 70 vs 141.93). This suggests that Bayesian optimization's 's GP--baedd function prior becomes less effective on the smoothed but highly non convex landscapes of robust models, where SGSA's multi-scale saliency guidance provides more reliable directional information. The measured saliency correlations (p = 0.39 0.41 on robust models) indicate that gradient magnitude patterns transfer sufficiently to guide efficient attacks even when exact loss surfaces diverge.
Additionally, on high resolution ImageNet models, SGSA demonstrates superior efficiency: DenseNet (c = 0.1), SGSA requires 55.85 queries versus PBO's 100.67, a 44.52% reduction, while achieving comparable success rates (93.99% vs 93.33%). This performance gap highlights a fundamental scalability advantage: saliency-based guidance directly exploits spatial locality in high dimensional image spaces, whereas GP- basod function approximation faces increasing complexity with dimensionality. The computational overhead of Bayesian optimization (kernel matrix inversions, acquisition function optimization) grows substantially in higher dimensions, while SGSA's gradient-map computation scales linearly with inpul. size.
6.2 SEGI
TASI: Quality of Initial Candidates
Our results consistently show TGEA-SEGI achieving higher attack success rates than TGEA-TASI across most experimental settings. This performance gap stemss from fundamental differences in how the two meth- ods generate imitial candidate populations. TASI constructs its population by combining multiple predefined attack strategies (PGD, Square Attack, Boundary Attack) with Gaussian noise perturbations, essentially performing a diverse but undirected sampling around known attack vectors. In contrast, SEGI employs a mela-optimization process that runs a complete Genetic Algorithm on the surrogate model for G generations, evolving candidates specifically toward the adversarial objective through iterative selection, crossover, and mutation. This surrogate- based evolution allows SEGI to explore the adversarial subspace more systemati- cally, producing a population that has already undergone fitness based refinement before being transferred to the target model. Critically, SEGT's fitness function incorporates both diversity penalty term Raiv (Equation 9) and centroid distance term Deentroid that encourages movement away from the true class rep- resentation in latent space while approaching incorrect class centroids. These components ensure the evolved population maintains coverage of promising regions rather than converging prematurely, providing CMA-FS with a well-distributed initialization that captures the geometric structure of the adversarial manifold while being positioned near decision boundaries. While TASI benefits from the complementary strengths of di- verse attack types, SEGI's evolved candidates represent gemuinely optimized starting points that have been tailored to the surrogate's loss landscape, which, given the mcasured transfcrability correlations, provides better initialization for atlacking architecturally similar target models.
6.3 Model Architecture and Attack Performance
Our ImageNet experiments (Table 5) reveal architecture-dependett performance patterns. SGSA achieves its strongest results on DenseNet (93.99% ASR, 55.85 queries at e = 0.1), while TGEA-SEGI performs best on EfficientNel (89.10% ASR al 0.05).
We observe that models with stronger feature locality (DenseNet's dense connections) tend to exhibit higher saliency transferability, benefiting local search methods like SGSA. Models with more distributed repre- sentations (EffficietNee compound scaling) require the global exploration capabilities of TGEA-SEGT to consistently find adversarial examples. However, we emphasize that these are empirical observations rather than definitive causal relationships; a complete understanding would require systematic landscape analysis beyond the scope of this work.
6.4 Practitioner Guidance
Based on our experimental results, we provide the following cvidence-based guidance for practitioners. For applications with a strict query budget, we recommend SGSA, which averaged fewer queries than bascline methods. However, for scenarios where high attack success rate (ASR) is the priority, TGEA-SEGI
16

---

Published in Transactions on Machine Learning Research (03/2026)
is the better choice, achieving higher ASR (e.g., 98.36% on InceptionV3). In the context of adversarial training. SGSA demonstrated superior robustness against defenses (e.g.. on WRN-94-16). Finally, for simple datasets (e.g., MNIST), both methods are highly effective, with SGSA offering a slight advantage in both query efficiency and ASR.
7 Limitations and Future Work
While PriSM demonstrates strong empirical performance, we do not provide formal theoretical guarantees on query complexity or convergence rates. Our design rationale (Sections 4.2.4 and 4.3.5) offers intuitive justification based on transferability assumptions, but rigorous analysis would strengthen the theoretical foundation. Nevertheless, our empirical results across diverse settings provides substantial evidence for the practical effectiveness of our approach.
Future work could explore: (1) adaptive surrogate selection strategies that estimate transferability online, (2) meta-learning approaches to automatically tune hyperparameters per target model, (3) theoretical char- acterization of when saliency-guided local search outperforms global optimization, and (4) extension other atlack settings (e.g., targeted attacks, different threat models).
8 Conclusion
We introduced PriSM, a framework that leverages surrogate decision boundary geometry and loss landscape topography to guide black-bot box attacks. Through TGEA and SGSA, we demonstrated that aligning search strategies with specific surrogate priors reduces query costs by 30-60% across MNIST, CIFAR-10, and Ima- geNet. Our evaluation reveals a strategic trade off: TGEA maximizes success rates via global evolutionary exploration, while SGSA excels in efficiency through saliency-guis guided local refiement. Notably, SGSA out- performs baselines on adversarially trained models, effectively exploiting the perceptually aligned gradients of robust networks. PriSM demonstrates that strategic exploitation of surrogate information matched to search algorithu characteristics substantially improves query efficiency in black-box robustness evaluation.
Broader Impact Statement
This work develops query-effeiient adversarial attacks for black-box settings. Our primary motivation is to evaluate and ultimately improve the robustness Deep Neural Networks (DNNs) deployed in safety-critical domains such as medical diagnostics and aulonomous systems. We recognize that the techniques we st study could, in principle, be misused evade deployed models with fewer queries, particularly against produc- tion API-based services (c.g., cloud ML platforms, content moderation systems) where existing rate limiting may prove insufficient. We explicitly do not endorse such misuse. To mitigale these risks, we recommend multi-layered defenses including: (1) query pattern detection to identify structured exploration, (2) response randomization to reduce information leakage per query, (3) adaptive rate limiting based on suspicious ac- tivity, and (4) ensemble diversity to reduce cross-model transferability (Tang et al., 2024]. Systematically probing systems for failure modes through responsible red-teaming is a cornerstone of security practice. By demonstrating that surrogate priors substantially reduce query costs, our results highlight the urgent nced for defenses that reduce the transferability of decision- geometry and saliency patterns. PriSM enables developers to stress-test models against sophisticated, anoource-constraine adversaries, identify weaknesses before deployment, and iteratively build more robust learning systems.
References
Moustafa Alzantot, Yash Sharma, Supriyo Chakraborty, Huan Zhang, Cho-Jui Hsieh, and Mani B. Sri- vastava. Genattack: practical black-box attacks with gradient- free optimization. In Proceedings of the
Genetic and Evolutionary Computation Conference, GECCO '19, pp. 1111-1119, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450361118. doi: 10.1145/3211707.332217 URL httpss//abl..rr//1..11/////// 33217449
17

---

Published in Transactions on Machine Learning Research (03/2026)
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: A query-efficient black-box adversarial attack via random search. In Andrea Vedaldi, Horst Bischol, Thomas Brox, and Jan-Michael Frahm (eds.), Computer Vision ECCV 2020, pp. 484 501, Cham, 2020. Springer International Publishing. ISBN 13.---9.--
Brian R. Bartoldson, James Diflenderfer, Konstantinos Parasyris, and Bhavya Kailkhura. Adversarial ro- bustness limits via scaling-law and human-alignment studies, 2024. URL https://axxiv.. http::////aai.ooo/aaa///0/0000 org/abs/2404. 09349
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models, 2018. URL http::://aaxiv.rrg/////
04248
Zikui Cai, Chengyu Song, Srikanth Krishnamurthy, Amit Roy-Chowdhury, and Salman Asif. Blackbox attacks via. surrogate ensemble search. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Newral Information Processing Systems, volume 35, pp. 5348-5362 Cur- ran Associales, Inc., 2022. URL httpss//proceeddggs nwwrips..pppp.... www..ps..ccppppeeeeeeeeeee files//ppprr/0022//1 22554444000iaaaaeeaaaaaaaaaaaaaa
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE
Symposium on Security and Privacy (SP), pp. 39 57, 2017. doi:
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang, Unlabeled data improves adversarial robustness. In H. Wallach, H. Larochelle, A. Beygelzimer, F. V'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Newral Information Processing Systems, volume 32. Cur- ran Associates, Inc., 2019. URL ttctps://proceediiiee
32000144774333300211111666111113 Paper, paf.
Jianbo Chen, Michael I. Jordan, and Martin J. Wainwright. Hopskipjumpattack A query-efficient decision- based attack. In 2020 IEEE Symposium Security and Privacy (SP), pp. 1277-1294, 2020. doi: 10.
(U09/SP.0000022200000444
Jinghui Chen and Quanquan Gu. Rays: A ray searching method for hard-label adversarial attack. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '20, pp. 1739 1747, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 450379994. doi: 10.1145/3394 339444863343222 URL htttt::///ddt http::////o.ooo//..../ 1145/33944886 3403225
Pin-Yu Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsich. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the
10th ACM Workshop Artificial Intelligence and Security, AlSec '17, pp. 15 26, New York, NY, USA,
2017. Association for Computing Machinery. ISBN 9781450352024. doi: 10.1145/3128572.3144444 URL https://doi. 10. 1101111//11877.3140000
Shuyu Cheng, Yibo Miao, Yinpeng Dong, Xiao Yang, Xiao-Shan Gao, and Jun Zhu. Efficient black-box adversarial attacks via bayesian optimization guided by a function prior, 2024. URL https://arxiv.
org/abs/2405. 19098
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: standardized adversarial robustness benchmark, 2021. URL httpss///axxi /2//00///000......///
Zeyu Dai, Shengcai Liu, Qing Li, and Ke Tang. Saliency attack: Towards imperceptible black-bor adversarial altack. ACM Trans. Inlell. Syst. Technol, 14(3), April 2023. ISSN 2157-6904. doi: 10.1145 10.1145/3582563. 33525663 URL https:///oi.ooo org/10. 10/ 1145/3582563
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE Conference on Computer Vision and Patlern Recognition (CVPR), June 2018.
18

---

Published in Transactions on Machine Learning Research (03/2026)
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2021. URL
https://aarsi http://aaxxvv....aaa// 2010. 11929
Ilya Figotin. Imagenet-mini. Accessed: 2025-10-20.
10000 2019.
Junjie Fu, Jian Sun, and Gang Wang. Boosting black-box adversarial attacks with meta learning. In 2022 41st Chinese Control Conference (CCC), pp. 7308-7313, 2022. doi: (CA19//CCC56666222222
lan Goodfellow, Jonathon Shlens, and Christian Szegedy, Explaining and harnessing adversarial examples, 2015. URL https://arxxv..rrg/ hts://bs//444/ wttp://wwwwwr////11...... 6572
Chuan Guo, Jacob Gardner, Yurong You, Andrew Gordon Wilson, and Kilian Weinberger. Simple black-box adversarial attacks. In Kamalika Chaudhuri and Ruslan Salakhutdinov (cds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Muchine Learning Research, pp. 2484-22220 2493. PMLR, 09-15 Jun 2019. URL https://procceedingg/
mlr ppwss//97/guo// http//uuo19aaaaa..
Nikolaus Hansen. The cma evolution strategy: A comparing review. In J. A. Lozano, P. Larrañaga, 1. Inza, and E. Bengoetxea (eds.), Towards a New Evolulionary Compulation, volume 192 of Studies in Fuzziness and Soft Computing, pp. 75-102. Springer, Berlin, Heidelberg, 2006. doi: 10.1007 540-32494--1 4.
Nikolaus Hansen and Andreas Ostermeier. Completely derandomized adaptttin in evolution strategies. Evolutionary Computation, 9:130-195, 06 2001. doi:
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
Warren He, James Wei, Xinyun Chen, Nicholas Carlini, and Dawn Song. Adversarial example defense: Ensembles of weak defenses are not strong. In 11th USENIX Workshop Offensive Technologies (WOOT 17), Vancouver, BC, August 2017. USENIX Association. URL httpp::/wwwe uswnix.ee.............. wrrpiiooooroooofeeceeee.. poott//oookksoo-ppprrraaapeeeeee
Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q Weinberger. Denscly connected convolu- tional networks, 2018. URL https://arxavvoorr/aaa///// 06993
Liang-Jung Huang and Tian-Li Yu. Taga: a transfer-based black-box adversarial allack with genetic al- gorithms. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '2, pp. 712-720, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450392372. doi: 04.145/355122903555560 URL https://abi.orr//1.. 0ttpg///...14///122900.. 356869
Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, and Pieter Abbeel. Adversarial attacks on neural network policies, 2017. URL http:::/aarxxv..og/abb///// 02284.
Yi Huang and Adams Wii Kong. Transferable adversarial attack based integrated gradients, 2022. URL
Andrew Nlyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In Jennifer Dy and Andreas Krause (ods.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Reseorch, pp. 2137 2146. PMLR, 10-15 Jul 2018. URL https::/procceeddngg mlr. ppres//80/ilyassttt hhmm
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In F. Percira, C.J. Burges, L. Bottou, and K.Q Weinberger (eds.), Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012. URL http::/tt///
19

---

Published in Transactions on Machine Learning Research (03/2026)
Xiaohui Kuang, Hongyi Liu, Ye Wang, Qikun Zhang, Quanxin Zhang, and Jun Zheng. A ema-es-based adversarial attack on black-box deep neural networks. IEEE Access, 7:172988-722947, 2019. doi: 10.1109/ ACCESS.2011.29550
Yaupei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks, 2017. URL attps://arxiii httpp:://arxxw.or///bb//////////
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards doep learning models resistant to adversarial attacks, 2019. URL
wwt..orr//bbb////.// 06083
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Practical black-box at Lacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, ASIA CCS '17, pp. 506 519, New York, NY, USA, 2017. Association for Computing Machinery. ISBN 9781450349444. doi: 10. 10.1455/305297333330000 URL https:
Huy Phan. huyvophan/pytorch cifar10, January 2021. URL
000..5528//zenooo.44441033333 4431043
Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli. Adversarial robustness through lo- cal lincarization.
In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garneut (eds.), Advances in Neural Informalion Processing Systems, volume 32. Curran Associates, Tnc., 2019. URL bberdd33eceee000100cccc000eeeeee Papp pdf
http://prooeeedd....euurrrereeee pttppp/sse.ppppppeeeeee///pppppp hiless/paprr/219/EEiII
Hao Qiu, Leonardo Lucio Custode, and Giovanni Iacca. Black-box adversarial attacks using evolution strate- gies. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO '21, pp. 1827-1833, New York, NY, USA, 2021. Association for Computing Machinery. ISBN doi: 10.1145/8444772634663333
URL https: / http://...rr//1..1//4444//////11
3463137
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition, 2015. URL
Sampriti Soor, Alik Pramanick, Jothiprakash K, and Arijit Sur. A generative adversarial approach to adversarial attacks guided by contrastive language-image pre-trained model, 2025. URL https://arxil:
org/abs/2511 01317
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 23(5):828 841, 2019, doi: 10.1109/TEVC.
2019.2890858.
Fnu Suya, Jianfeng Chi, David Evans, and Yuan Tian. Hybrid batch attacks: Finding black- box adversarial examples with limited queries. Tn 29th USENIX Security Symposium (USENIX Security 20), pp. 1327- 1344. USENIX Association, August 2020. ISBN 978-1-939133-17-5 URL httpp:///rrr uswnix.e.......... org/ con.lrrraeeeennnnnssseeeettorooa
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks, 2014. URL httpps//aaxii wwweexivv..og//aae org/abs//////oooooooooooooe. /1tss//11..... 6199
Christian Szegedy, Vincent Vanhoucke, Sergey Toffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pallern Recognition (CVPR), June 2016.
Mingxing Tan and Quoc Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov (ods.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 6105-6114. PMLR, 09-15 Jun 2019. URL https://proceedings mlr. www.sppreessetaaaaeeeeeee wtt.///9t/ttt....a// html
20

---

Published in Transactions on Machine Learning Research (03/2026)
Bowen Tang, Zheng Wang, Yi Bin, Qi Dou, Yang Yang, and Heng Tao Shen. Ensemble diversity facilitates adversarial transferability. In 2024 TEFE/CVF Conference on Compuler Vision and Paltern Recognation (CVPR), pp. 24377 24386, 2024 doi: (ALL10//CVPR52273.22220002
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Je- gou. Training data-elficient image transformers & distillation through attention. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning. vol- ume 139 of Proceedings of Machine Learning Research, pp. 10347-10357. PMLRR, 18-24 Jul 2021. URL https://proceediing mlr. wtt.:m.www....//////e/ 2Tess/v139/touvron21. html
Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Bonch, and Patrick McDaniel. The space of trans- ferable adversarial examples, 2017. URL https://arxiv wttp://wwwwr///a///...00.0
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, lan Goodfellow, Dan Boneh, and Patrick McDaniel. 07204
Ensemble adversarial training: Attacks and defenses, 2020. URL http:::///xxvv...aaaaaaa
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy, 2019. URL
12152
Yixuan Wang, Wei Hong, Xueqin Zhang, Qing Zhang, and Chunhua Gu. Boosting transferability of ad-
versarial samples via saliency distribution and frequency domain enhancement. Knowledge- Based Sys- tems, 300:112152, 2024. ISSN 0950-7051. doi: https://doo.orr/100016//// http:///..rrrr......oooooaoa.aa0 202111.002..112155. URL
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L. Yuille. Improving transferability of adversarial examples with input diversity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Meixi Zheng, Xuanchen Yan, Zihao Zhu, Hongrui Chen, and Baoyuan Wu. Blackboxbench: A comprehen- sive benchmark of black-box adversarial attacks. IEEE Transactions on Pallern Analysis and Machine Intelligence, (((9))786-77887) 2025. doi: 1AAL10//TPAMLL22255557444
A Hyperparameter and Configuration Details
This appendix details the specific hyperparameter values used for the proposed adversarial attacks. Table 12 outlines the coefficients used in the fitness functions, while Table 13 lists the specific algorithmic settings.
We explicitly verify the impact of these hyperparameter choices in Appendix B. Please refer to the Ablation Studies therein for a detailed sensitivity analysis and validation of the configuration values listed above.
Table 12: Fitness Function Coefficients. We distinguish between Target Optimization parameters (as, 7x) and Surrogate Initialization parameters (s) to highlight their distinct roles in exploitation vs. exploration.
Component
Parameter Symbol
Value
Or (Max Incorrect)
0.4
Target Opt.
7t (True Class Penalty)
1.1
0g (Max Incorrect)
0.3
(True Class Penalty)
1.3
Surrogate (SEGI)
8 (Centroid Distance)
0.9
Diversity Multiplier
10
Weight Asymmetry. Supported by Appendix B2, we employ asymmetric weights to balance exploration
and exploitation. SEGI uses a higher penalty (7+ = 1.3) and lower targeted weight (a, = 0.3) to enforce diversity, driving candidates away from the true class without converging a single easiest incorrect class. In contrast, the target stage employs balanced weights (7 = 1.1,a, 0.4) to facilitate stable convergence into missclassificatio.
21

---

Published in Transactions on Machine Learning Research (03/2026)
Table 13: Algorithmic Hyperparameters.
Component
Parameter
Value
Population Size
Parents Mating
Mutation Rate
10%
SEGI (GA)
Top Solutions + CMA-ES
13
Injection Noise (a)
0.035
Population Size
16
TGEA (CMA)
Max Itcrations
90
Active CMA Update
True
Saliency Scale Factor
10.0
Gaussian Sigma (7)
Update Frequency
50 iters
Fallback Threshold
100 iters
SGSA
Random Fallback Prob.
5%
Multi-scale Scales
[L.0, 0.5, 0.25)
Temporal Decay (A)
Attention Weight (1)
0.3
General
Max Query Budget
1000
B
Ablation Studies
To validate the contributions of key components in PriSM, we perform targeted ablation studies both TGEA and SGSA. All ablations use the same evaluation protocol with CIFAR-10 10 and ImageNet datasets 0.1 unless otherwise specified. Best results per metric are highlighted with gray shading.
B.1 Transfer-Guided Evolutionary Attack (TGEA)
B.1.1 Analysis of Surrogate Fitness Function
We ablate individual terms SEGI's fitness function: ass Yo. and 8. Results appear in Table 14
Table 14: SEGI fitness ablation on CIFAR-10 (e
Table 15: SEGI fitness ablation on ImageNet (e 0.1).
0.1).
InceptionV3
VGG16
NenseNet.
EfficientNNet
Variant
ASR 1
AQ
ASR J.
AQ
Variant
ASR f
AQ 4
ASR
AQ 4
Full Function
91.43
105.26
89.80
127.82
Full Function
89.47
111.25
89.10
148.12
O,
91.37
110.77
89.48
139.58
0, = 0
83.83
92.79
83.81
133.13
% =
90.18
107.11
86.93
136.
7. = 0
75.40
80.89
82.54
138.31
8 ==0
88.39
97.10
87.30
124.15
8
85.19
112.37
86.79
160.83
As expected, in the untargetted attack scenario, the 7s term proves most critical, its removal (7+ 0) causes the largest ASR drops across all models (up to 14% on DenseNet). This validates the importance of aggressively suppressing the true class during surrogate evolution. The alpha, and delta terms provide smaller but consistent improvements, with occasional query efficiency gains when removed at the cost of reduced ASR.
In addition to evaluating the impact of removing these terms entirely, we conducted grid search to optimize their specific magnitudes. We varied as, Ys, and 8 across the range of values ag {0.8, 1.5), 7 E (0.5, 1.0} and 5 E {0.5, 1.5} to verify that our chosen asymmetric configuration provides a good balance between exploration and exploitation compared to other feasible settings.
The analysis reveals that the default configuration outperforms the grid-searched variants across all models. Notably, increasing a, to 0.8 or 1.5 (which aggressively rewards incorrect class confidence) leads to slight degradation in ASR and increased query costs. This validates our intuition. The surrogate evolution stage (SEGT) should prioritize exploration via lower a, and higher 7s, ensuring diverse population of candi-
22

---

Published in Transactions on Machine Learning Research (03/2026)
Table 16: SEGI fitness grid search (InceptionV3 & VGG16).
Table 17: SEGI fitness grid search (DenseNet Ef- ficientNet).
InceptionV
VGCI6
DenseNet
NeficienNeet
Pazams
AST
AQ
ASR
AQQ L
Parrns
ASR T
AQ2
ASR 7
XQ.
0.3, 13,6 (Delaal))
91.45
105.26
89.80
127.82
113,6 =016=000 (Delaul)
89.47
111.25
89.10
148.-2
0, -.- 088,% 1.0.6
88.88
122.55
83.12
165.03
0.8%% 1.0,6 1.5
57.34
127.37
86.83
178.97
1.-08,%% 110,6- 0.5
89.15
118.40
83,25
158.90
0.8.% 1.0,6
0.5
58.10
124.10
87.50
169.20
6, 0,=088,% -0..6
85.04
127.20
84.12
184.75
0.8 7 0.5,6
1.5
86.22
128. 52
85.88
178-12
8, 0, -0.56=
83.50
178.40
0.8 0.5, 5
8, = 1.5,7 =0.5,6
87.50
119.50
$3,80
180.20
1.5. 7 1.0,6
0.5
69.05
123. 83
89.01
163. 09
9, 1,5.7. -0.,6
86.90
a, 15,7
= =0.6=0.5
89.99
115 23
83.92
175.73
1.5. 7, 0.5.6 6 1.5
66.80
130 50
86, 601
176,80
o. 15.%
88.91
120.31
172.16
1.5.%
178.5
dates rather than greedily optimizing for a specific incorrect label, which better handled by the subsequent CMA-ES phase.
B.1.2 Analysis of Target Fitness Function
To determine the optimal balance for the target fitness function, we performed grid search over the hyperparameters a, E {0.1, 1.0, 1.5] and yr E 100, 1.6). We compare these variations against the default settings, results are presented in Table I8
Table 18: Target fitness grid search (InceptionV3 & VGG16).
Table 19: Target fitness grid search (DenseNet & EfficientNeet..
InceptionV33
VGG16
DenseNet
Ethiient
Params
ASR 1
AQ
ASR
AQ
Params
ASR
AQ
ASR
AQ
On 0.4.
1.1 (Defauit)
91.43
105.26
89.80
127.82
0.1.%
1.1 (Default)
89.47
111.25
89.10
148.42
= 0.1,7 1.0
88.39
119.22
83.89
176.01
0.1. 1.0
85.42
105.73
81.82
135.44
0, 1.0,% 1.0
89.09
115.69
84.64
173.22
0r 1.0.% 1.0
93. 15
132.10
89.17
147.05
a = 1.5,7 = 0.5
89.08
138.19
83.85
175.08
dr 1.5, 7 = 0.5
80.73
105.38
88.11
165.29
0 0.1,7
88.75
175.50
0, 0.1, 0.5
88.20
114.50
84.50
140.10
0, 1.5, 1.6
89.12
119.95
84.72
173.10
0r 1.5.
1.6
93.22
132.80
89.25
148.50
0.1.7 1.6
87.95
116.51
83.20
177.50
0.1, 1.6
82.50
102.10
78.40
129.80
1.0,7 1.6
88.85
119.43
84.35
174.40
1.0, 1.6
90.80
124.60
87.65
142.30
6. 1.0,7 0.5
89.05
136.32
84.22
174.80
1.0, 7 0.5
86.40
116.20
88, 65
158.40
0. 1.5 1.0
89.15
121.59
84.50
173.90
dr 1.5, 7V 1.0
89.50
122.90
88.90
152.10
The grid search results highlight distinct. behaviors across architectures. For InceptionV3 and VGG16. the default scttings (o: 0.4, % 1.1) remain superior, achieving the highest ASR and lowest query counts. This suggests these models benefit from a balanced approach where the larget class is encouraged moderalely.
In contrast, the more complex architectures, DenseNet and EfficientNet, perform better with higher scaling factors (ar 2 1.0). DenseNet specifically achieves a +3.7% boost in ASR (93.22%) when using ar = 1.5, 7 = 1.6 compared to the default. This indicates that for highly connected or optimized networks, the attack requires stronger gradients towards the target class to successfully guide the optimization out of local minima.
B.1.3 Analysis of Population Size
Table 20 evaluates CMA-ES population sizes: 6 (small), 16 (default), and 26 (large).
On 10, smaller populations excel in both metrics, suggesting SEGI's initialization enables rapid convergence on simpler datasets. On ImageNet, the default size achieves highest ASR while small populations remain most query-efficient indicating a complexity dependent trade off where larger search spaces benefit from moderate population sizes to avoid premature convergence.
23

---

Published in Transactions on Machine Learning Research (03/2026)
Table 20: SEGI population size on CIFAR-10 (e 0.1).
Table 21: SEGI population size on ImageNet (e 0.1).
InceptionV33
VGG16
DenseNet
EfficientNet
Pop. Size
ASR f
AQ +
ASR 1
AQ |
Pop. Size
ASR 1
AQ (
ASR t
AQ +
Small (6)
92.99
90.27
92.22
117.48
Small (6)
87.04
94.90
87.45
126.60
Default (16)
91.43
105.26
89.80
127.82
Default (16)
89.47
111.25
89.10
148.42
Large (26)
84.70
128.02
81.51
161.28
Large (26)
85.23
176.10
86.40
134.93
B.1.4
Analysis of Computational Overhead Suliency-Guided Square Attack (SGSA)
B.2
B.2.1
Analysis of Guidance Components
We evaluate two ablations: Two-Scale variant using scales [1.0, 0.5] and Single-Scale (original resolution saliency only). Table 22 shows results.
Table 22: SGSA guidance ablation on CIFAR-10
Table 23:
SGSA guidance ablation on ImageNet
(e (e=0.) 0.1).
(e 0.1).).
33
VGG16
DenseNet
Efficient
Variant
ASR f
AQ 4
ASR 1
AQ 4
Variant
ASR f
AQ 4
ASR f
AQ 4
Default
85.64
53.47
82.85
77.62
Default
93.99
55.85
86.99
89.56
Two-Scale
84.77
58.80
81.80
68.89
Two-Scale
92.81
76.34
87.11
82.02
Single-Scale
79.06
57.60
78.12
77.81
Single-Scale
94.25
59.84
85.86
98.63
Single-scale saliency underperforms across metrics (e.g., +10% queries on Efficient.Net). The two-scale vari- ant shows competitive performance, but the default threc-scale configuration achieves the best overall bal- ance, validating that multi-scale aggregation captures transferable structural patterus multiple feature hierarchies.
B.2.2 Analysis of Saliency Scale Factor
We compare scale factors 2, 5, and 10 (default). Results in Table 24
Table 24: SGSA scale factor CIFAR-10 (e = 0.1).
Table 25: SGSA scale factor ImageNet (e = 0.1).
InceptionV3
VGG16
DenseNet
Efficient
Scale
ASR
AQ
ASR 1
AQ +
Scale
ASR t
AQ J
ASR
AQ
Delault (10)
85.64
53.47
82.85
77.62
Default (10)
93.99
55.85
86.99
89.56
Factor 5
79.50
57.53
78.39
73.59
Factor
93.75
59.20
86.17
92.29
Factor 2
80.00
57.48
77.86
69.31
Factor
93.95
59.32
85.24
91.53
Lower scale factors occasionally reduce queries (e.g., factor 2 on VGG16) but consistently harm ASR. The default factor of 10 achieves highest or near highest ASR universally, demonstrating that stronger smoothing generates more stable and transferable guidance maps.
B.2.3 Analysis of saliency update frequency
We evaluate the sensitivity of the attack to the saliency update frequency k, comparing the default (k = 50) against more frequent (A: E (10, 25)) and lazier (k = 100) schedules. This parameter rons the trade-off between using fresh gradient information and the computational overhead updating the surrogate priors.
The results indicate that the default update frequency (A: 50) provides trade-off between the freshness of the surrogate priors and the computational overhead of updating them. While lazier updates (k 100)
24

---

Published in Transactions on Machine Learning Research (03/2026)
Table 26: Update frequency & on CIFAR-1O.
Table 27: Update frequency & on ImageNet.
InceptionV33
VGC16
DenscNet
Efficient
Value
ASR 1
AQ 4
ASR 1
AQ 4
k Value
ASR
AQ
ASR
AQ +
10
85.16
68.83
81.97
72.59
10
93.09
62.62
86.25
73.75
25
85.94
53.55
81.56
70.72
25
92.50
90.07
87.12
101.04
50
(Default)
85.64
53.47
82.85
77.62
50
(Default)
93.99
55.85
86.99
89.56
100
86.28
56.30
82.35
67.49
100
92.81
62.07
83.71
117.46
occasionally improve efficiency on simpler landscapes, they risk working with old gradient information more complex decision boundaries, leading to increased query costs. Overly frequent updates (k 10) yield dininishing returns, often increasing computational cost without significant gains in attack success rate.
B.2.4 Analysis of attention weight
We analyze the impact of the attention guidance weight 3, which controls the influence of the surrogate's attention map on the perturbation direction. We compare the default $ 0.3 against weaker (B 0.1), stronger (B (0.5,0.75)) guidance, and attention (3=0).
Table 28: Attention weight 3 on CIFAR-10.
Table 29; Attention weight 3 on ImageNet.
InceptionV3
VGG16
DenseNet
Etficient
Value
ASR f
AQ
ASR f
AQ
3 Value
ASR
AQ 4
ASR t
AQ +
0 (No Attention)
79.06
59.02
78.72
73.30
0 (No Attention]
93.85
58.75
85.37
92.30
0.1
84.90
50.58
82.81
74.05
0.1
93.42
58.82
88.02
89.49
0.3
(Default)
85.64
53.47
82.85
77.62
0.3 (Default)
93.99
55.85
86.99
89.56
85.36
46.97
81.25
61.32
0.5
93.17
61.77
101.86
0.75
85.41
56.73
81.77
64.51
0.75
93.30
58.39
88.02
90.00
Removing attention (B 0) degrades ASR significantly on CIFAR-10 (-6.6% on InceptionV3), confirming that attention maps provide complementary guidance beyond gradient-based saliency. While increasing attention guidance improve query efficiency and ASR specific architectures, the default B 0.3 provides the most stable performance across all models, particularly for DenseNet.
B.2.5 Analysis of Temporal Decay
We evaluate the scnsitivity of the temporal mechanism the docay factor X, which determines how much historical gradient information retained. We compare the default X 0.9 against lower values representing faster decay.
Table 30: Temporal decay X on CIFAR-10.
Table 31: Temporal decay on ImageNet.
InceptionV3
VGG16
DenseNet
EfficientNet
Value
ASR
AQ
ASR
AQ 1
À Value
ASR f
AQ ]
ASR f
AQ J
0.1
84.38
61.22
81.25
69.05
0.1
94.12
62.83
88.54
90.27
0.5
83.04
57.41
80.80
67.99
0.5
94.49
61.80
87.50
89.70
0.75
85.58
53.10
82.20
71.82
0.75
93.33
53.74
87.00
92.69
0.9 (Default)
85.64
53.47
82.85
77.62
0.9 (Default)
93.99
55.85
86.99
89.56
The results suggest that a higher decay factor () R 0.9) is generally efficient, balancing ASR and query efficiency. While lower values (c.g., = 0.1) occasionally yicld marginal ASR gains (as seen in EfficientNet), they often result in higher query costs or reduced stability on other architectures (e.g., VGG16), justifying the choice of x= 0.9 effectively smooth the gradient estimation over time.
25

---

Published in Transactions on Machine Learning Research (03/2026)
B.3 Computational Overhead
Saliency-Guided Square Attack (SGSA)
To quantify the cost of surrogate guidance, we measured the wall clock time of SGSA against the baseline Square Attack on 1000 samples. Table 32 summarizes the computational overhead introduced by multi-scale saliency map generation and guidance operations.
Table 32: Computational overhead of SGSA saliency guidance.
Dataset
Total Time (s)
Pure Search (s)
Saliency Overhead
CIFAR-10
1.85
1.70
0.15 (8.1%)
ImageNet 5.23
5.12
0.11 (2.16%)
The saliency overhead remains minimal, accounting for only 8.1% CIFAR-10 and 2.16% Image Crucially, this overhead is amortized across the entire attack budget. Since the guidance substantially reduces the total number of queries needed, the net wall clock time decreases despite the per-iteration cost, confirming that surrogate guidance provides practical computational efficiency.
Consfer-Guided Evolutionary Attack (TGEA)
To quantify the computational cost of transfer-guided evolutionary optimization, we measured the wall clock time of TGEA's components on 1000 samples. Tables 33 and 34 show the breakdown for each variant.
Table 33: TGEA-SEGI computational profile (seconds per attack).
Dalaset
Random Population Time (s)
Total Time (s)
Population Init (s)
Attack (s)
CIFAR-10
3.29
6.34
4.24
2.10
ImageNet
19.36
48.68
32.53
16.14
Table 34: EGEA-TASI computational profile (seconds per attack).
Dataset
Total Time (s)
Population Init (s)
Attack (s)
CLFAR-10
5.34
3.21
2.13
ImageNet
22.27
6.80
15.47
TGEA introduces computational overhead for transfer-based population initialization, which accounts for 67% of total time for SEGI on CIFAR-10 and 67% for SEGI on ImageNet. For TASI, initialization overhead is lower at 60% on CIFAR-10 10 and 15% on ImageNet.
Overall, the overhead of the attacks must be evaluated in context: both attacks achieve substantially higher allack success rales while using fewer queries. In real-world settings where query budgets are strictly limited (e.g., production APIs with rate limiting, detection systems, service costs), the ability to succeed with fewer queries is often more valuable than raw wall-clock time. The modest computational investment (8% for SGSA, 1.9-2.5x for TGEA) enables attacks that would otherwise fail entirely within the query budget, making both methods highly practical for security evaluation and adversarial robustness testing.
B.4 Robustness Surrogate Model Selection
To assess PriSM's sensitivity to surrogate choice, we evaluate SGSA and SEGI using GoogLeNot and DenseNet as surrogates (vs. default ResNet18), attacking InceptionV3 and VGG16 on CIFAR-10 at 0.1.
Results in Table 35
26

---

Published in Transactions on Machine Learning Research (03/2026)
Table 35: Impact surrogate model on SGSA and SEGI performance (CIFAR-10, c = 0.1).
InceptionV3
VGGI6
Method
Surrogate
ASR t
AQ 4
ASR
AQ 0
ResNet18 (default)
85.64
53.47
82.85
77.62
SGSA
GoogLeNet
85.99
53.66
82.33
75.03
DenseNet
85.41
52.27
83.05
70.67
ResNet18 (default)
91.43
105.26
89.80
127.82
SEGI
GoogLeNet.
88.73
121.38
84.69
167.73
DenseNet
88.48
119.55
86.38
151.74
Both methods exhibit robustness across surrogate architectures, with SGSA demonstrating particularly sta- ble performance. SGSA maintains consistently high ASR and low query counts regardless of surrogate choice, with DenseNet achieving the lowest query count on InceptionV33 (52.27). This stability is remarkable given the architectural diversity tested, SGSA's 's query efficiency varies by less than 50% across surrogates, whereas traditional transfer attacks can see 2-3x degradation with mismatched architectures. SEGI shows slightly more sensitivity with 3-5% ASR variation, though ResNet18 performs best overall. Notably, performance degradation remains modest even with architecturally dissimilar surrogates (e-g.: GoogLeNet's depthwise- separable convolutions vs. VGG16's standard convolutions), validating that multi-scale saliency patterns transfer robustly across diverse architectures. The consistency suggests that readily available pretrained models can be used surrogates without substantial performance loss, with SGSA being especially resilient to surrogate selection.
C Attack Convergence Analysis
To provide a deeper insight into the trade off between query efficiency and attack success rate, we visualize the convergence behavior of SGSA and TGEA against baseline methods. The following plots illustrate the Attack Success Rate (ASR) as function of the number of queries.
C.1 MNIST Models
Figure 7 shows the convergence on MNIST for both 0.2 and 0.3.
Kich
(a) Model A (e 0.2)
(b) Model B (c 0.2)
Decra Tata 154
Osas Budget
Okay Brdget
(c) Model (e = 0.3)
(d) Model B (E = 0.3)
Figure 7: MNIST Models: Convergence analysis for c = 0.2 and c <= 0.3.
27

---

Published in Transactions on Machine Learning Research (03/2026)
C.2
CIFAR-10 10 Standard Models
Figure compares performance on InceptionV3 and VGG16.
Budget
(a) InceptionV3 (e = (.1)
(b) InceptionV3 (c 0.2)
Ouay Sudget
Jusy Sudget
(c) VGG16 (e 0.1)
(d) VGG16 (e= 0.2)
Figure 8: CIFAR-10 Standard Models: ASR Average Queries.
C.3 CIFAR-10 Robust Models
Figure g
displays results on adversarially trained networks. Notably, on the SOTA WRN-94-16, SGSA matches or exceeds the success rate of baselines while maintaining superior query efficiency.
Sc0mm
2000
38
Ouary Budgett
960
180
30
Duary Dudget
800
180
(a) WideResNet (e= 0.1)
(b) WideResNet (f= 0.2)
WRN-94416 (e 0.1)
(d) WRN-94-16 (c= 0.2)
Figure 9: CIFAR-10 Robust Models: ASR vs. Average Queries.
28

---

Published in Transactions on Machine Learning Research (03/2026)
C.4
ImageNet Models
Figure 10 illustrates performance on high resolution ImageNet models. SGSA (brown) consistently demon-
strates steeper convergence curves in the early query phase.
55SA
Qvary Eudget
800
130
30
Quary Budget
802
180
(a) DenseNet (e = 0.05)
(b) DenseNet (e = 0.1)
Quary Sudget
Juayy Sudget
(c) Efficient (c= 0.05)
(d) Efficient (e 0.1)
Figure 10: ImageNet Models: ASR Average Queries.
29


Reviews:
[{"review_id": "SHEoGbFSxY", "reviewer": "Reviewer_xB8V", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes Prior-guided Search Methods, PriSM, a framework for improving query efficiency in black-box adversarial attacks by systematically exploiting transferable priors obtained from surrogate models. The key idea is to distinguish different types of transferable information and to match them with proper search strategies. The paper introduces two attack methods: Transfer-Guided Evolutionary Attack (TGEA) and Saliency-Guided Square Attack (SGSA). TGEA uses surrogate-informed initialization to warm-start CMA-ES, and SGSA incorporates multi-scale saliency information from a surrogate model into Square Attack. Experiments on the image benchmark datasets show improved reductions in query usage, while maintaining competitive attack success rates.\n\n[Strengths]\n\n* The paper presents a decomposition of transferable priors and argues that different priors should be exploited by different optimization methods.\n* The focus on query efficiency in the adversarial black-box setting is timely and practically relevant.\n\n[Weaknesses]\n\n* A large number of hyperparameters are introduced, yet the empirical analysis of their sensitivity is limited.\n\n* The conceptual relationship between TGEA and SGSA within a single unified framework remains somewhat unclear. The methods currently appear more like parallel cases studies than tightly integrated components.\n\n* SGSA relies heavily on Square Attack and its implicit CNN inductive bias, raising concerns about generalization to non-convolutional architectures such as vision transformers.", "claims_and_evidence_yn": "No", "claims_explanation": "The empirical results partially support the claim that surrogate-guided priors can reduce query counts in black-box attacks across several datasets and models.\n\nHowever, several central claims are not fully supported by convincing by convincing evidence. In particular, although the paper emphasizes the robustness and general applicability of its methods, systematic sensitivity analyses of the many introduced hyperparameters are largely absent from the main paper. The appendix mainly contains ablation studies that toggle components on or off, rather than exploring performance trends across a range of parameter values. This makes it difficult to assess tuning cost, stability, and reproducibility.", "audience_yn": "Yes", "audience_explanation": "The problem of query-efficient black-box adversarial attacks is of clear interest to the community, particularly researchers working on robustness, security, and black-box optimization. The idea of explicitly categorizing transferable priors and aligning them with specific search mechanisms offers a useful perspective that goes beyond incremental algorithmic tweaks. Moreover, the paper’s focus on practical query budgets and its evaluation on standard benchmarks suggest that its findings would be relevant to both academic and applied audiences.", "requested_changes": "[Major]\n\n* The paper introduces many hyperparameters, yet their influence is not systematically analyzed.I strongly believe that the main manuscript should include sesitivity analysis of key parameters over meaningful ranges, rather than relegating limited ablations to the appendix. In particular, it would be important to analyze how optimal hyperparameters choices vary across datasets and target models.\n\n* While the paper argues that different priors call for different exploitation strategies, the unifying principle of PriSM remains underdeveloped. The authors could more clearly articulate: 1) why specific priors are matched to specific search algorithms, 2) whether the two methods are complementary or mutually exclusive, and 3) how a practitioner should choose between them under different constraints.\n\n* Since SGSA is fundamentally based on Square Attack and spatial saliency, and the paper itself notes the connection to CNN inductive bias, it is critical to test SGSA on transformer-based models (e.g., ViT). If SGSA performs poorly, the paper need to analyze why and discuss possible adaptations.\n\n\n[Minor]\n\n* The citation style appears inconsistent with common conventions. Currently, nearly all citations place author names explicitly in the main text. (may need to change citet to citep in latex)\n\n* In Section 3.2, the definition of the \"Project\" function used in the Square Attack formulation appears to be missing.\n\n* Across several figures, the font size of text inside the figures is too small to be easily readable.\n\n* While Figure 1 provides an overall intuition for surrogate-based initialization, it does not clearly explain why TGEA is expected to move toward a global optimum. Strengthening the intuition would help readers better understand why improved initialization leads to performance gains.\n\n* It is unclear whether the parameters $\\alpha$, $\\gamma$ used in the surrogate-specific fitness function are intended to be identical to those used in the target model fitness function. If they are the same, a justification would be helpful.\n\n* The motivation for TGEA appears to be closely related to margin-based considerations. However, the fitness function assigns different weights to the two terms in the margin, and the rationale for this asymmetry is not clearly explained. Providing theoretical or empirical justification for using different weights would improve the interpretability of the method.", "broader_impact_concerns": "The proposed methods lower the query cost of black-box adversarial attacks, which could increase the risk of practical misuse against deployed systems with limited query budgets. While the paper acknowledges defensive motivations, the Broader Impact discussion could be strengthened by more concretely addressing realistic attack scenarios (e.g., API-based services) and by discussing mitigation strategies such as rate limiting, response randomization, or attack detection mechanisms.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "FHQVq8Q9e4", "reviewer": "Reviewer_KPep", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes **PriSM (Prior-Guided Search Method)**, a novel framework that incorporates adversarial transferability to enhance black-box adversarial search. Specifically, PriSM exploits two types of transferable surrogate information: decision boundary geometry and loss landscape topology.\n\nThe authors further demonstrate the importance of these transferable priors by introducing two complementary attacks:\n(1) **TGEA (Transfer-Guided Evolutionary Attack)**, which leverages boundary information, and\n(2) **SGSA  (Saliency-Guided Square Attack)**, which leverages loss topography.\n\nFinally, the authors evaluate their attacks on the MNIST, CIFAR-10, and ImageNet datasets. The results demonstrate the effectiveness of the proposed methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "To the best of the reviewer's knowledge, the claims made by the authors are accurate, convincing, and clearly presented. The justification provided for the proposed TGEA and SGSA methods appears to be sound.", "audience_yn": "Yes", "audience_explanation": "The reviewer strongly believes that the TMLR's audience would be very much interested  in the findings of this paper.", "requested_changes": "As black-box adversarial searches are notoriously computationally expensive, the reviewer is concerned about the additional computational overhead that the TGEA and SGSA methods may incur. The authors provide only vague and minimal information in the Appendix regarding the computational costs of their methods. While the proposed approaches improve attack success rates, it remains unclear why and how they would be practical in real-world settings, given that incorporating adversarial transferability into black-box searches may further increase computational overhead.\n\nThe reviewer recommends that the authors include one or more tables reporting the time required to generate successful adversarial examples using TGEA and SGSA, and compare these times to those of the selected baselines. These experiments could be conducted using CIFAR-10 and/or ImageNet.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ebZ2Lifo3p", "reviewer": "Reviewer_z2u9", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes the PriSM method to balance the query efficiency and attack success rate under black-box attack constraints.\n\nThe main contributions include \n(1) Transfer-guided evolutionary attack (TGEA), which initializes CMA-ES (Covariance Matrix Adaptation Evolution Strategy) with the transfer-based priors of the surrogate model, with\nTASI (Transfer-Attack Seeded Initialization) for generating diverse initial candidates from multiple types of attacks to the surrogate model;\nAnd SEGI (Surrogate-Evolved Genetic Initialization) for evolving a high-quality initial population.\n\n(2) Saliency-guided Square Attack (SGSA), which uses multi-scale gradient maps from a surrogate model to guide perturbation placement and sizing.\n\n(3) Evaluation on standard benchmarks demonstrate the effectiveness of the method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The major claims are well-supported by extensive experiments, ablation studies, and intuitive justifications:\n\n(1) the strategic trade off: TGEA improving ASR and SGSA reducing query costs are consistently supported by Tables 2-9; \n\n(2) Competitive with state-of-the-art hybrid attacks is supported by the comparisons with PBO;\n\n(3) Robustness to surrogate choice is supported by Table 21, Appendix B.3, where SGSA and SEGI tend to maintain stable performance across three CNN-based surrogate models.", "audience_yn": "Yes", "audience_explanation": "The paper is highly relevant to TMLR's readership.\n\n(1) Adversarial robustness is an important topic in secure and reliable machine learning.\n\n(2) Query-efficient black-box attacks are important for real-world applications.\n\n(3) The paper demonstrates a new way to balance query efficiency and attack success rate, with extensive evaluations and ablations.", "requested_changes": "(1) Please clarify the baseline choices. The latest method for comparison is PBO, which was published in 2024. Please justify why the newly proposed methods are not involved. Please also justify the selected defense strategies.\n\n(2) While Table 10 of Appendix A lists all hyperparameters, a discussion of the sensitivity would strengthen the reproducibility.\n\n(3) Please add a discussion of potential misuse and societal implications.", "broader_impact_concerns": "The paper does not present a broader impact statement, yet the method (the topic of black-box attacks) may have potential misuse and societal implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)