QuadAttac$K$: A Quadratic Programming Approach to Learning Ordered Top-$K$ Adversarial Attacks
Authors: Thomas Paniagua
Ryan Grainger
Tianfu Wu
Venue: NeurIPS 2023 poster (2023)
Keywords: Ordered Top-K Clear-Box Targeted Adversarial Attack
Deep Neural Networks
Quadratic Programming
Robustness
TL;DR: a quadratic programming based formulation to address the ordered top-k adversarial attack problem

Abstract: The adversarial vulnerability of Deep Neural Networks (DNNs) has been well-known and widely concerned, often under the context of learning top-$1$ attacks (e.g., fooling a DNN to classify a cat image as dog). This paper shows that the concern is much more serious by learning significantly more aggressive ordered top-$K$ clear-box targeted attacks proposed in~\citep{zhang2020learning}. We propose a novel and rigorous quadratic programming (QP) method of learning ordered top-$K$ attacks with low computing cost, dubbed as \textbf{QuadAttac$K$}. Our QuadAttac$K$ directly solves the QP to satisfy the attack constraint in the feature embedding space (i.e., the input space to the final linear classifier), which thus exploits the semantics of the feature embedding space (i.e., the principle of class coherence). With the optimized feature embedding  vector perturbation, it then computes the adversarial perturbation in the data space via the vanilla one-step back-propagation. In experiments, the proposed QuadAttac$K$ is tested in the ImageNet-1k  classification using ResNet-50, DenseNet-121, and Vision Transformers (ViT-B and DEiT-S). It successfully pushes the boundary of successful ordered top-$K$ attacks from $K=10$ up to $K=20$ at a cheap budget ($1\times 60$) and further improves attack success rates for $K=5$ for all tested models, while retaining the performance for $K=1$.

QuadAttack: A Quadratic Programming Approach to Learning Ordered Top-K Adversarial Attacks
Thomas Paniagua, Ryan Grainger and Tianfu Wu Department of Electrical and Computer Engineering, NC State (tapaniag, rpgraing, twu19)@ncsu. edu httpa:://thonaapaaiaaaaa github. io/ hooo/quaatttckk_eeeeeeee
Abstract
The adversarial vulnerability of Deep Neural Networks (DNNs) has been well- known and widely concerned, often under the context of learning top-1 attacks (e.g., fooling a DNN to classify a cat image as dog). This paper shows that the concern is much more serious by learning significantly more aggressive ordered top-K clear- box targeted attacks proposed in (Zhang and Wu 2020]. We propose a novel and rigorous quadratic programming (QP) method of learning ordered top-K attacks with low computing cost, dubbed as QuadAttacK. Our QuadAttack directly solves the QP to satisfy the attack constraint in the feature embedding space (i.e., the input space to the final linear classifier), which thus exploits the semantics of the feature embedding space (i.e., the principle of class coherence). With the optimized feature embedding vector perturbation, it then computes the adversarial perturbation in the data space via the vanilla one-step back socc--roppggtion. In experiments, the proposed Atta/ is tested in the ImageNet- k classification using ResNet- 50, DenseNet- 121, and Vision Transformers (ViT-B and DEiT-S). It successfully pushes the boundary of successful ordered top-K attacks from K 10 up to 20 at a cheap budget (1 60) and further improves attack success rates for K 5 for all tested models, while retaining the performance for K = 1.
Introduction
As the development of machine deep learning and Artificial Intelligence (AI) continues to accelerate,
the need to address potential vulnerabilities of Deep Neural Networks (DNNs) becomes increasingly crucial towards building safe-enabled and trustworthy learning and Al systems. Among these
vulnerabilities, adversarial attacks [Xie et al. 2017 Kos et al. 2018 Sharif et al. 2016 Ebrahimi
et al., 2018 Qin et al., 2019, Lin et al.] 22017 Papernot et al. [201 Liu et al., 2017
Xie et al.
2019b Dong et al., 2019) Goodfellow et al. 2. that can almost arbitrarily manipulate the prediction of a trained DNN for given testing data by learning visually imperceptible perturbations, especially under clear-box target attack settings, are of particular interest in terms of revealing the shortcut learning of diseriminativeel-traae DNNs Geirhos et al., 2020). Clear-box targeted adversarial attacks are often learned under the top-1 attack setting with the objective of causing catastrophic performance drop of the top-1 accuracy on adversarial examples against the clean counterparts.
al.) 2015 Kannan al.) 2018 Madry et al.
2017 Xie
To generalize the vanilla setting of learning top-1 attacks, much more aggressive ordered top-K attacks have been proposed Zhhang and Wu 2020), which aim learn adversarial examples that manipulate trained DNN to predict any specified K attack targets in any given order as the top-K predicted classes. As shown in Fig. 5 in the ImageNet- 1k [Russakovsky et al. 2015 classification
This is often referred as white/black- box attacks in the literature. We choose to adopt neutral terminology, clear/opaque- box attacks in this paper, and omit the prefix bocar-box for simplicity.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Clean Image (agama)
Clean Attention
Adv. Example
Adv. Perturbation
Adv. Attention
DISTAS
VJI-B3
Adv. Example
Adv. Pesturbation
Example
Adv. Perturbation
RewNet-Si 50
DenseNer-122
Figure 1: Adversarial examples Icarned by our QuadAttacK (K = 20) for a same clean image using four different networks: two popular convolutional neural networks (ResNet-50 [[e al. 2016) and DenseNet-121 Huang al. 2017)), and widely used Vision Transformers (ViT-B Dosovviskiy al. 2020 and DEiT-S [Touvron et al. (02T). The ground-truth label agama. The ordered top-20 targets (randomly sampled and kept unchanged for the four models) are: [sea cucumber, ringneck snake, snail, tiger shark, Pembroke, altar, wig, submarine, macaw, combination lock, ram, Irish wolfhound, confectionery, buckle, chime, garden spider]. For example, the top-20 classes of the clean image by DEiT-S are lagama, frilled lizard, shovel, reel, common iguana, Yorkshire terrier, coho, alligator lizard, hand blower, meerkat, ostrich, mongoose, fiddler crab, eft, wing, bustard, green lizard, whiptail. brass, American chameleon], which have more or less visual similarities. More examples are the Appendix
barrow, odometer, bloodhound, min-of-the-woor
task, consider clean testing image of 'Agama' that can be correctly classified by some trained DNNs such as the four networks we tested, let's 's say a desired list of the ordered top-20 attack targets as shown in the caption. An ordered top-K attack method would seek to find the adversarial perturbation to the input image that manipulates the DNN's predictions to ensure that the top-20 predicted classes of the perturbed image match the specified list in the given order.
Why to learn ordered top-K attacks? They facilitate exploiting the principle of "class coherence" that reflect real-world scenarios where the relative importance priority of certain classes is crucial to recognize the relationships or logic connecting classes within ordinal or nominal context. Unlike unordered top-K or top-1 attacks, ordered top-K attacks can subtly manipulate predictions while maintaining coherence within the expected context.
An Ordinal Example. Imagine cancer risk assessment tool that analyzes 2D medical images (e.g.. mammograms) to categorize patients' cancer risk into the ordinal 7-level risk ratings (/Exiremely High Risk, Very High Risk, High Risk, Moderate Risk. Low Risk, Minimal Risk, No RiskJ), An oncologist could use this tool to triage patients, prioritizing those in the highest risk categories for immediate intervention. An attacker aiming delay treatment might use an ordered top-3 adversarial attack to change prediction for patient initially assessed as Very High Risk. They could target the classes [Moderate, Low; Minimal], subtly downgrading the urgency without breaking the logical sequence of risk categories. An unordered attack, in contrast, might lead to a sequence like Loww Very High, Minimal], disrupting the ordinal relationship between classes. Such a disruption could raise red flags, making the attack easier to detect.
A Nominal Example. Traffic control systems could use deep learning to optimize flow by adjusting the timing of traffic lights based on the types of vehicles seen. Priority might be given to certain vehicle classes, such as public transit or emergency vehicles, improve response times. Imagine a city's 's traffic control system, which has specific traffic light timing behavior for the nominal vehicle categories /Emergency Vehicle, Public Transit, Commercial Vehicle, Personal Car, Bicycle]. Public transit might be given slightly extended green lights during rush hours to encourage public transportation use. An attacker wanting to cause delays for personal cars without raising alarms could launch an ordered top-2 adversarial attack, targeting the sequence /Commercial Vehicle, Public Transit]. This would cause the system to interpret most personal cars as commercial vehicles during the attack, applying the extended green light times meant for public transit to lanes primarily

---

Adversarial Penturbation
A A-)( 1H9,
Sumegga lots C1R
The pelor (ng, AdvDietilty
Chean
16, M
Ordered Top-K Targets
Chemp
Fascee Buckhaze
Fuaz
Chana
c [0. 10.1
A B1
C R
IC R
TargetK
Adveesazial Porturbation
A- 118
Falla
Aol inear constia loghs
Progamming
Quodiatic
Our QuadAttacK
Figure 2: Illustration of the proposed QuadAttack method in comparison with the prior art (e.g., the adversarial distillation (AD) method [Zhang and Wu 2020)..
used by commercial vehicles. An unordered top-2 attack that may result in (Emergency Vehicle, Commercial Vehicle]. would likely be quickly detected, as emergency vehicle priority changes are significant and could be easily noticed by traffic operators (this weakness is exacerbated any top-1 attack or unordered attacks).
Successful ordered top-K attacks can potentially provide several advantages: enabling better controllability in learning attacks that are more difficult to detect, revealing deeper vulnerability of trained DNNs, and testing the robustness of an attack method itself, especially when K relatively large (e.g., K 2 15) and the computing budget is relatively low (c.g., 60 steps of optimization).
Learning ordered top-K attacks is an extremely challenging problem. The adversarial distillation method [Zhang and Wu [020] is the state of the art method (see Sec. Sec.(3.T) which presents a heuristic mnowledge-orienter design of the ordered top- K target distribution, and then minimizes the Kullback- Leibler divergence between the designed distribution and the DNN output distribution (after softmax). It often completely fails when K >>>>>>>> 10. In this paper, we show it is possible learn those attacks for a variety of DNNs, including ResNet-50 [He et al., (2066). DenseNet-1e [Huang et al. 2017 and Vision Transformers [Dosovitskiy et al. 2020 Touvron al. 2021].
The key to learning clear-box targeted attacks (K 2 1) lies in the objective function for optimization, which usually consists of two terms, one is the lp norm (c.g., E2) of the learned adversarial perturbation (to be as small as possible to be visually imperceptible), and the other is the surrogate loss capturing the specified attack constraints such as the top-K extended C&W (hinge) loss [Carlini and Wagner [017] and the adversarial distillation loss proposed in [Zhang and Wu 2020) (see B.1p. The trade-off off between the two terms are often searched in optimization with respect to a certain computing budget (e.g.. 9 x 30 means to test 9 different trade-off parameter assignments based on linear search in predefined range, and run 30 forward-backward computation iterations of the DNN per trade-off parameter search step). After the optimization, Attack Success Rates (ASR, higher is better) and some l, norms (e.g.. & and E2) of learned successful perturbations (smaller is better) are used as evaluation metrics. As illustrated in Fig. 2 in this paper, we propose novel formulation which is different from the prior in the three aspects follows:
We identify that while sufficient to capture the top-K K attack constraint, hand-crafted surrogate losses are not necessary and often introduce inconsistency and artifacts in optimization (see Sec. 3.2). We eliminate the need of introducing surrogate losses. Instead, we keep the top-K attack constraints in the vanilla form and cast the optimization problem quadratic programming (QP). We solve the QP by leveraging a recently proposed differentiable QP layer (for Py Torch)) [Amos and Kolter 2017]. We present efficient implement parallelize the batched QP layer for any ordered top-K targets specified individually for each instance in a batch.
We observe that directly minimizing the l, norm of learned perturbations together with the hand- crafted surrogate loss could miss the chance of exploiting semantic structures of the feature embedding space (i.e., the input to the final linear classifier). Instead, we minimize the Euclidean distance between the feature embedding vectors at two consecutive iterations in the optimization. This can be understood as the latent perturbation learning versus the raw data perturbation learn- ing(see Sec. 3.3). Our proposed latent perturbation learning enables more consistent optimization trajectories in pursuing the satisfaction of the specified top-K attack constraints. The minimized Fuclidean distance is then used as the loss together with the Ep norm of the learned perturbation in computing the adversarial perturbation via back pock-propagation at each iteration.
3

---

2 Related Work and Our Contributions
Adversarial Attacks have remained as a critical concern for DNNs, where imperceptible perturba- tions to input data can lead significant misclassification [Xie al.
2017 Hendrik Metzen et al.)
2017 Chen et al. 2019 Liu et al. 2018). Various approaches have been proposed to investigate the vulnerabilities of DNNs and exploit their sensitivity to non-robust features. Notable works include the seminal discovery of visually imperceptible adversarial attacks (Szegedy et al. 2013), which highlighted the need to evaluate the brittleness of DNNs and address it with explicit defense methods [Madry et al.] 2017 Cohen et al. 2019 Wang et al.) [020 Wong et al.] 2020]. Clear-box attacks, which assume full access to the DNN model, have been particularly effective in uncovering vulnera- bilities [Madry et al. 2017]. The C&W method [Carlini and Wagner 2017]. for instance, introduced
loss functions that have been widely adopted to generate adversarial examples. Mommenuum-asse methods like (Dong et al. 2018 Lin et al. 2019] and gradient projection techniques like PGD [Madry ct al. 2017] have also been successtul in crafting adversarial examples.
Unordered top-K Attacks aim to manipulate the top- K predicted classes of a DNN without enfore- ing a specific order among them. A common approach is to optimize loss function that combines multiple objectives, such as maximizing the probability of the target classes while minimizing the probabilities of other classes simultaneously. (Tursynbek et al., 2022] presents geometry inspired method that allows taking gradient steps in favor of simultaneously maximizing all target classes while maintaining a balance between them. Other approaches, for example, may use sorting strategies Kuumano et al. 2022] to limit the set of logits involved simultaneously in loss and target specific logits that contribute most to a misclassification. The "Superclass" attacks have been proposed in [Ku- mano et al. 2022]. for which ordered top- K attacks can been seen as generalization. Non-targeted
top-K attacks are studied in (Zhang et al. 2022
Ordered top-K Attacks (Zhang and Wu 2020 require preserving both the attack success rate and the order of the top- K predicted classes. Following the intuitions from [Kumano et al., 2022 we note that adversarial sample detection and defense methods [Lee et al. [0188] Huang and Lij 2021
Aldahdooh et al. [022] may benefit from the fact that many adversarial attacks tend to generate a nonsensical class prediction (e.g. the top-K K predictions may all be from a different super-class
Kumano et al. ]. An ordered top- K adversarial attack may choose to create an attack that is semantically plausible and have higher potential to fool detection methods and defense methods. The sorting constraint in ordered top-K attacks adds layer of complexity to the optimization problem. By enforcing a specific order among the top-K K predictions, the attacker must not only manipulate the logits to maximize the target classes' probabilities but also ensure that the predicted order aligns with the desired order. In /hhan and Wu 2020). two methods are presented through the use of semantic information in the form of class language embedding.
Constrained Optimization in DNNs, such as OptNet (Amos and Kolter tiable optimization works (Agrwwal et al., 2019 Butler and Kwon| 2003). have introduced powerful formulations for integrating optimization layers within the network. These types of works have produced many DNN based problem solutions while also being able to use domain knowledge to constrain solutions and reduce data requirements (Sangalli et al. 2021] and solve problems otherwise intractable with DNNs [Wang et al. 2019 Mandi et al. 2020 While OptNet's quadratic solver is typically used as a layer, our focus is attacking trainrdined model with fixed architecture. Thus, we leverage the principles of constrained optimization to formulate an objective function that captures the constraints of ordered top- adversarial attacks. This adaptation allows us guide the attack process to satisfy the ordering constraints while maximizing target class probabilities, which provides new insights for enhancing attack effectiveness and robustness.
2017 and other differen-
Our Contributions This paper makes two main contributions to the field of learning clear-box targeted adversarial attacks:
It presents quadratic programming (QP) approach to learning ordered top- K attacks, dubbed as QuadAttacK. It eliminates hand-crafting surrogate loss functions to capture the top-K attack constraint. It provides QP based formulation to better exploit the semantics of the feature embedding space in capturing the top-K K attack requirement.
It obtains state-of-the-art adversarial attack performance in ImageNet- Ik classification using both ConvNets (ResNet-50 and DenseNet- 121) and Transformer models (ViT-B and DEiT-S). It pushes the limit of the number of targets, K to a large number, for which the prior art completely fails.

---

3 Approach
In this section, we first define the problem of learning ordered top- K attacks following Zhang and Wu 2020]. well as the extended top-K C&W (CW*) method and the adversarial distillation (AD) method. We then present details of our proposed QP based formulation, i.e., QuadAttacK.
3.1 Learning Ordered top-K Clear-Box Targeted Attacks: the Problem and the Prior Art
We consider image classification DNNs, which consist of feature backbone and head linear classifier. Denote by F I E R3x W + 2 E Px the feature backbone which transforms an input image to its feature map, where input image is often RGB image of the spatial height and width, H and W (e.g., 224 x 224), clean or perturbed, and the feature map D-dim feature space with the the spatial height and width, h and w (e.g.. 7 7) based the overall spatial downsampling stride implemented in the feature backbone. Let 2 E IR" be feature embedding vector for æ via any spatial reduction method (e.g. global average pooling). Denote by f 2 E R E RC the linear head classifier, where C' is the number of classes (e.g., 1000 in ImageNet- Ik [Russakovsky et al. 2015) ) and / is the output logit vector. We have, [= (F(2: 0); A,
in
(I) where 0 collects all the model parameters of the feature backhone, and A E CRCD and B E RC the weight and bias parameters the head linear classifier. learning clear-box attacks, we assume all the information of the network are available, and the parameters (0, A, B) are frozen throughout.
A B,
Denote by (z. pair of clean image and its ground-truth label (y E y (1,2, CD). For learning attacks, we assume z can be correctly classified by the network, i.e., Denote by T the ordered list of attack target(s) with the cardinality K [T). where the ground- truth label is excluded, the adversarial perturbation be learned. An ordered top-K adversarial example is defined by + 8(x,T; F, f) if the top-K prediction classes for d equal to T based on its logits ( Eqn.
arg max, 4.
& T, and by 7 = - T the complement set. Let 0(x, T; F) be
Learning ordered top-K K attacks is often posed as a constrained optimization problem, minimize subject to
||0|pp
(2)
h, > h
i E [1, - -1], 4 ETT, tk ET, VjETs
fex > 1
a x 88 10,
i= (FF;;:):
where the first two constraints capturing the ordered top- K attack requirement. This traditional formulation leads to the challenge in optimization, even with K = 1 as pointed out in the vanilla C& W method [Carlini and Wagner 2017]. Some surrogate loss, L(²), necessary to ensure the first two terms are satisfied when [() is minimized. We have,
minimize
X L() |101pp
subject to - 8 E [0, 10,13xxx.w
where À is the trade-off parameter between the visual imperceptibility of learned perturbations and the ASR. The remaining constraint can be addressed via a projected descent optimization. So, the optimization can enjoy the straightforward back-propagation algorithm that is used in training the DNN on clean images.
The extended top-K C&W (hinge) loss Zhang and Wu 2020 is defined by,
K
K CK
(
max 0,
max
i min
i. 1n)
(4)
1=1
And, the adversarial distillation loss [Zhnn and Wu 2020 is defined by.
CK (+) = (ADD) = -E 1,E7
pr. (log p., - log PAD) ) +
Σ
py (log log PAD)), (5)
JET=
where و = Softmax ()) and PAD is the knowledge-oriented heuristically-desiinee adversarial distri- bution with the top-K K constraints satisfied (see [Zhang and Wu 2020) for details). KT(--I-) the Hilbbakk-eibble divergence between two distributions.
5

---

3.2 Limitations of the Prior Art
From the optimization perspective, we observe there are two main drawbacks in the aforementioned
formulations (Eqns. 5):
The two surrogate loss formulations (Eqns. 4 and 5 are sufficient, but not necessary. They actually introduce inconsistency and artifacts in optimization. The extended top- C&W loss in Eqn. F is not aware of, and thus can not preserve, the subset of targets whose relative order has been satishied For example, consider there are 5 classes in total, and specified ordered top-3 list of targets, [2,3, 1]. Assume at a certain iteration, the predicted classes for & in sort are [4, 2, 3, 5, 1], in which the relative order of the specified 3 targets has been satisfied. The loss LCw = 2m (44 4). which mainly focuses on pushing down the logit 2, and/or pulling up the logits, i (i 1,
So, at the next iteration, it may results in the sorted prediction like [1.3, 2, 5). leading to a totally wrong relative order. The adversarial distillation loss in Eqn. has similar problems, i.e., the first part, Lier P; (log Pr log P:AD) is not aware of some satisfied relative order. It further enforces order between non non-target classes since the adversarial distribution pAD needs to be specified before the optimization, as shown the second part, Ejer- Py (log log P:AD).
Directly minimizing ||0||p in Eqn. (i.e., adversarial learning in the data space) may actually hinder the effectiveness of learning adversarial examples due to the fact that the lp norm is totally unaware of the underlying data structures in the complex data space. Since trained DNN is kept frozen in learning attacks, we can first perform adversarial learning in the feature embedding space (i.e., the head linear classifier's input space, 2 in Eqn. [], which has been learned to be discriminatively and/or semantically meaningful. With a learned adversarial perturbation for 2, we can easily compute the perturbation for the input data.
We address these limitations in this paper by proposing novel QP based formulation - Quad/Attack.. We present the detail our QtadA K the following sub-sections.
3.3 The Proposed QtadAttacK
We first show that any specified ordered top-K attack requirements. T' can be cast as linear constraints compact matrix form, denoted by Dr. Consider E, norm of Eqn. 2 we can rewrite it as,
minimize || x112,
subject to
ii> 0, Dr (-1,0,,1)cc-1
$ = x ==++dE [0,e(01)) [0, 1|3
i (();
where Dr is a C x C matrix constructed from the specified targets T, and C' is the number of classes in total. Consider the aforementioned toy example where C' = 5 and the ordered top-3 attack targets are = 32,3, ; 1]. we have,
1 ig >
0
1
1
0
0
f > 0,
(7)
Dr 1ix00 0, Dr
1
0
1
1
0
0
-1
0
1 4, > 0,
0
1 i >0, 0,
(8)
where Eqn.
7 is the vanilla form of expressing the specified top-3 attack requirements, and Eqn. is
the equivalent compact matrix form.
However, Eqn. 6 can not be easily solved via QP due to the highly nonconvex nature of the feature backbone F' in the third term of the constraints. This nonconvexity hinders the ability solve the
problem and find an optimal solution while satisfying constraints. Eqn. aaim to directly seek the adversarial perturbations in the data space, which has to include the nonconvex feature backbone F in the optimization. Since F is a frozen transformation in learning attacks, and the head classifier f linear function, we can separate the learning of ordered top-K attacks in two steps:
We first satisfy the ordered top-K constraints without resorting to hand-crafted surrogate losses via QP in the feature embedding space (i.c., the output space of F and the input space of f). The last

---

constraint in Eqn. 6 will be replaced by 1 A- B (see Eqn. the current iteration. We have the current feature embedding vector 2 Mean(F 8)). our proposed QuadArtack K first solves the perturbed & via QP: minimize
Denote by ò the image perturbation
|| - 1--1 2
subject to DT 0,.00
A-
ii After having found å we then use the residual (Euclidean distance) || 2212 as the loss to find the perturbed image á. Specifically, with the optimized & (i.e., the closest point in latent space where top-K K attack constraints back- propaaropagttion with a learning rate y and a loss weighting parameter A. We find that both 7 or may be used control the tradeoff between attack success and magnitude of the perturbation found (see Fig. and Appendix
-
satisfied), we compute the image perturbation using vanilla one-step
A). We have,
8
(A 81 E|12 118112)
(10)
35
8 Clamp( + 0, 0) - 2 2= + 8,
where Clamp(-: 0, is an element-wise projection of the input onto [0,
Understanding Quad : In QuadAttacK, the matrix A plays a crucial role in encoding useful semantic class relationships. For instance, if classes like "cat" and "building" have distinct and separate representations in the latent space, the matrix A will reflect these differences. As result, the optimization problem would naturally prioritize target logits that have high activations for either "cat" or "building," but not both simultaneously. This semantic constraint in the feature embedding space helps guide the search towards relevant and meaningful perturbations that maintain the desired order constraints while avoiding conflicting activations for disparate classes. By incorporating the matrix A to capture semantic relationships, our QuadAttac K not only overcomes the nonconvexity challenge of the original problem but also leverages meaningful class information to guide the search and generate effective adversarial perturbations.
3.4 Fast and Parallel Quadratic Programming Solutions
An efficient solver crucial for addressing the QP formulation of learning ordered top- K adversarial attacks. To that end, a fast, parallel, and GPU-capable quadratic programming solver is required. In this context, the qpth package created by [Amos and Kolter 2017] emerges as a suitable choice, providing a TTorch-enabled differentiable quadratic programming solver that enables efficient optimization while harnessing the power of GPUs,
minimize subject to
(11)
G# $ h,
W = b.
The main challenge lies transforming our current formulation into one compatible with the qpth package (Eqn. IT]. This involves structuring the objective and constraints to match the required standard form, as well as building the required matrices Q, G and h from attack targets in an efficient and parallel way. Finding Q and is straightforward from an expansion of our squared Euclidean distance objective since ||a - 2 = a & 22 2 + E z. The term 2 2 is a constant in our optimization so we can just consider the optimization of 2T 22 2. From this
trivially Q = 21 where I is the identity matrix and = -28 Fuutherr since we need no equality constraints W = 0, b 0. To formulate G and h we can rewrite the constraints in Eqn Das follows,
(12) where 7) is the slack variable which is small non-zero constant to allow our constraints to define closed-convex set allowing equality in our constraint but still maintaining top-K order when the constraint is satisfied. We have found work. Intuitively if our current value for E does not satisfy our top- constraints then 1) = would find a on the boundary of the latent space that satisfies our constraints. The higher 7) is the further away from the boundary and inside the set becomes. From the above rearrangement we can easily
Dr (A& + B) > 0
-Dr A8 S Dr B 7,
0.2 is an acceptable value, but other small values will also
see G = -Dr A and h Dr B 17 in Eqn. 11

---

(a) DEIT-S and ResNet-50 w/ K 5 and 30
(b) DEÎT-S and ResNet-50 w/ K 10 and x 60
60
AB
60
0.3
32
15
Figure 3: ASR vs energy tradeoff curves, which holistically compare the capacity of our QuadAttacK against the prior art - the adversarial distillation method (Zhang and Wu, 2020), verifying our QuadAttack's advantages.
4 Experiments
In this section, we evaluate our QuadAttacK with K = 1,5, 10, 15, 20 in the ImageNet- 1 bench- mark [Russakovsky et al.) 2015] using two representative pretrained ConvNets: the ResNet-50 (He)
itt al. 2016) and the DenseNet-122 Huang et al. [007]], and two representative pretrained Transform- ers: the vanilla Vision Transformer (Base) Dosovitskiy al. 2020) and the data-efficient variant DEiT (small) [Touvron et al. 2021]. The ImageNet-Ik Ik pretrained checkpoints of the four networks from the mmpretrain package
Contributors 2023 in which we implement our QuadAttac
and re- produce both CW and AD Zhang and and
Wu 2020].
Data and Metric. In ImageNet-Ik [Russakovsky al. 2015]. there are 50, 000 images for validation. To study attacks, we utilize the subset of images for which the predictions of all the four networks are correct. To reduce the computational demand, we randomly sample smaller subset following (Zhang) and Wu 2020]: We iterate over all 1000 categories and randomly sample an image labeled with it, resulting in 1000 test images in total. For each K, we randomly sample 5 groups of K targets with ground-truth (GT) label exclusive for each image in our selected test set, and then compute the Best, Mean and Worst ASRs, as well as the associated e,, &2 and Eo energies. We mainly focus on low-cost learning of attacks (within 60 steps of optimization) for better practicality and better understanding of the underlying effectiveness of different attack methods. We also learn attacks with higher budgets (9 60 and 9 x 30) for K = 5, 10. We use default value for the trade-off parameter in Eqn. and Eqn. 10 Details are provided in the Appendix A and our released code repository.
Baselines. We compare our QuadAttac with previous maat--ff-the-ar methods, namely. the top-K extended CW method and the Adversarial Distillation (AD) proposed in (Zhang and Wu 2020). We reproduce them in our code repository and test them on the four networks under the same settings for fair comparisons.
An important detail in optimization: In optimization, perturbations are initialized with some small energy white Gaussian noise. During the initial steps of optimization, the optimizer takes steps with large increases in perturbation energy since happens to be away from many required energies for successful attack. These large increases in energy induces momentum in the optimizer, which makes it difficult to reduce L.2 energy in future iterations even if our objective function's 's gradient points towards direction with minimal energy. By introducing small number of warmup steps (e.g.. 5, as commonly done in training a network on ImageNet from scratch) after which the optimizer's s state is reset, we observe the performance of all analyzed methods are significantly improved. Our QuadAttacK benefits most.
Results. The comparison results are shown in Table 1 Our proposed QuadAttack retains perfor- mance comparably for K = 1. and obtains consistently better performance, often by a large margin, for K 5, 10, 15, 20. Especially, it addresses the challenges associated with large values of K (c.g. K > 10) under the low-cost budget setting (1 x 60). The prior art completely fails, while our QuadAttacK can still obtain appealing ASRs.
Analyses on the Trade-Off Between ASRs and Attack Energies. Since we have two metrics (ASR and attack energy), the trade-off between them needs to be compared in order to comprehensively understand different methods. The trade-off curves (Fig. 3 explore the concept of how a higher success rate may be achieved by choosing to have higher energies and conversely lower energy may be achieved by choosing to have lower success rate. They holistically compare the capacity of QuadAttack against the prior art the adversarial distillation method [Zhang and Wu 2020).
8

---

Protocal
Ataak Method
ASRT
6,
Best
1xc 4
ASRT
Meun
60
ASRT
6
Woest
ResNet-50
st..
2016
Top-20
QuadAttac pot
0.2760
2350.79
7.71
0.0941
0.2546
2363.38
7.74
0.0039
0.3420
2381.00
7.80
0.0936
Top-15
QuadAtiack 1920
QuadAttacK
0.9520
0.3540
2275.78
1900.29
7.31
6.45
00011
0.0763
0.9400
0.22N0
1957.25
227091
0.64
7.32
0
0.0710
0.290
013080
1963.93
2766333
6.46
7.28
020706
0,0767
AD
0.1150
1584.19
5.23
0.0887
0.1002
1576.03
5.21
0,0685
0.0880
1578.91
5.22
0,0682
QundAttacK
0,999
1538.45
5,08
0.8500
0.9982
153400
5,06
0.851
0.9970
1527.30
5.04
0,0559
Top-10
OundAttac
A73
0.9400
0.2210
1341
1789.45
4.49
5:75
0.0338
0.041
0.9776
0.2020
1350.55
1792-20
0,0443
0.841
09530
0.000
1795.44
1320.06
4.58
5:77
0.0637
0,0154
QuadAtfacK in
538.1
0,9992
540.04
1.94
0/433
537.46
0,0431
QundAttac pv:0
03950
5.55
00471
03925
305256
3.55
074470
00900
1025:70
3.33
0:0471
CW
0.2100
1577.22
4.40
0.0413
0.1934
1578. 16
4.40
0.0415
0.1800
1374.54
4.39
0.9414
AD
0.8480
1551.05
4.33
0.0899
08324
1552. 17
4.32
0.0300
0.1100
1357.45
4.34
0,039]
Top-5
QualAAtacR
0.9450
765.54
2.55
0.9289
0.9300
759.47
2.54
0
0.9300
757.77
2.53
0.0289
0.4470
5.25
00082
1020200
02055
0
T012210
3:40
0.0354
AD
0,9550
549.73
1.91
0.0322
0.350
553.08
0.034]
QundAttacK 2000
0.9970
462.81
0.0277
0.9948
465.20
1.61
0
0.9930
461.15
1.60
0,0277
1.0000
483.86
1,53
0.142
0.9978
483.5 88
1.53
0.0142
0.9960
485.16
1.54
0.0142
Top 1
AD
1,0000
465.62
1.48
0.0141
0.9990
467.67
1.48
0,8141
0,9980
469.54
1:49
0.0142
QuadAttac 1:30
1,0000
446.84
1.44
0.0143
0,9990
448.22
1.44
0.0143
0,9980
448.70
1.45
0.0144
DenseNet 121
[Luang et al. 2207
Top-20
QuadAttack
0:9310
2304 04
7.82
0.0907
0.9184
2388.01
7.50
0.7988
0.9070
2387.15
7,80
0.0008
QuadATiac p.10
0.3250
202642
000795
0.3204
2630.12
8.43
0.3164
8.44
Top-15
QuadMttucK uo
QuadATiack pel0
0.9910
03130
217690
1890.34
6.17
638
010644
0.0682
10000
0
2174.17
1882.11
6.7
00683
00044
0,9790
00020
0090901000000 1876.26
216797
6.15
6.95
0,0679
0.0039
0.1650
2088 38
6.74
D.0755
0.1388
2090.22
6.75
0.0500
0.1260
2082.13
6.73
0,0750
ADyyc
0,5200
1432 66
4.78
0.0650
0,110
1420.53
4.76
000466
0.4920
1429.76
4.77
0.0641
QuadAttack
0.9980
1N87.24
4.61
0.0495
0.9960
1395.19
4.63
0.049
0,9930
1392.97
4.62
0.8497
QuadAtiac pad0
2123
61471
00894
1626.75
5.24
3.27
014770
Top-10
0,2630
19
0.026
0,470
1895221
6.16
00724
0,4120
192.44
6.18
0.072
Al2yct
8,6830
1070.25
0.0588
06648
1073.88
3.67
0.0877
0,6460
1071 64
3.66
0.0587
QuadArtae avdd
0,9090
451.30
1.61
0.0394
0,9084
443.78
1.59
0.0393
0,9970
446.99
1.60
0.0399
AD
0,0910
1555,57
5:17
0.088
00778
1572.15
5.12
010578
0.6900
1572.70
5.13
0705S0
QuadAttac pwap
0,9990
838.91
2.84
0.0406
0.9956
841.88
2.85
0,0407
0.9930
843.84
2.86
0.0407
CWWff.
0.5560
1062.48
0.0376
0.5358
1064.42
3.46
0.0077
0.5130
1057,52
3.44
0.0378
ADivan
0,9120
716.61
2.46
0.0355
09040
218.59
2.45
0.8890
710.45
2.44
0.0354
Top
QuadAttuck 1wa
0,9400
676.94
2.28
0.0269
0,9284
680.24
2.28
0.0268
0.9130
677.48
2.28
0
D. 8650
690.5
2.97
0.0365
03566
M05.02
2.99
00364
0.430
505.64
2.59
0.034
ADDussn
0,9810
438,53
1.55
0.0526
0.9694
461.40
1.56
0.0399
0.9570
447.02
1.58
0,0032
QuadAttacK pu30
0,9980
427.58
1.47
0,0259
0,9932
431.84
1.49
0.0261
0.9900
424.47
1.46
0.0262
1,0000
437.41
0.0134
0.9988
442.44
0.0135
0.9980
443.43
1.41
0,0135
Top-1
ADy-nn
1,0000
422 10
1.35
0.0137
0,9990
426.98
1.36
0.0137
0.9970
128.25
1,37
0.0137
Quad 2,330
1.0000
377.99
1.25
0.0138
0.9990
383,13
1.26
0.0133
0.9970
386.19
1.27
0,0138
VIT-B Dosoviskly at
2020
Top-20
QuadAttack s9s0
0:6100
2651.86
8.68
0.0847
| 0.5964
2642.73
8.65
00041
0.5810
2640.33
8.65
0.0841
Top 15
QuadAttacK
0,7150
1919.22
6.28
0.0618
0,70622
1920.08
6.28
0.0618
0.6950
1927.03
6.30
0,0620
QuadAttack
0,8680
1582 15
5.09
0.0422
0.8480
1579.52
5.06
0.0121
0,8230
1595.84
5.13
0,0120
Top-10
QwadMilacK even
ADozen
0,9850
1301.90
3,36
011472
0,0363
016900
0,9802
978.89
1398335
0,0383
00000
0.9730
1300.
4.32
004476
Quad awd0
1300
330
00X30
119333
3.78
01006
01470
116.33
3.685
0.034
AD
0,2650
1076,10
3.42
0.0286
0.2450
1079.95
3.43
0.085
023200
1089.83
3.46
0,0285
QuadAtback
0.4960
1060.31
3,35
0J0267
0.4692
1062.17
3,35
0.9267
0.4380
1060.54
3.35
(J6267
Top-5
6,1110
999.13
3,18
0,0276
0.1070
954.000
3.14
0.0277
0,970
963.58
3,05
030275
ADoyaa
0,4760
947.39
0.0282
0,4580
950.24
3.07
0.4300
945,25
3,05
0,0282
QuadAttae avas
910.88
2.95
0.0264
0.75M)
922.78
2.98
0.0264
0.7390
945,00
0.0266
CWff.a
0,99.0
418.85
1.44
0.0162
0,9914
420.89
1.45
0.0163
0.9890
423.64
1,46
0.0164
Top-1
8:9910
418.44
1.46
0.0176
0.9898
414.87
1.45
0.0176
0.9870
410.74
1.43
0.0174
Quadktttac
0.9910
451,55
1.53
0.0169
0.9884
464.51
1.52
0.0166
0.9840
434.08
1.48
0.0167
DeTTS
Touvron et al
2021
Top-20
QuadAtttac 1o00
0,7670
2164.35
7.02
0.0707
0.7496
2170.06
7.04
0.0711
0.7320
2179.99
7.08
0.0713
1op-15
Quad 1.m
QuadAttac paa
0,9400
003090
1710.90
4.50
5.55
010079
0,0537
0.9008
1712.43
T41699
5.55
0.0351
0,0535
0,0410
0,9170
1706.33
1427.73
5.53
030379
0.0534
AD:x0
0.0100
1247.46
4.05
0.0130
1224.28
3.97
0.0119
0.0680
1225.32
0:0:21
Quad p.m
0.9800
1161.60
3.75
0.0356
0.9778
1163.31
3.76
0.038
0.9750
1168.52
3.78
0.0358
UuadAttack 1ol0
0,000
107648
3.1
030788
0.1933
1007.14
3.38
0.1275
0.1920
1002.29
3.36
0.0275
Top-10
0.0990
1080.92
5.34
0.0360
0.0334
1000.35
3.25
0.0352
8,0000
987.35 21
3.19
0.8350
A.Douco
0 1250
812,31
2.71
0.0056
0.1122
829.00
99
2.76
0,0356
0.1000
819.51
2.73
0.6350
QuadAttaak poom
0.9990
546.66
1.95
0.0294
0.9978
543.50
0,029
0.9950
542.95
1.95
0.0298
QuadAttacK ovaan
0.4410
1019.14
3.25
0.0274
0.4222
1033.34
3.29
0.0277
0.6090
1027.87
0.0275
CW
0.0730
924.23
2.94
0.0266
0.0670
913.38
2.91
0.0265
0,0620
900.37
2.88
0.0263
ADuua
0.4830
1010.58
5.20
0.0271
0.4640
1016.17
3.22
0.0272
0,4000
1019.77
3.23
0.0273
Top-5
CWV-2
QuadAAttac
0.2310
0.8250
909.93
$49.16
2.91
2.72
0.0268
0.6246
0.2204
0.9888
843.14
2.88
2.71
0.0256
0.0244
0.2160
0.9000
896.11
$46.60
2.87
2.72
0.0264
0.0245
A.Duyai
0,7510
858.41
2.75
0.0267
$36.38
0,0261
0.7080
2.72
0.0265
QuadAttack 9xc0
0.9550
592.60
0.0229
0.9528
579.32
1.96
0.0127
0.9500
573.48
1.94
0.0226
CWC
0.9990
417.62
1.39
0.0149
0.9976
415.49
1.38
0.0148
1.9970
410.32
1.36
0.0147
Top-1
ADusan
1,0000
407.16
1.57
0.0155
0.9982
24
1.36
0.0157
0.9970
397.20 99..99
1.34
0.0136
QuadAttaaR R sxan
0.9960
294,88
1,68
0.0153
0.5936
295.00
1,08
0.0153
0.9900
297.33
1.00
0.0154
Table 1: Comparisons under the ordered top- K targeted attack protocol using randomly selected and ordered K
targets (GT exclusive) in ImageNet using four popular models (ResNet-50, DenseNct-121, ViT-B and DFYT-S). The CW K and AD methods are proposed in [hang and Wul Wul 2020). We test x 30 for K = 5. and 9 (30, 60) for K = 10. For each protocol (e.g., top-20) and each budget (e.g., 1 x 60). if an attack method (c.g., CWK or AD) is not listed, it means that it fails completely, i.e., zero ASR, and thus is ommitted in the table for clarity.
9

---

Qualitative Results. We show more examples in Appendix B From those, we note that for our QuadAttacK, when the target classes deviate significantly from the original predicted classes, we often observe perturbation that achieves the prescribed top-K K targets without substantial margin between each of the top- targets. This outcome reflects the desirable effect of our approach, as the primary objective of the ordered top-K attack is to enforce a specific class order rather than optimizing for class probability differences. Our method's strength lies in its ability to handle such scenarios without relying explicit assumptions about the distances between classes. By prioritizing the order constraints, our QuadAttacK K offers a robust solution that aligns with the fundamental goal of enforcing class order in adversarial attacks.
Limitations of Our QuadAttacK
There are two main limitations that worth further exploring. Our proposed QuadAttacK is specifically designed for clear-box attack setting, which makes it not directly applicable to opaque-box attacks. The attacks learned by our QuadAttack K are not easily transferable between different networks, as they are specifically optimized in the feature embedding space for the target model. Exploring the transferability of attacks and developing more generalizable strategies across various networks could be an intriguing direction for future research. In addition, our QuadAttack entails solving QP at each iteration, which introduces additional computational overhead compared to methods like CW and AD [Zhang and Wu 2020). For example, for ResNet-50, we observed an average QuadAttac K performs 2.47 attack iterations per second whereas AD performs 32.02 iterations per second (a factor of 12.96). For ViT-B, QuadAttaK performs 2.96 attack iterations per second whereas AD performs 11.86 iterations per second (a factor of 4). We note that as the target model becomes larger, the adversarial loss constitutes a smaller fraction of total runtime thus the ratio tends toward one. Further, we note that quicker attack iterations of QuadAttack on ViT-B which indicate our QP solver converges faster on ViT-B attacks. To address the overhead of our QuadAtttck we will explore and compare how the QP solver could be adjusted to initialize the QP solver at the previous iteration's solution to nearly eradicate the cost of the QP solver in future work.
6 Broader Impact
Our proposed QuadAttacK method showcases the effectiveness of utilizing QP techniques in the challenging domain of learning ordered top-K clear-box targeted adversarial attacks. The underlying QP formulation offers opportunities for exploring other applications beyond adversarial attacks. For instance, it could be leveraged to design new loss functions going beyond the traditional cross-entropy loss or the label smoothing variant (Szegedy et al. 2013], and thus jointly optimizing accuracy and robustness. We can enforce semantically meaningful class orders in training network from scratch, thus allowing for the incorporation of explicit constraints in neural network predictions and potentially resulting
more interpretable and controlled decision process.
Potential Negative Societal Impact. As discussed in the introduction, there are some potential scenarios in practice for which the proposed ordered top- K adversarial attacks may be risky if applied. However, since we focus on clear-box attacks, they are less directly applicable in practice compared to opaque-box attacks, which makes the concern less serious.
7 Conclusions
This paper presents a quadratic programming (QP) based method for learning ordered top- K clear- box targeted attacks. By formulating the task as constrained optimization problem, we demonstrate the capability to achieve successful attacks with larger values of K (K > 10) compared to previous methods. The proposed QuadAttac K is tested in the ImageNet- 1k classification using ResNet-50 and 121, and ViT-B and DEÏT-S. It successfully pushes the boundary of successful ordered top- K attacks from K- = 10 up to K = 20 at a cheap budget (1 x 60) and further improves attack success rates for K = 5 for all tested models, while retaining the performance for K = 1. The promising results highlight the potential QP and constrained optimization as powerful tools opening new avenues for research in adversarial attacks and beyond.
10

---

Acknowledgements
This research is partly supported by ARO Grant W91INFI810229 ARO Grant NSF JSS-199644, NSF CMMI-2024688 and NSF IUSE-2013451 10141.. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed implied, of the ARO, NSF, the U.S. Gov- emment. The U.S. Government is authorized to reproduce and distribute reprints for Govermmental purposes not withstanding any copyright annotation thereon.
References
Akshay Agrawal, Brandon Amos, Shane Barratt, Stephen Boyd, Steven Diamond, and J Zico Kolter. Differentiable convex optimization layers. Advances in neural information processing systems, 32, 2019.
Ahmed Aldahdooh, Wassim Hamidouche, Sid Ahmed Fezza, and Olivier Déforges, Adversarial example detection for dnn models: A review and experimental comparison. Artificial Intelligence Review, 55(6):4403 4462, 2022.
Brandon Amos and J Zico Kolter. Optnet: Differentiable optimization layer in neural networks. In International Conference on Machine Learning, pages 135-145. PMLR, 2017.
Andrew Butler and Roy Kwon. Efficient differentiable quadratic programming layers: an admm approach. Computational Optimization and Applications, (4(2):449-470 2023.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium security and privacy (sp), pages 39-57. Icee, 2017.
Shang- Taseg-Tse Chen, Cory Comellus,, Jason Martin, and Duen Horng Chau. Shapeshifter: Robust physical adversarial attack on faster r-cnn object detector. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September 10-14, 2018, Proceedings, Part 18, pages 52 68. Springer, 2019.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In international conference on machine learning, pages 1310-1320. PMLR, 2019.
MMPreTrain Contributors. Openmmlab's pre-training toolbox and benchmark. https://githi:
commoppnn-mlal/mmmpppnn htttoommmmmmmmmmpeeeeeeee hwwpll/mmmmmeetriin..mmmm 2023.
Com/open-mng
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 9185 9193, 2018.
Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses transferable adversarial examples by intilaaioonivvrrian attacks. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA. USA, June 16-20, 2019, pages Evading. 2019. URU/tttp//opeeaccess thecvf. com/ content_tttttccccccoo __C 1a1RR_2019/hhhaa 10019/html//ong Def Defenses to Transferable Adversaria. Examples bt_pee_______________ Attacks_Ciie CV___VVR____0 2019 papprrrperr.... html
Invariant.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.111 11929, 2020.
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-box adversarial examples text classification. Proceedings of the 56th Annual Meeting of the Association for Computa- tional Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 2: Short Papers. pages 31-36, 2018. doi: 10. 18853/v1/P18-2000 URL C httph:ttt/////ttt //wp:: aclweb. wawlweeborg//eoo.....o anthology/ P18 20062006/

---

Robert Geirhos, Jörn-Henri Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(1 11:665666622 673, 2020.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA. USA, May 7-9, 2015, Conference Track Proceedings, 2015. Uttthtpp::/aaae org/ / 1412.6572
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. IEEE Conference on Computer Vision and Patlern Recognition (CVPR), 2016.
Jan Hendrik Metzen, Mummadi Chaithanya Kumar, Thomas Brox, and Volker Fischer. Universal adversarial perturbations against semantic image segmentation. In Proceedings of the IEEE international conference on computer vision, pages 2755-2764, 2017.
Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
Rui Huang and Yixuan Li. Mos: Towards scaling out-of-distribution detection for large semantic space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8710-8719, 2021.
Harini Kannan, Alexey Kurakin, and Ian J. Goodfellow. Adversarial logit pairing. CoRR, abs/ (oo//1833,0373, 2018. httttp:://arxi.orrg/bbbbb//////e 00030/6333333333333333
Jernej Kos, Ian Fischer, and Dawn Song. Adversarial examples for generative models. In 2018 IEEE Security and Privacy Workshops, SP Workshops 2018, San Francisco, CA, USA, May 24, 2018, pages 36-42, 2018. doi: 10. 10.1109//PW.2018.00001 URL https:://oi. org/ 10. 1109/ SPW 2018. 00014
Soichiro Kumano, Hiroshi Kera, and Toshihiko Yamasaki. Superclass adversarial attack. arXiv preprint arXiv:2205. 1t.t22205..429.. 14629, 2022.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distriuution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018.
Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E Hoperoft. Nesterov accelerated gradient and scale invariance for adversarial attacks. arXiv preprint arXiv: 2019.
Yen-Chen Lin, Zhang- Wei Hong, Yuan- Hong Liao, Meng-Li Shih, Ming- Yu Liu, and Min Sun. Tactics of adversarial attack on deep reinforcement learning agents. In Proceedings of the Twenty- Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017. Melbourne, Australia, August 19-25, 201 7, pages 3756-3762, 2017. doi: 10.24963/ijcai.2017/555 //tps/// org/10. 10. 24963/ 1 jacai.i.... 2017/525
Xin Liu, Huanrui Yang, Ziwei Liu, Linghao Song, Hai Li, and Yiran Chen. Dpatch: An adversarial patch attack on object detectors. arXiv preprint
2018.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017. URL https://openreviek wwwp:/eeeeoooooodtteteee.. Syy.idddxxsxxxx11111..
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: aurxiv:1766...0 1706.06083, 2017.
Jayanta Mandi, Peter J Stuckey, Tias Guns, et al. Smart predict-and-optmmia for hard combina- torial optimization problems. Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 1603-1610, 2020.
12

---

Nicolas Papernot, Patrick D. McDaniel, Ian J. Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Practical black-bor box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, AsiaCCS 2017, Abu Dhabi, United Arah Emirates, April 2-6, 2017, pages 506-519, 2017. doi: 10.1 10.1145/3052973. 3053009. URL htttps://dot org/ 10. 1 1141/336000 3052973. 3053009
Yao Qin, Nicholas Carlini, Garrison W. Cottrell, lan Goodfellow, and Colin Raffel. Imperceptible. robust, and targeted adversarial examples for automatic speech recognition. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, pages 5231-5240, 2019. URL htttp::/pproceednggg mlr press/ v97 237//in19a.hnr html
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115 (3):211-252, 2015. doi: 10.1007/s1 1263-015-0816-y..
Sara Sangalli, Ertunc Erdil, Andeas Hötker, Olivio Donati, and Ender Konukoglu. Constrained optimization train neural networks on critical and under-represented classes. Advances in Neural Information Processing Systems, 34:25400-2541,, 2021.
Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K. Reiter. Accessorize to crime: Real and stealthy attacks on state-of-the-art face recognition. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, October 24-28, 2016, pages 1520, 2016. doi: 10.1145/2976749.2998999 URL http::///ool org/10
1145/2976749 2978392
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jonathon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. CoRR, abs/1512.000577, 2015. URL
http:://araa http://aaxvv..v http:///aaxii..orgbbb/////s///// 15120/001710
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers distillation through attention. International conference on machine learning, pages 10347-10357. PMLR, 2021.
Nurislam Tursynbek, Aleksandr Petiushko, and Ivan Oseledets. Geometry -iinntrr--nspieed top-k adversarial perturbations. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3398- -3077,0777200 2022.
Po-Wei Wang, Priya Donti, Bryan Wilder, and Zico Kolter. Satnet: Bridging deep learning and logical reasoning using differentiable satisfiability solver. In International Conference on Machine Learning, pages 3545-6554 PMLR, 2019.
Yisen Wang, Difan Zou, Jinfeng Yi. James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Representations, 2020.
Eric Wong, Leslie Rice, and J Zico Kolter. Fast is better than free: Revisiting adversarial training. arXiv preprint 2xX.x:0001.0399. 2020.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Loddon Yuille. Adversarial examples for semantic segmentation and object detection. 2017 IEEE International Conference on Computer Vision (ICCV), pages 378-1387, 2017.
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan L. Yuille, and Kaiming He. Feature de- noising for improving adversarial robustness. In IEEE Conference on Computer Vision and Puttern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 501-509, 2019a. URt//tttpp//oppeacccess thecvf. com/ content, Seattent_CVRRR____ 2019/htmll/// httmm//iieFetturee..... Feature Denoising for fomoisiin_oor__mprrrnnnn. Adversarial Robustness _cceess___________ 2e1a_2219___0 1111_paperr_... .htll
13

---

Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang. Zhou Ren, and Alan L. Yuille. Improving transferability of adversarial examples with imput diversity. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. pages 2730-2739, 2019b. URL htttp://peenaccess. thecvf. www.c/ccoooooooooooooooooooooooo wwom//oonttt_CVVVV /
html/Xia Xie
httml/Kie_Ipproving. Improving- Transferability. of Adversarial. Examples, Fiith_Inputs
Diversatty_CVPK_2019_PP
html
Chaoning Zhang, Philipp Benz, Adil Karjauv, Jae Won Cho, Kang Zhang, and In So Kweon. Investigating top-k white-box and transferable black-box attack. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 15085-11594, 2022.
Zekun Zhang and Tianfu Wu. Learning ordered top-k adversarial attacks via adversarial distillation. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition Workshops, pages 776-777, 2020.
14

---

A Experimental Settings
The selection of learning rates y and values (see Eqn. 10p in the context of learning attacks requires careful consideration to achieve the desired trade-offs (see Fig. B) and optimize the attack performance. In this regard, the learning rate is adapted based on the value the number of targets, K. Empirically, we observed that as K increases, higher learning rate tends to yield better balance between Attack Success Rate (ASR) and perturbation energy consumption. Specifically, for different ranges of K, the learning rate is adjusted accordingly, ensuring appropriate scaling factor to guide the optimization process. Additionally, we find convolutional models require a larger learning rate than Transformer models to reach desirable attacks in all tested methods.
If K < 5, the learning rate is set to 7 0.75e 3 (for all the four models).
. 5 < K < 10, the learning rate is set to y 1.0e (for ViT-B and DEiT-S) and 2.0c - (for ResNet-50 and DenseNet-121)..
1f S K < 15, the learning rate is set to 7 1.0e 3 (for ViT-B and DEÎT-S) and 3= -3 3 (for ResNet-50 and DenseNet-12.]
*
If 15 < K < 20, the learning rate is set to 7 (for ResNett-50 and 1enseNet-121).
1.5c - 3 (for ViT-B and DEÎT-S) and
7 3.5e
If K 2 20, the learning rate set to y = 2e (for VYT-B and DEiT-S) and y = 4e (for ResNet-50 and DenseNet-121).
Similarly, the choice of the weight parameter À in the loss function (Eqn. 10 also plays crucial role. We use À to weight the first term in both Eqn [Q (the loss term that finds a successful attack when optimized) and we leave the second term (the energy penalty) unweighted. In challenging attacks where
2 5, the range of suitable À values that achieve desirable trade-offs between ASR and energy is significantly wider compared to easier attacks (K 1). Given the multitude of appropriate À values for difficult attacks, we do not perform explicit tuning of À in computing results in Table since different choices of À would correspond to different points along the energy/ASR trade-off curve (which are used in generating the trade-off curves in Fig 3
For attacks with K = 1, selecting an excessively high À can lead to inefficient energy usage. For instance, consider an attack with À achieving an ASR of 1.0 and an energy cost of 2.0, while increasing À to 10 maintains the ASR at 1.0 but raises the energy cost to 5.0. In this case, the choice of X 5 is preferable as it achieves the desired ASR with lower energy consumption.
For our QuadAttack K which operates in high-dimensional latent space with much higher loss magnitudes than CWK and AT (which operate in the logit or probability space), a lower value of A is necessary to reach the optimal point for K = 1. Hence, we set À = 0.5 for QuadAttack and for the logit/probability space losses for the K = 10 since these attacks are more challenging and require higher weight on the top-K term to attain the desired ASR and none of the tested methods reach ASR saturation points on K >= 5 for the chosen X.
case. For all other values of K, we use
B More Qualitative Results
In addition to quantitative evaluations in Table we provide detailed visualizations and qualitative analysis to gain deeper insights into the behavior and impact of our QuadAttacK method on the classification models. These visualizations offer comprehensive understanding of the attack process, showcasing the changes in both image perturbations and attention maps (for Transformer models). By examining the visual patterns and comparing the distributions of clean and attacked class predictions, we can explore the effects of our QuadAttac K on the attacked models' predictions and gain valuable insights into the robustness of these models. These visual analyses serve as valuable complement to our quantitative assessments, providing a holistic perspective on the performance and behavior of our Qtaa across various attack scenarios.
Furthermore, it is important to note that while our QuadAttac does not prioritize maximizing the margins between class probabilities, its quadratic programming approach fundamentally enables the
15

---

Clean Image & Attn
Adv. Example
Adv. Perturbation
Adv. Attention
CW
AD
QuadAttar Atttacc K
Figure 4: Examples of attacking DeIT-S Touvron et al. 202 with K 5: the 1st and 2nd rows show results for CW* and Adversarial Distillation (AD) (Zhang and Wul 2020) respectively, and the 3rd row shows results for QuadAttacK. The ground-truth label Weimaraner. The ordered top-5 targets (randomly sampled and kept unchanged for the different attack methods) are: [table lamp, langur, wig, hip, piggy bank], while for example the original top-5 predictions of DEiT-S are [Weimaraner, Bedlington terrier, German short-haired pointer, Great Dane, Yorkshire terrier].
integration of such constraints if desired for other applications. The flexibility of our method allows for the incorporation of additional objectives or constraints that prioritize class separability, enabling researchers to tailor the optimization process according specific needs. This versatility opens up avenues for exploring variations of the QuadAttac K framework and adapting to diverse scenarios where increasing the margins between class probabilities is desirable objective.
16

---

Clean Image & Atin
Adv. Example
Adv. Perturbation
Adv. Attention
sos
AV
QuadA Attac
Figure 5: Examples of attacking DEIT-S Touvron et al. 2021 with K = 10: the 1st row shows results for Adversarial Distillation (AD) [Zhang and Wu 2020 and the 2rd shows results for QuadAutac. and CW fails for this example. The ground-Iruth label is Weimaraner. The ordered top-10 targets (randomly sampled and kept unchanged for the different attack methods) are: [table lamp, langur, wig, hip, piggy bank, American Staffordshir terrier, school bus, crossword puzzle, entertainment center, ibex]. while for example the original top-10 predictions of DEiT-S [Weimaraner, Bedlington terrier, German short-haired pointer, Great Dane, Yorkahire terrier, Siamese cat, butcher shop, silky terrier, Italian greyhound, vizsla].
(a) Clean Image
(b) Adversarial Image
(c) Adversarial Perturbation
005
30
084
75
100
043
125
047
120
25C
(d) Clean Image's Class Prob.
(e) Adversarial Image's 's Class Prob.
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
[o/4]
12
191
110
0.0
0.0
200 400 600 800 1000
200
600
800
1000
Figure 6: Top-15 attack example 1/2 on ResNet-50 He et al. original Top-15 predictions are beaver, mink, otter, weasel, platypus, porcupine, American coot, water snake, red-breasted merganser, guinea pig, sea lion, red-backed sandpiper, European gallinule, little blue heron, limpkin J. The ordered Top-15 targets (randomly sampled) are: [ oystercatcher, desktop computer, shovel, bib, crane, spatula, torch, acorn, dalmatian, cheetah, gown, bull mastiff, microwave, cornet, puffer ].
2016 with QuadAttac (ours). The
17

---

(a) Clean Image
(b) Adversarial Image
(c) Adversarial Perturbation
0.000
0.00
0.020
175
0.000
2CD
0.000
(d) Clean Image's Class Prob.
(c) Adversarial Image's 's Class Prob.
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
10
722
033
22
[71/21
0.0
0.0
200 400 600
800
1000
200
600
800
1000
Figure 7: Top-15 attack example 1/2 on DenseNet-121 (Huang et al. 2017 with QuadAttacK
(ours). The original Top-15 predictions are [ chimpanzee, gorilla, siamang, orangutan, gibbon, sloth bear, colobus, howler monkey, guenon, American black bear, spider monkey, baboon, macaque, patas, water buffalo
The ordered Top-15 targets (randomly sampled) are: window shade, garden spider, cowboy hat, comic book, iPod, mortarboard, leaf beetle, stupa, zucchini, cock, beer bottle, cab, sunglass, fiddler crab, Windsor tie ].
18

---

(a) Clean Image
(b) Adversarial Image
(c) Adversarial Perturbation
120
0.00
135
0.005
200
0,000
(d) Clean Image's Attention
(e) Adversarial Image's Attention
(f) Clean Image's Class Prob.
(g) Adversarial Image's Class Prob.
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
100 P1
0.2
0.2
[12]
[12)
100
[13]
144
-51
[100
0.0
0.0
200
400
600 800
1000
200
600
800
1000
Figure 8: Top-15 attack example 1/2 on DeïT-S Touvron et al. 2021 with QuadAttac K (ours).
The original Top-15 predictions are [ nematode, isopod, digital clock, jellyfish, crossword puzzle, flatworm, drumstick, jack-o' -lantern, knot, bassoon, safety pin, paper towel, thunder snake, matchstick, trombone J. The or- dered Top-15 targets (randomly sampled) are: axolotl, quail, hyena, carbonara, hen, oboe, mud turtle, robin, Italian greyhound, oystercatcher, space shuttle, airliner, Bedlington terrier, miniature pinscher, iron ].
19

---

(a) Clean Image
(b) Adversarial Image
(c) Adversarial Perturbation
0.033
0.025
0.03
120
135
200
(d) Clean Image's Attention
(e) Adversarial Image's Attention
(f) Clean Image's Class Prob.
(g) Adversarial Image's Class Prob.
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
[lepp
0.2
0.2
216]
[9]
[166]
[1]
0.0
0.0
200 400
600
800
1000
200
600
800
1000
Figure 9: Top-15 attack example 1/2 on VIT-B Dosovitskiy et al., 2020 with QuadAttacK (ours). The original Top-15 predictions are Band Aid, airship, warplane, airliner, wing, mouse, space bar, face powder, missile, ballpoint, space shuttle, kite, modem, revolver, speedboat J. The ordered Top-15 targets (randomly sampled) are: [ padlock, king snake, screwdriver, Welsh springer spaniel, bookshop, triumphal arch, shoe shop, Italian greyhound, diamondback, missile, drilling platform, worm fence, snake, African elephant, joystick ].
20

---

(a) Clean Image
Adversarial Image
(c) Adversarial Perturbation
0.25
0.000
0.00
0.003
2003
(d) Clean Image's Class Prob.
(e) Adversarial Image's Class Prob.
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
(2
0.0
0.0
Figure 10: Top-10 attack example 1/2 on ResNet-50 (He et al. 2016) with QuadAttac (ours). The original Top-10 predictions are mink, beaver, hippopotamus, chimpanzee, weasel, otter, American alligator, mud turtle, Rottweiler, terrapin J. The ordered Top-10 targets (randomly sampled) are: [ tiger cat, black-and-tan coonhound, dung beetle, hook, banded gecko, hognose snake, skunk, ashcan, patio, admiral ].
21

---

(a) Clean Image
(b) Adversarial Image
(c) Adversarial Perturbation
45-
0.000
90
75
0.020
200
125
120
0,000
0.005
Shawon
Shown
50
1000
150
2003
(d) Clean Image's Class Prob.
(e) Adversarial Image's Class Prob.
1380]
200
400 600
1000
Figure 11: Top-10 attack example 1/2 on DenseNet-121 [Huang et al., 2017 with QuadAttac K (ours). The original Top-10 10 predictions are coyote, red wolf, grey fox, dhole, dingo, red fox, kit fox, lynx, timber wolf, hyena J. The ordered Top-10 targets (randomly sampled) are: [ chain, hermit crab, tiger cat, sweatshirt, packet, European fire salamander, electric locomotive, mobile home, fiddler crab, car mirror ].
22

---

(a) Clean Image
(b) Adversarial Image
Adversarial Perturbation
29-
99
75
220
125
120
30
1300
150
200
(d) Clean Image's 's Attention
(e) Adversarial Image's s Attention
(f) Clean Image's 's Class Prob.
g
Adversarial Image's Class Proh.
200 400 600 800 1000
Figure 12: Top-10 attack example 1/2 on DeiT-S [Touvron al., 2021] with QuadAttac K (ours). The original Top-10 predictions are wooden spoon, ladle, French horn, snowmobile, mortar, spatula, paddle, rocking chair, fly, frying pan J. The ordered Top- 10 targets (randomly sampled) are: tennis ball, langur, toy terrier, pool table, patio, ballpoint, bee eater, fireboat, toy poodle, soup boxl ].
23

---

(a) Clean Image
(b) Adversarial Image
(c) Adversarial Perturbation
52
75
160
125
0.0000
150
09973
0,000
0.9925
0.200
(d) Clean Image's Attention
(c) Adversarial Image's 's Attention
(f) Clean Image's 's Class Prob.
(g) Adversarial Image's Class Prob.
1.0
1.0
0.8
0.6
0.6
0.4
0.4
0.2
0.2
[8)
0.0
0.0
200 400 600
800
1000
200 400
600
800
1000
Figure 13: Top-10 attack example 1/2 on VIT-B [Dosovitskiy et al. 2020 with QuadAttac (ours). The original Top- 10 predictions are [ chocolate sauce, ice cream, trifle, pomegranate, bakery, fig, hay, strasberry, burrito, rapeseed ]. The ordered Top-10 targets (randomly sampled) are: [ goblet, shopping cart, titi, maillot, racer, espresso maker, zebra, Shih-Tzu, hand blower, speedboat ].
24

---

(a) Clean Image
(h) Adversarial Image
(c) Adversarial Perturbation
0.066
90
0.022
75
200
0.000
125
120
0,004
135
200
(d) Clean Image's Class Prob.
(e) Adversarial Image's Class Prob.
0.0
200 400
600
800
1000
200 400 600 800 1000
Figure 14: Top-5 attack example 1/2 ResNet-50 (He et al., 2016) with QuadAttack (ours). The original Top-5 predictions are Gordon setter, cocker spaniel, Irish setter, Sussex spaniel, English setter The ordered Top-5 targets (randomly sampled) are: indri, television, grasshopper, espresso maker, Yorkshire terrier ].
25

---

(a) Clean Image
(b) Adversarial Image
(c) Adversarial Perturbation
0.026
45-
90
0.022
75
220
125
120
0.006
0.002
50
1000
150
2003
(d) Clean Image's Class Prob.
(e) Adversarial Image's Class Prob.
200 400
600
1000
Figure 15: Top-5 attack example 1/2 on DenseNet-121 [Hangg et al., 2017] with QuadAttac K (ours). The original Top-5 predictions are Gordon setter, Yorkshire terrier, Bernese mountain dog, black-and-tar coonhound, Brabancon griffon J. The ordered Top- 5 targets (randomly sampled) are: indri, television, grasshopper, espresso maker, Yorkshire terrier ].
26

---

(a) Clean Image
(h) Adversarial Image
(c) Adversarial Perturbation
0.044
59
0.012
75
2C0
125
0.006
220-
135
0,000
(d) Clean Image's Attention
(e) Adversarial Image's 's Attention
(f) Clean Image's Class Prob.
(g) Adversarial Image's Class Prob.
0.2
0.0
200 400 600
800
1000
200 400
600
800
1000
Figure 16: Top-5 attack example 1/2 on DeïT-S Touvron et al. 2021 with QuadAttac K (ours). The original Top-5 predictions are home theater, cinema, projector, theater curtain, pool table The ordered Top-5 targets (randomly sampled) are: tow truck, snow leopard, cairn, EntleBucher, Leonberg ].
27

---

(a) Clean Image
(b) Adversarial Image
(c) Adversarial Perturbation
25-
50
75
0.123
100
0.200
125
0.97
150
01930
175
C9923
300
(d) Clean Image's Attention
(e) Adversarial Image's Atlention
(f) Clean Image's Class Prob.
(g) Adversarial Image's Class Prob.
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
200 400 600
800
1000
200
800 1000
Figure 17: Top-5 attack example 1/2 on VIT-B DDosovisskiy et al. 2020 with QuadAttacK (ours). The original Top-5 predictions are Band Aid, airship, warplane, airliner, wing J. The ordered Top-5 targets (randomly sampled) are: padlock, king snake, screadriver, Welsh springer spaniel, bookshop ].
28

---

(a) Clean Image
(b) Adversarial Image
(c) Adversarial Perturbation
0.005
0.004
0.003
150
0.002
175
200
0.001
(d) Clean Image's Class Prob.
(e) Adversarial Image's Class Prob.
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
200
400
600
800
1000
200
400
600
800
1000
Figure 18: Top-1 attack example 1/2 ResNet-50 He et al.
2016 with QuadAttacK K (ours). The
original Top-1 I predictions are ashcan eggnog ].
J. The ordered Top-1 targets (randomly sampled) are:
29

---

(a) Clean Image
(b) Adversarial Image
(c) Adversarial Perturbation
150
175
200
0.000
(d) Clean Image's Class Prob.
(e) Adversarial Image's Class Prob.
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
200
400
600
800
1000
200
400
600
800
1000
Figure 19: Top-1 attack example 1/2 on 1e1 Huang et al. 2017 with QuadAttac K (ours). The original Top-1 predictions are starfish sampled) are: crayfish ].
The ordered Top-1 targets (randomly
30

---

(a) Clean Image
(h) Adversarial Image
(c) Adversarial Perturbation
0.006
0.004
500
125
1:0
0,002
17%
200
0.000
(d) Clean Image's Attention
(e) Adversarial Image's Attention
K
(f) Clean Image's Class Prob.
(g) Adversarial Image's Class Prob.
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
200
400
600
800
1000
200
400
600
800
1000
Figure 20: Top-1 attack example 1/2 on DeiT-S [Tovron et al. 2021 with QuadAttac K (ours). The original Top-1 predictions are sandal The ordered Top-1 targets (randomly sampled)
are: espresso ].
31

---

(a) Clean Image
(b) Adversarial Image
(c) Adversarial Perturbation
0.007
0.006
500
125
0.006
130
0.003
17%
0.000
(d) Clean Image's Attention
(e) Adversarial Image's Attention
(f) Clean Image's Class Prob.
(g) Adversarial Image's Class Prob.
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
200
400
600
800
1000
200
400
600
800
1000
Figure 21: Top-1 attack example 1/2 on VIT-B [Dosovitskiy et al., 2020) with QuadAttack (ours).
The original Top-1 predictions are bluetick are: hen ].
The ordered Top-1 targets (randomly sampled)
32


Reviews:
[{"review_id": "Gmm07iwm8C", "reviewer": "Reviewer_2efR", "summary": "This paper introduces QuadAttackK, a new approach to compute ordered top-K adversarial attacks. The main contribution of this paper is to formulate and efficiently solve the top-K adversarial attack problem via quadratic programming (QP). The experiment results on ImageNet models show that the proposed method improves the attack success rate for large K while maintaining a cheap budget. ", "questions": "None", "limitations": "Yes", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Clean formulation + efficient implementation: Introduces a Quadratic Programming (QP) approach to learn ordered top-k clear-box target attacks. The solver leverages recents methods in constrained optimization within neural networks (e.g., based on OptNet) to get an efficient batched QP implmentation. \n\n- State-of-the-art (SOTA) empirical results. The proposed method obtains SOTA attack success rates on ImageNet models such as DenseNet, ResNet and ViTs. It also enables top-k adversarial attacks with large K + low cost budget.\n", "weaknesses": "- The problem of computing *ordered* top-k adversarial attacks lacks some motivation. The motivation in the paper is either too general (\"enabling better controllability in learning attacks that are more difficult to defend, revealing deeper vulnerability of a trained DNN\") or vague / confusing (\"testing the robustness of an attack method itself, especially when K is relatively large\"). \n\n- Presentation and writing can be significantly improved (e.g., figures and tables are hard to parse, section 4 does not describe the baselines and evaluation metric clearly)  \n\n- As noted in the paper, QuadAttackK is slow compared to baselines (need to solve a QP after every iteration) and the attacks do not transfer well to other models.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time reviewing our paper. In the following, we address your comments point by point.\n\n**Comment 1:** \\\"The problem of computing ordered top-k adversarial attacks lacks some motivation.\\\"\n\n> **Response:** Please refer to the \\\"Elaborated Motivations of Learning Ordered Top-$K$ Adversarial Attacks\\\" in our global response. We will carefully discuss them in the revision.\n\n**Comment 2:** \\\"Presentation and writing can be significantly improved.\\\"\n\n> **Response:**  Thank you for your recommendations regarding paper presentation. We will carefully revise the paper. Regarding noted issues, our matrix in Eqn. 8 indeed has too many nonzero items in the second row. The item in row 2 column 4 should be a $0$ and the whole matrix should be as follows.\n$$\nD_T = \\begin{bmatrix}\n    0 & 1 & -1 & 0 & 0;  \\\\\n    -1 & 0 & 1 & 0 & 0;  \\\\\n    1 & 0 & 0 & -1 & 0 ; \\\\\n    1 & 0 & 0 & 0 & -1  \\\\\n    \\end{bmatrix} \n$$\n\n> With respect to a missing term in Eqn. 12, we agree there is a missing\nbias matrix term. The formulation in line 287 though, is correct. Eqn.\n12 will be corrected to\n$$D_T\\cdot (A\\hat{z} + B) > 0 \\quad \\Rightarrow \\quad -D_T\\cdot A\\hat{z} \\leq D_T\\cdot B - \\eta,$$\n\n> With respect to clearer descriptions of the baselines and evaluation metric, we will revise the paper to make it self-contained regarding those aspects. \n\n**Comment 3:** \\\"QuadAttac$K$ is slow compared to baselines (need to solve a QP after every iteration) and the attacks do not transfer well to other models.\\\"\n\n> **Response:**  We acknowledge the limitations raised in terms of the transferability and the QP Solving computational overhead. \n\n> With respect to the transferability of learned attacks,  we deliberately chose to focus on the complexity of this optimization problem in the clear-box setting, and on the learnability of ordered top-$K$ attacks, especially for large $K$'s. We notice that learning transferrable clear-box attacks is a challenging problem even for the traditional top-$1$ setting. We leave the study of attack transferability as a future endeavor. One potential starting point could be to investigate the problem of how to apply our QuadAttac$K$ for multiple different networks simultaneously. Another potential direction is to first gain better understanding of the alignment of the latent spaces (the input space to the linear classifier) between different networks, and then to guide the learning of attacks to focus more on those aligned sub spaces.   \n\n>With respect to the overhead of solving a QP problem at every iteration, our QuadAttac$K$ method has a computational overhead to solve the QP problem at each iteration. We acknowledge this limitation in our response to other reviewers and in our original submission. For a precise understanding of runtime, we have profiled our QuadAttac$K$ and the AD attack on ResNet50 and ViT-B. \n\n>> For ResNet-50 we have found on average QuadAttac$K$ performs 2.47 attack iterations per second whereas AD performs 32.02 iterations per second (a factor of 12.96). For ViT-B QuadAtta$K$ performs 2.96 attack iterations per second whereas AD performs 11.86 iterations per second (a factor of 4). \n\n>> We note that as the target model becomes larger, the adversarial loss constitutes a smaller fraction of total runtime thus the ratio tends toward 1. Further, we note the quicker attack iterations of QuadAttac$K$ on ViT-B which indicates our QP solver converges faster on ViT-B attacks. We will discuss the mean runtimes between different methods in revision. \n\n> To address the overhead of our QuadAttac$K$, we will also explore and compare how the QP solver could be adjusted to initialize the QP solver at the previous iteration's solution to nearly eradicate the cost of the QP solver. "}, {"review_id": "o8MOHk0DKK", "reviewer": "Reviewer_xrmU", "summary": "This work proposes a novel approach, QuadAttack to learning ordered top-K adversarial attacks with a low cost. The method is based on a quadratic programming formulation that optimizes the attack objective. Notably, this work extends to a larger K(Top-K). For example, the K is improved from 10 to 15 compared to previous works. QuadAttacK outperforms state-of-the-art methods in terms of attack success rate and query efficiency on various datasets and architectures.", "questions": "See above", "limitations": "None", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This work firstly  uses quadratic programming to learn the adversarial attack. The novelty can help this community.\n2. The work extends the Top-K attack from Top-10 to Top-15 even larger with a low cost, brining insights to feature works.3\n3. When the K=15, the ASR of QuadAttacK is much better than all SOTA methods.\n4. Comprehensive results show the generalization of the method.", "weaknesses": "1. QuadAttack is not always better than baseline methods. Some deep analysis is lacked.\n2. The related work[1] is not mentioned. It could also be considered as a baseline method.\n3. It would be better if some results about efficiency are provided.\n\n[1] Zhang, Chaoning et al. “Investigating Top-k White-Box and Transferable Black-box Attack.” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022): 15064-15073.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your efforts reviewing our submission. We address your concerns as follows. \n\n**Comment 1:** \\\"QuadAttack is not always better than baseline methods. Some deep analysis is lacked.\\\"\n\n> **Response:**  Thank you for your detailed review. We appreciate the concern raised regarding the comparisons between QuadAttac$K$ and baseline methods (such as AD), particularly in the context of ASR vs Energy tradeoff. \n\n> We first would like to point out that any mentions of *efficiency* in our paper refer precisely to the adversarial compute budget (total number of model backward passes). With that said, consider the $K=5$ and $1\\times 30$ DeiT-S configuration results in our original submission (Table 2). Here, our QuadAttac$K$ obtains an ASR of 0.77 with an L2 energy of 3.34, while AD achieves an ASR of 0.34 with an L2 energy of 2.52. This comparison may give the illusion that while QuadAttac$K$ handles more cases, AD appears as to be a lower-energy attack. To disprove this notion, it's crucial to emphasize that the loss weighting of QuadAttac$K$ can be adjusted (so can the baseline methods'). This loss weight allows trading ASR for lower energy (a trade that is typically exponential). Specifically, if we adjust the weight so that QuadAttac$K$'s ASR is 0.34 in this configuration, its L2 energy will be much lower than that of the AD method. For holistic comparisons in terms of the trade-off between ASR and L2 energy, please refer i) to *Table 1 in our global response PDF* which shows our QuadAttac$K$ consistently outperforms the baseline methods, and ii) to *Figures 1 \\& 2 in our global response PDF*, which show that our QuadAttac$K$ significantly outperform the AD method. \n\n> Additionally, we expanded our results to include the $9\\times *$ configurations. For every attack instance, this configuration performs 9 binary search steps to determine the lowest possible energy for a successful attack. In other words, this configuration reduces the concept of a tradeoff curve to provide more holistic results since the search eliminates the effect of a loss weight choice. This configuration again shows the large margin between QuadAttac$K$ and its baseline methods.\n\n**Comment 2:** \\\"The related work [1] is not mentioned. It could also be considered as a baseline method.\\\"  \n\n> **Response:** We really appreciate the additional reference suggestion. We will discuss this excellent work by Zhang, Chaoning et al. in revision. In comparisons, we would like to emphasize that \\[1\\] is not working on the same problem as our QuadAttac$K$. They investigate a different attack setting as discussed in Section 7 (Discussion) of \\[1\\]. They refer to our baseline work (AD, Zhang and Wu) and explicitly  state their Top-$K$ optimization definition is not the same problem. To elaborate, we would like to point out there may exist 3 different kinds of \\\"Top-$K$\\\" attacks in the literature as follows.\n> -   Untargeted Top-$K$ Adversarial Attack (Easiest): Ground truth shouldn't be in the Top-$K$ classes, Top-$K$ classes can be anything\n    but ground truth.\n> -   Unordered Top-$K$ Adversarial Attack (Harder): Provides specific target Top-$K$ classes that should be in the Top-$K$ predictions\n    after the attack but no particular order of appearance is enforced as long as each target class is somewhere in the Top-$K$\n    predictions.\n> -   Ordered Top-$K$ Adversarial Attack (Hardest): Provides specific target Top-$K$ classes in order and the Top-$K$ predicted classes\n    after attack must match this exact order.\n\n> (Zhang, Chaoning et al.) explore the *Untargeted Top-$K$ Adversarial Attack* whereas we focus on the *Ordered Top-$K$ Adversarial Attack*, so we may not be able to straightforwardly compare with their work.\n\n**Comment 3:** \\\"It would be better if some results about efficiency are provided.\\\"\n\n> **Response:** Our QuadAttac$K$ method has a computational overhead to solve the QP problem at each iteration. We acknowledge this limitation in our response to Comment 3 by the reviewer 2efR and in our original submission. For a precise understanding of runtime, we have profiled our QuadAttac$K$ and the AD attack on ResNet50 and ViT-B. \n\n>> For ResNet-50 we have found on average QuadAttac$K$ performs 2.47 attack iterations per second whereas AD performs 32.02 iterations per second (a factor of 12.96). For ViT-B QuadAtta$K$ performs 2.96 attack iterations per second whereas AD performs 11.86 iterations per second (a factor of 4). \n\n>> We note that as the target model becomes larger, the adversarial loss constitutes a smaller fraction of total runtime thus the ratio tends toward 1. Further, we note the quicker attack iterations of QuadAttac$K$ on ViT-B which indicates our QP solver converges faster on ViT-B attacks. We will discuss the mean runtimes between different methods in revision. \n\n> To address the overhead of our QuadAttac$K$, we will also explore and compare how the QP solver could be adjusted to initialize the QP solver at the previous iteration's solution to nearly eradicate the cost of the QP solver. "}, {"review_id": "lYHdiZbkrf", "reviewer": "Reviewer_Xdne", "summary": "It identifies that while sufficient to capture top-K attack constraints, hand-crafted surrogate losses are not necessary and often introduce inconsistency and artifacts in optimization. It eliminates the need of introducing surrogate losses. Instead, it keeps the top-K attack constraints in the vanilla form and cast the optimization problem as quadratic programming (QP). It solves the QP by leveraging a recently proposed differentiable QP layer (for PyTorch). ", "questions": "The presentation can be improved. For example, the second row of $D_T$ in equation 8 seems to be incorrect. The expression in equation 12 does not seem to be correct, which is inconsistent with line 287. \n\nIt is better to discuss the potential practical applications to highlight the importance of the attack. \n\nFor the baselines, it typically needs multiple steps of binary search and a number of iterations of optimization for each trial of binary search (such as 9x30). But in the paper, the baselines do not perform binary search and the configuration is just like 1x30 or 1x60. It is expected that the baselines without binary search does not perform well. Comparing with these simpler versions of baselines does not demonstrate the method is really better, since the baselines are not used in their original way as designed. Besides, since all baselines and the proposed method use the same 1x30 or 1x60 configuration, and the proposed method has additional cost with the QP solver, it is hard to claim that the proposed method is more efficient with less computations. Maybe it is better to also demonstrate the results under 9x30 for the baselines to show the performance of the full version, and it is also easier to claim efficiency.  ", "limitations": "It is better to discuss the potential negative societal impact of this work as it proposes an attack method in deep learning. ", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "It observes that directly minimizing the lp norm of the learned perturbation together with the hand-crafted surrogate loss could miss the chance of exploiting semantic structures of the feature embedding space ((i.e., the input space to the final linear classifier). Instead, it minimizes the Euclidean distance between the feature embedding vectors at two consecutive iterations in the optimization. This can be understood as the latent perturbation learning versus the raw data perturbation learning. Its proposed latent perturbation learning enables more consistent optimization trajectories in pursuing the satisfaction of the specified top-K attack constraints. The minimized Euclidean distance is then used as the loss together with the lp norm of the learned perturbation in computing the adversarial perturbation via back-propagation at each iteration. \n\nWith the proposed QP formulation, it aims to learning top-K attacks efficiently in terms of the computing budget. It eliminates searching the trade-off parameters. Instead, it uses the low-cost 1×S setting for better practicality (e.g. S = 30 or 60), i.e., using a default trade-off parameter. For large K’s (e.g., K > 10), it shows that the QuadAttacK can still achieve appealing ASR, while the prior art completely fails. ", "weaknesses": "The presentation can be improved. For example, the second row of $D_T$ in equation 8 seems to be incorrect. The expression in equation 12 does not seem to be correct, which is inconsistent with line 287. \n\nIt is still not clear how the proposed attack can be applied in practice. Although it lists some advantages of the successful ordered top-K attacks such as some potential directions, it does not provide practical example usages. It is better to discuss the potential practical applications to highlight the importance of the attack. \n\nIt mentions that the computation cost of the method is low. However, it does not discuss the complexity or computation cost theoretically. In experiments, all methods seem to adopt 60 steps optimization. It does not really demonstrate that the computation cost is low. Besides, the QP solver may introduce additional costs. The claim to be more efficient with less computation cost may be inaccurate. It is better to provide more discussions or experiments to show the low cost. \n\nFor the baselines, it typically needs multiple steps of binary search and a number of iterations of optimization for each trial of binary search (such as 9x30). But in the paper, the baselines do not perform binary search and the configuration is just like 1x30 or 1x60. It is expected that the baselines without binary search does not perform well. Actually the baseline under 9x30 can achieve 100% ASR for top-5 attack. Comparing with these simpler versions of baselines does not demonstrate the method is really better, since the baselines are not used in their original way as designed. Besides, since all baselines and the proposed method use the same 1x30 or 1x60 configuration, and the proposed method has additional cost with the QP solver, it is hard to claim that the proposed method is more efficient with less computations. Maybe it is better to also demonstrate the results under 9x30 for the baselines to show the performance of the full version, and it is also easier to claim efficiency.  ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking your time reviewing our paper. We address your concerns as follows. \n\n**Comment 1:** \\\"The presentation can be improved.\\\"\n\n> **Response:** We agree and will carefully revise and proofread the paper. \n\n> Regarding noted issues in our matrix in Eqn. 8, it indeed has too many nonzero items in the second row. The item in row 2 column 4 should be a 0 and the whole matrix should be as follows. \n$$\n\\begin{aligned}\n    D_T = \\begin{bmatrix}\n    0 & 1 & -1 & 0 & 0; \\\\\n    -1 & 0 & 1 & 0 & 0; \\\\\n    1 & 0 & 0 & -1 & 0; \\\\\n    1 & 0 & 0 & 0 & -1 \\\\\n    \\end{bmatrix} \n\\end{aligned}$$\n\n> With respect to a missing term in Eqn. 12, we agree there is a missing bias matrix term. The formulation in line 287 though, is correct. Eqn.12 will be corrected to,\n$$D_T\\cdot (A\\hat{z} + B) > 0 \\quad \\Rightarrow \\quad -D_T\\cdot A\\hat{z} \\leq D_T\\cdot B - \\eta,$$\n\n**Comment 2:** \\\"It is better to discuss the potential practical applications to highlight the importance of the attack.\\\"\n\n> **Response:** Please refer to the \"Elaborated Motivations of Learning Ordered Top-$K$ Adversarial Attacks\" in our global response. We will carefully discuss them in the revision.\n\n**Comment 3:** $9\\times *$ vs $1\\times *$ budgets and efficiency concerns.\n\n> **Response:** While we want to emphasize our claims of efficiency are with respect to adversarial budget (total number of model gradient calls), you raise a great point regarding the need to compare our baseline methods in their original configuration. For this reason, we have added $9\\times *$ results for $K=5$ and $K=10$ configurations in Table 1 in our global response PDF. Our QuadAttack$K$ achieves much better results consistently. \n\n> Additionally since $1\\times *$ configurations can be tuned to trade ASR for lower energy and vice versa, we compute and plot full ASR vs L2 Energy tradeoff curves for two attack settings. \n\n> With respect the computational overhead of our QuadAttac$K$ method due to solve the QP problem at each iteration. Qualitatively, we do observe that our method is slower than baseline methods. As our response to Comment 3 by the reviewers 2efR and xrmU, we acknowledge the limitation. For a precise understanding of runtime, we have profiled our QuadAttac$K$ and the AD attack on ResNet50 and ViT-B. \n\n>> For ResNet-50 we have found on average QuadAttac$K$ performs 2.47 attack iterations per second whereas AD performs 32.02 iterations per second (a factor of 12.96). For ViT-B QuadAtta$K$ performs 2.96 attack iterations per second whereas AD performs 11.86 iterations per second (a factor of 4). \n\n>> We note that as the target model becomes larger, the adversarial loss constitutes a smaller fraction of total runtime thus the ratio tends toward 1. Further, we note the quicker attack iterations of QuadAttac$K$ on ViT-B which indicates our QP solver converges faster on ViT-B attacks. We will discuss the mean runtimes between different methods in revision. \n\n> To address the overhead of our QuadAttac$K$, we will also explore and compare how the QP solver could be adjusted to initialize the QP solver at the previous iteration's solution to nearly eradicate the cost of the QP solver. To address the overhead of our QuadAttac$K$ , we will also explore and compare how the QP solver could be adjusted to initialize the QP solver at the previous iteration's solution to nearly eradicate the cost of the QP solver.\n\n**Comment 4:** \\\"It is better to discuss the potential negative societal impact of this work as it proposes an attack method in deep learning.\\\"\n\n> **Response:** On the one hand, we elaborate in the global response some potential scenarios in practice for which the proposed ordered top-$K$ adversarial attacks may be risky if applied.  On the other hand, since we focus on clear-box attacks, they are less directly applicable in practice compared to opaque-box attacks, which makes the concern less serious. We will make these clear in the Broader Impact section in revision."}, {"review_id": "dMns612YSP", "reviewer": "Reviewer_wPrk", "summary": "The paper proposes a new method for generating top-K adversarial perturbations -- modifying the input such that the classifier predicts the specified K classes, in order. The work addresses this with a two-stage approach, where the first stage computes an adversarial perturbation to the representation, subject to the top-K constraints, and the second stage modifies the input to match this representation.", "questions": "- What alternatives, if any, are the authors aware of to the ordered top-K objective, which achieves the same goal (attacks that are harder to detect based on the dependence between predicted class probabilities)?\n\n- The paper notes that qpth is a differentiable QP solver. Is the differentiability used?\n", "limitations": "yes", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper makes a case for top-K adversarial perturbation identification as a step towards more robust systems -- since such perturbations are harder to detect, and a model that is robust to them would be harder to fool. The optimization problem, which is the quadratic objective combined with linear constraints on the non-linear model outputs, is hard to solve hence the proposed two-stage approach which allows the use of methods beyond gradient descent -- specifically, QP on the representation, followed by gradient-based input modification to match this representation.\n\nThe method is simple and well-motivated, and comparisons with prior works show that the proposed method is able to perform well where the prior work fails (for the same search budget -- Adversarial Distillation is able to find the adversarial top-K perturbations but at a higher cost).", "weaknesses": "- While the paper clearly argues on the value of ordered top-K attacks, it is less clear what might be the alternatives that achieve similar goals but may be easier to optimize\n\n- Second row of (8) seems to have too many non-zeros\n\n- In (12), the last occurrence of D_T is missing a B term\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and efforts reviewing our submission. We address your concerns as follows.\n\n**Comment 1:** Alternatives to the Ordered Top-K Attack which are easier to optimize.\n\n> **Response:** As discussed in our global response, ordered Top-$K$ adversarial attacks exploit the principle of \\\"Class Coherence,\\\" recognizing the relationships or logic connecting classes within ordinal or nominal frameworks. Unlike unordered or Top-1 attacks, ordered Top-$K$ attacks can subtly manipulate predictions while maintaining coherence within the expected context.\n\n> With respect to alternative easier-to-optimize methods of learning ordered Top-$K$ attacks, one potential direction could be to learn more informative ordered Top-$K$ satisfying distribution to extend the AD method, rather than using the heuristic design method. To that end, a generative model may be trained to become an optimal transport between an easy to sample space with no topological holes onto the space of images that generate class coherent scores (e.g., a bijection from $[-1, 1]^N$ onto the space of images class coherent logit vectors) and a less refined adversarial attack may optimize on the input space of this generative model. By doing this, only perturbations that do not disturb the expected or logical inter-class relationships in the predicted logit vectors would be explored. This would rely on the unlikely requirement that the generative model would span meaningful perturbations, and further a generative model would have to be separately trained for each target model (e.g. ResNet-50).\n\n**Comment 2:** Is the differentiability of qpth used?\n\n> **Response:** Thank you for the very observant question. Yes, we make us of the differentiability of qpth. Imagine the minimizer to Eqn. 6 in our paper is given by the function $\\delta_{min} = G(x)$. Then we can phrase our loss as $L = \\| x - \\hat{x} \\|, \\hat{x} = x + \\delta_{min}$ or equivalently $L = \\| \\delta_{min} \\|$. The differentiability of qpth allows us to directly use the gradient of our perturbation with respect to our feature vector $\\frac{d}{dx}[\\| \\delta_{min} \\|]$ for optimization. On the other hand a non-differentiable solver would force\nus to treat $\\hat{x}$ as a constant and minimize the loss $L = \\| x - C \\|$ where $C$ is a constant. While our localized quadratic program is convex, our loss in general is not and thus a loss with a constant target may force use to follow an ever moving target, if we minimize the distance to our solution, then the target may actually become even further in our next iteration. Having access to the gradient $\\frac{d}{dx}[\\| \\delta_{min} \\|]$ keeps us from having to rely on a surrogate that may not have a good picture of how the solution the QP itself changes as we move in the search space.\n\n**Comment 3:** Presentation and typos.\n\n> **Response:** Regarding noted issues, our matrix in Eqn. 8 indeed has too many nonzero items in the second row. The item in row 2 column 4 should be a 0 and the whole matrix should be as follows,\n$$\\begin{aligned}\n    D_T = \\begin{bmatrix}\n    0 & 1 & -1 & 0 & 0; \\\\\n    -1 & 0 & 1 & 0 & 0; \\\\\n    1 & 0 & 0 & -1 & 0; \\\\\n    1 & 0 & 0 & 0 & -1 \\\\\n    \\end{bmatrix}, \n\\end{aligned}$$\n\n> With respect to a missing term in Eqn. 12, we agree there is a missing bias matrix term. The formulation in line 287 though, is correct. Eqn.12 will be corrected to,\n$$D_T\\cdot (A\\hat{z} + B) > 0 \\quad \\Rightarrow \\quad -D_T\\cdot A\\hat{z} \\leq D_T\\cdot B - \\eta$$\n\n\n\n"}]

Author rebuttal:
We sincerely thank all reviewers for their constructive feedbacks which help us to greatly improve our submission.  We first address some common concerns.  

**Drastically Improved Results.** Please refer to *Table 1 in our global response PDF* for the improved results. In optimization, perturbations are initialized with some small $\epsilon$ energy white gaussian noise. During the initial steps of optimization, the optimizer takes steps with large increases in perturbation energy since $\epsilon$ happens to be away from many required energies for a successful attack. These large increases in energy induces a momentum in the $AdamW$ optimizer, which makes it difficult to reduce L2 energy in future iterations even if our objective function's gradient points towards a direction with minimal energy. By introducing a small number of warmup steps (e.g., 5, as commonly done in training a network on ImageNet from scratch) after which the optimizer's state is reset, we have managed to improve the performance of all analyzed methods. Our QuadAttac$K$ benefits most.  


**Analyses on the Trade-Off Between Attack Success Rates and Attack Energies.** Please refer to *Figures 1 \& 2 in our global response PDF*. These tradeoff curves explore the concept of how a higher success rate may be achieved by choosing to have higher energies and conversely a lower energy may be achieved by choosing to have a lower success rate. Nonetheless, these curves holistically compare the capacity of QuadAttac$K$ against the baseline method. 
> Additionally, we expanded our results to include the $9\times \*$ configurations. For every attack instance, this configuration performs 9 binary search steps to determine the lowest possible energy for a successful attack. In other words, this configuration reduces the concept of a tradeoff curve to provide more holistic results since the search minimizes the effect of a loss weight choice. This configuration again shows the large margin between QuadAttac$K$ and its baseline methods.

**Elaborated Motivations of Learning Ordered Top-$K$ Adversarial Attacks:**
These attacks exploit the principle of \"Class Coherence\", recognizing the relationships or logic connecting classes within ordinal or nominal frameworks. Unlike unordered or Top-1 attacks, ordered Top-$K$ attacks can subtly manipulate predictions while maintaining coherence within the expected context.

In an ordinal context like credit ratings (*\[Extremely High Risk, Very High Risk, High Risk, Moderate Risk, Low Risk, Minimal Risk, No
Risk\]*), they can downgrade a rating without disrupting the logical flow, making it less detectable. In a nominal context such as a
recommendation system attacking predicted user interests in shopping classes from *\[Books, Movies, Beauty, Furniture, Fashion\]* into
*\[Fashion, Books, Movies, Beauty, Furniture\]* in order; subtly pushing Fashion onto the user while keeping them engaged with their true
interests.

An unordered change might disrupt these logical groupings, (e.g. push Fashion to the top, but also inadvertently pushing Furniture up as well) leading to a clumsy attack, raising a red flag to a human user, or even trivial detection in a system that validates class coherence. These refined and subtle manipulations can have diverse consequences, from affecting financial decisions to subtly influencing user behavior in online platforms. They are more challenging to detect than other types of attacks, both quantitatively or from a human standpoint, and represent a nuanced and significant threat that warrants deeper exploration in adversarial machine learning. For completeness, we provide more specific examples below.

> Ordinal example: Imagine a cancer risk assessment tool that analyzes 2D medical images like mammograms to categorize patients' cancer risk into the ordinal 7-level risk ratings (as the credit ratings). An oncologist could use this tool to triage patients, prioritizing those in the highest risk categories for immediate intervention. An attacker aiming to delay treatment might use an ordered top-5 adversarial attack to change a prediction for a patient initially assessed as Very High Risk. They could target the classes *\[High Risk, Moderate Risk, Low Risk, Minimal Risk, Very High Risk\]*, subtly downgrading the urgency without breaking the logical sequence of risk categories. An unordered attack, in contrast, might lead to a sequence like *\[Low Risk, Very High Risk, Minimal Risk, Moderate Risk, High Risk\]*, disrupting the ordinal relationship between classes. Such a disruption could raise red flags, making the attack easier to detect.

> Nominal class example: Traffic control systems could use deep learning to optimize flow by adjusting the timing of traffic lights based on the types of vehicles seen. Priority might be given to certain vehicle classes, such as public transit or emergency vehicles, to improve
response times. Imagine a city's traffic control system, which has specific traffic light timing behavior for the nominal vehicle
categories *\[Emergency Vehicle, Public Transit, Commercial Vehicle, Personal Car, Bicycle\]*. Public transit might be given slightly
extended green lights during rush hours to encourage public transportation use. An attacker wanting to cause delays for personal
cars without raising alarms could launch an ordered Top-2 adversarial attack, targeting the sequence *\[Commercial Vehicle, Public
Transit\]*. This would cause the system to interpret most personal cars as commercial vehicles during the attack, applying the extended green light times meant for public transit to lanes primarily used by commercial vehicles. An unordered top-2 attack that may result in
*\[Emergency Vehicle, Commercial Vehicle\]*, would likely be quickly detected, as emergency vehicle priority changes are significant and
could be easily noticed by traffic operators (this weakness is exacerbated in any Top-1 attack or unordered attacks).

Meta-review: This paper proposes a new method for constructing top-k adversarial examples. Reviewers found the the method interesting, and the results compelling. The authors constructively engaged with the reviewers during the rebuttal period. There are some justified concerns about the quality of presentation and writing. Authors, make sure to implement the promised changes to address reviewer concerns.
Decision: Accept (poster)