MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifiers
Authors: Yatong Bai
Mo Zhou
Vishal M. Patel
Somayeh Sojoudi
Venue: Accepted by TMLR ()

Abstract: Adversarial robustness often comes at the cost of degraded accuracy, impeding real-life applications of robust classification models. Training-based solutions for better trade-offs are limited by incompatibilities with already-trained high-performance large models, necessitating the exploration of training-free ensemble approaches. Observing that robust models are more confident in correct predictions than in incorrect ones on clean and adversarial data alike, we speculate amplifying this "benign confidence property" can reconcile accuracy and robustness in an ensemble setting. To achieve so, we propose "MixedNUTS", a *training-free* method where the output logits of a robust classifier and a standard non-robust classifier are processed by nonlinear transformations with only three parameters, which are optimized through an efficient algorithm. MixedNUTS then converts the transformed logits into probabilities and mixes them as the overall output. On CIFAR-10, CIFAR-100, and ImageNet datasets, experimental results with custom strong adaptive attacks demonstrate MixedNUTS's vastly improved accuracy and near-SOTA robustness -- it boosts CIFAR-100 clean accuracy by $7.86$ points, sacrificing merely $0.87$ points in robust accuracy.

Published in Transactions on Machine Learning Research (08/2024)
MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifiers
Yatong Bai University of Colifornia, Berkeley
yotong battong_bo@@berreleeee edu
Mo Zhou
mahou32 edu
John Hopkins Universiby
Vishal M. Patel John Hopkins University
upatel36@fhu.edu
Somayeh Sojoudi
sojoudi@berkeley.coo edu
University of California, Berkeley
Reviewed on OpenReview: https: // operrevee........ net/ forun? id- pyD6cu Uml.
Abstract
Adversarial robustness often comes at the cost of degraded accuracy, impeding real-life applications of robust classification models. Training-based solutions for better trade-offs are limited by incompatibilities with already-trained high-performance large models, necessitating the exploration training-free free ensemble approaches. Observing that robust models are more confident in correct predictions than incorrect ones on clean and adversarial data alike, we speculate amplifying this "benign confidence property" can reconcile accuracy and robustness in an ensemble setting. To achieve so, we propose "MixedNUTS", a training-free method where the output logits of a robust classifier and standard non-robust classifier processed by nonlincar transformations with only three parameters, which are optimized through an elficient algorithm. MixedNUTS then converts the transformed logits inlo probabilities ies and mixes them as the overall output. On CIFAR-10, CIFAR-100, and ImageNet datasets, experimental results with custom strong adaptive attacks demonstrate MixedNUTS's vastly improved accuracy and near-SOTA robustness it boosts CIFAR-100 clean accuracy by 7.86 points, sacrificing merely 0.87 points in robust accuracy.
1
Introduction
Neural classifiers are vulnerable to adversarial attacks, producing unexpected predictions when subject to purposefully constructed human-inpperepptilee input perturbations and hence manifesting severe safety risks Madry et al. 2018). Existing methods for robust deep neural networks (Madry et al. 2018 Zhang et al. 2019) often suffer from significant accuracy penalties on clean (unattacked) data Tsipras al. 2019 Zhang et al., 2019 Pang al. 2022) As deep learning continues to form the core of numerous products, trading clean accuracy for robustness is understandably unattractive for real-life users and profit-driven service providers. As result, despite the continuous development in adversarial robustness research, robust models are rarely deployed and practical services powered by neural networks remain non-robust [[yas et al. 2018 Borkar & Chen 2021].
[Goodfellow et al. 2015
To bridge the gap bet ween robustness research and applications, researchers have strived to reconcile robust.ness and accuracy (Balaji et al.
2019 Chen et al.] 2021 Raghunathan et al. 2020 Rade Moosavi- Dezfool
2021
Liu & Zhao 2022
Pang
2022 Cheng et al. 2022). mostly focusing on improving robust training
This work was supported by grants from ONR and NSF.

---

Published in Transactions on Machine Learning Research (08/2024)
Accurate Base Classifier (8ms)
Argmax (One-hot)
Input
Raw
Processed Probabilities
Convex Combination
Mixed
Natural Log
Mixed
Logits
Probability
Logits
Robust Basc Classificr (h
Nonlincar Transform Eq.(4)
Softmax
Figure 1: Overview of the proposed MixedNUTS classifier. The nonlinear logit transformation, to be introduced in Section | significantly improves the rccuracy-robusttess balance while only introducing three parameters efficiently optimized with Algorithm
methods. Despite some empirical success, the training-basse based approach faces inherent challenges. Training robust neural networks from scratch is highly expensive. More importantly, training-based methods suffer from performance bottlenecks. This is because the compatibility between different training schemee unclear, making it hard combine multiple advancements. Additionally, it is hard to inlegrate robust training techniques into rapidly improving large models, often trained pre-trained with non-classification tasks.
To this end, an alternative training-frec direction has emerged, relieving the accuracy-robustuess tradc-off through an ensemble of a standard (often non-robust) model and a robust model (Bai el. al.) 2024b a). This ensemble is referred to as the mixed classifier, whereas base classifiers refers to its standard and robust components. The mixing approach is mutually compatible with the training-basod methods, and hence should be regarded as an add-on. Unlike conventional homogeneous ensembling, where all base classifiers share the same goal, the mixed classifier considers helerogeneous mixing, with one base classifier specializing in the benign attack-free scenario and the other focusing adversarial robustness. Thus, the number of base classifiers is naturally fixed as two, in turn maintaining high inference efficiency.
We observe that many robust base models share a benign confidence property: their correct predictions are much more confident than incorrect ones. Verifying such a property for numerous existing models trained via different methods (Peng al.)
2023:
Pang et al. 2022
Wang et al. 2023 Debenedetti et al., 2023: Na 2020
Gowal et al. 2020: Liu et al.
2023 Singh et al.
2023 speculate that strengthening this property
improve the mixed classifiers' trade-off even hothout changing the base classifiers' predicted classos.
Based on this intuition, we propose MixedNUTS (Mixed Ural classifiers with Nonlinear TranSformation), a training-free method that enlarges the robust base classifier confidence difference between correct and incorrect predictions and thereby optimizes the mixed classificr's accuracy-robustuess trade-off. MixedNUTS applies nonlinear transformations to the accurate and robust. base classifiers' logits before converting them into probabilities used for mixing. We parameterize the transformation with only three coefficients and design an efficient algorithm to optimize them for the best tradc-off. Unlike (Bai et al. 2024b), MixedNUTS does not modify base neural network weights or introduce additional components and is for the first time efficiently extendable to larger datasets such as ImageNet. MixedNUTS is compatible with various pre-trained standard and robust models and is agnostic to the base model details such as training method, defense norm (fo., l2, etc.); training data, and model architecture. Therefore, MixedNUTS can take advantage of recent developments in accurate or robust classifiers while being general, lightweight, and convenient.
Our experiments leverage AutoAttack &Croce & Hein 2020 and strengthened adaptive attacks (details in Appendix B to confirm the security of the mixed classifier and demonstrate the balanced accuracy and robustness datasets including CIFAR-10, CIFAR-100, and ImageNet. On CIFAR-100, MixedNUTS improves the clean accuracy by 7.86 percentage points over the state-of-the-art non-mixing robust model while reducing robust accuracy by merely 0.87 points. On ImageNet, MiXedNUTS is the first robust model to leverage even larger pre-training datasets such as ImageNet-2lL Furthermore, MixedNUTS allows for déferencetimee adjustments between clean and adversarial accuracy.
2 Background and Related Work
2.1 Definitions and Notations
This paper uses a: R + (0. 1) to denote the standard Softmax function: for an arbitrary E R', the 2th entry of (=)) is defined as a(2); = Σ exp(x;))
exp(2:) where 2; denotes the ęth entry of z. Consider the special case
2

---

Published in Transactions on Machine Learning Research (08/2024)
CIFAR-10 (f 4/SS)
CIFAR-100 100 (fw 9ysss)
ImageNet (f = 1/ass)
AutoAttack Accuracy (%)
60
40
05
55
60
35
50
Standalonw Rubust Models Standard Model (Bai et 2024) MixedNUUT (ours)
performente
90
92
94
96
98
65
70
75
80
85
90
73
76
79
82
85
Clean Accuracy (%)
Clean Accuracy (%)
Clean Accuracy (%)
AS refers to adaptive smoothing Bai ot al.
2024a another ensemble framework. Unlike AS, MixedNUTS requires no additional training.
Figure 2: MixedNUTS's accuracy- accuracy-robustnes robustnes balance compared to state-of-the-art models on RobustBBench MixedNUTS is more accurale on clean data than all standalone robust models. At the same time, MixedNUTS achieves the second- highhet robustness among all models for CIFAR-100 and ImageNet, and is the third most robust for CIFAR-10.
of 2, = +o0 for some i, with all other entries of 2 being less than +00, We define (2) for such 2 vector to be the basis (one-hot) vector eç. For a classifier h : TRd + R we use the composite function a h Rd -> [0, 1] to denote its output probabilities and use a o h2 to denote the 7th entry of it.
We define the notion of confidence margin of classifier as the prediction probability gap between the top two predicted classes:
Definition 2.1. Consider model h Rd + R,, arbitrary input Idd and its associated predicted label
Ū [c]. The confidence margin is defined as
ma (z) = a o hy()))
max o hx(x). iz
We consider classifier to be (G,-norm) robust at some input x Rdd if it assigns the same label to all perturbed imputs + such that [|8|| S c, where 2 0 is the attack radius. We additionally introduce the notion of the worst-case ad adversarial perturbation in the sense of minimizing the margin:
Definition 2.2. Consider adversarial attack against the confidence margin mz(z), defined as
min my (xr + 8).
|
We define the optimizer of this problem, (x), as the minimum-margin perturbation of M(-) around x. further define the optimal objective value, denoted as mx(x), as the minimuum margin of h(-) around x.
We
The attack formulation considered in Definition 2.2 is highly general. Intuitively, when the minimum margin is negative, the adversarial perturbation successfully changes the model prediction. When it is positive, the model robust at v, as perturbations within radius e cannot change the prediction.
2.2 Related Adversarial Attacks and Defenses
While the fast gradient sign method (FGSM) and projected gradient descent (PGD) attacks could attack and evaluate certain models (Madry et al.] 2018 Goodfellow et al. 2015), they are insufficient and can fail to attack non-robust models (Carlini & Wagner 2017 Athalye et al. 2018b] Papernot et al. 2017). To
this end, stronger adversaries leveraging novel attack objectives, black-box atlacks, and expectation over
transformation have been proposed (Gowal et al. 2019 Croce & Hein 2020 Tramer et al. 2020). Benchmarks
based on these strong attacks, such as Bench (Croce et al. 2021), ARES-Bench (Liu et al. 2023) and
BeDcc (Li et al. 2023), aim to unify defense evaluation.
AutoAttack (Croce& & Hein 2020 is a combination of white-box and black-box attacks (Andriushchenko ct al. 2020). It is the attack algorithm of RobustBench (Croce al. 2021]. where AutoAttackkevaluuted robust models are often agreed be trustworthy. We select AutoAttack as the main evaluator, with further strengthening tailored to our defense.
3

---

Published in Transactions on Machine Learning Research (08/2024)
Models aiming to be robust against adversarial attacks often incorporate adversarial training (adrry et al.
2018 Bai et al. 2022 2023). TRADES (Zhang et al. 2019), or their variations. Later work further enhanced the
adversarial robustness by synthetic training data (Wang et al. 2023 Sehwag et al. 2022). data augnenttttin
(Gowal et al. 2020 Rebuffi et al., 2021 Gowal ct al. 2021], improved training loss functions (Cui et al. 2023)
purposeful architectures (Peng el al. 2023), or elficient optimization (Shalahi et al. 2019). Nevertheless,
these methods still suffer from the trade-off between clean and robust accuracy.
To this end, there has been continuous interest from researchers to alleviate this trade-off (Zhang et al.
2019
Lamb et al. 2019 Balaji et al.) 2019 Chen et al., 2021 Cheng et al. 2022 Pfrommer et al. 2023
2024
Most methods are wrainingbbssed They are therefore cumbersome to construct and cannot leverage
already-trained state-of-the-art robust or non-robust models.
2.3
Ensemble and Calibration
Model ensembles, where the outputs of multiple models are combined to produce the overall prediction, have been explored to improve model performance (Ganaie et al. 2022) or estimate model uncertainty (Liu et al.
2019). Ensembling has also been considered to strengthen adversarial robustness (Pang et al. 2019 Adam
& Speciel 2020 Alam et al.
2022 Co et al. 2022). Theoretical robustness analyses ensemble models
indicate that the robust margins, gradient diversity, and runner-up class diversity all contribute to ensemble
robustness PPetrov al. 2023 Yang et al. 2022).
These existing works usually consider homogemeous ensemble, meaning that all base classifiers share the same goal (better robustness). They often combine multiple robust models for incremental improvements. In contrast, this work focuses on heterogeneous mixing, a fundamentally different paradigm. Here, the base classifiers specialize in different data and the mixed classifier combines their advantages. Hence, we focus on the two-model setting, where the role of each model is clearly defined. Specifically, the accurate base classifier specializes clean data and is usually non-robust, and the robust base classifier excels in adversarial data.
A parallel line of work considers mixing neural model weights (Ihaarco et al. 2022 Cai et al 2023 They generally require all base classifiers to have the same architecture and initialization, which is more restrictive.
Model calibration often involves adjusting confidence, which aims to align a model's confidence with its mispredicting probability, usually via lemperature scaling (Guo et al., 2017 Yu et al. 2022 Hinton et al., 2015). While adjusting the confidence of a single model generally does not change its prediction, this is not the case in the ensemble sotting. Unlike most calibration rescarch focusing on uncertainty, this paper adjusts the confidence for performance.
2.4 Mixing Classifiers for Accuracy-Robut Trade-Off
Consider a classifier Ista Rdd + RC, whose predicted logits are Itdd Istd, where dl is the input dimension and c the number of classes. We assumee Gsta(-) to be standard classifier trained for high clean accuracy (and hence may not manifest adversarial robustness). Similarly, we consider another classifier h Rd - R and assume it to be robust against adversarial allacks. We use accurale base classifier (ABC) and robust base classifier (RBC) to refer to (-)(.) and Nroh())
Mixing the outputs of a standard classifier and a robust classifier improves the accuracy-robussness trade- off, and it has been shown that mixing the probabilities is more desirable than mixing the logits from theoretical and empirical perspectives (Bai et al.) 2024b/a). Here, we denote the proposed mixed model with fmix: : _dd RRc. Specifically, the 7th output logit of the mixed model follows the formulation
fuix, == log ((1 a) (x) + aa.aa (z)),
(1)
for all E [c], where a E 1/2, 1] adjusts the mixing weight probability space, and the natural logarithm maps the mixed probability back to the logit space without changing the predicted class for interchangeability with existing models. If the desired output is the probability o fmix (-), the logarithm can be omitted.
| The mixing operation is performed in the
Bai et al. 2024b have shown that should be no smaller than 1/2 for fmix(-) to have non-trivial robustness.
4

---

Published in Transactions on Machine Learning Research (08/2024)
3 Base Classifier Confidence Modification
We observe that the robust base classifier hro often enjoys benign confidence property: it is much more confident in correct predictions than in mispredictions. Le., (-)ss confidence margin is much higher when it makes correct predictions. Even some input is subject to attack (which vastly decreases the confidence margin of correct predictions), if it correctly predicted, its margin is still expected to be larger than incorrectly predicted natural examples. Section 5.2 verifies this property with multiple model examples, and Appendix visualizes the confidence margin distributions.
As result, when mixing the output probabilities hrob(-) and Ista(-) on clean data, where Istd Ista is expected to be more accurate than h-ob(-), 9-ta(-) can correct (-obs-))s mistake because hrob(-) is unconfident. Meanwhile, when the mixed classifier is under attack and h-)o(()) becomes much more reliable than Iaaa-), (-ob high confidence in correct predictions can overcome (')ss))'s misguided outputs. Hence, even when (sa(()'ss 'ss robust accuracy is near zero, the mixed classifier still inherits most of I-ras-))s robustness. Combining the above two cases, we can see that the "benign confidence property" of h-ob(.) allows the mixed classifier to simultancously take advantage of (:sas)'s high clean accuracy and (-)bs))ss adversarial robustness. As result, modifying and enhancing the base classifiers' confidence has vast potential to further improve the mixed classifier.
Note that this benign confidence property is only observed on robust classifiers. Neural classifiers trained without any robustness considerations often make highly confident mispredictions when subject to adversarial attack. These mispredictions can be even more confident than correctly predicted unperturbed examples, often seeing confidence margins very close to 1. As a result, (et(()) does not enjoy the benign confidence property, and its confidence property is in general detrimental to the mixture.
3.1 Accurate Base Classifier Temperature Scaling
We start with analyzing the accurate base classifier Iota()), with the goal of mitigating its detrimental confidence property. One approach to achieve this is to scale up (sta())ss logits before the Softmax operation. To this end, we consider temperature scaling (Hinton et al. 2015). Specifically, we construct the temperature TT) whose 7th entry is scaled model gstd
Batd (ar) == (x)a
TS(T)
for all i, where T 2 0 is the temperature constant. To scale up the confidence, T should be less than 1.
To understand this operation, observe that temperature scaling increases gaa(-))s confidence in correct clean examples and incorrect adversarial examples simultaneously. However, because (-'s confidence under atlack already close to before scaling, the increase in atlacked misprediction confidence is negligible due to the saturation of the Softmax function. Since Ista(-) becomes more confident on correct examples with the mispredicting confidence almost unchanged, its detrimental confidence property is mitigated.
The extreme selection for the temperature T is 0, in which case the predicted probabilities gog a one-hot vector corresponding to 9eca(-)ss predicted class. By scaling with T = 0, the detrimental confidence property of (-)a(.) is completely climinated, as constant margin of preciscly 1 is enforced everywhere. Note that we still hope to preserve the ranking of class-wise outputs (+), that we can preserve the high accuracy of (+). Given this requirement, eliminating (ata()'s detrimental confidence property by enforcing a consistent margin is the best one can expect. Appendix D4 verifies that T = 0 produces the best empirical effectiveness among several temperature values. Appendix B.2 discusses how our attacks circumvent the non-differentiabblity resulting from using T = 0.
TS(O) (-) becomes
In addition to eliminating the detrimental confidence property of gaaa-),, selecting T 0 also simplifies the analysis on the robust base model hrob(.) by establishing a direct correlation between h-ob confidence and the mixed classifier's 's correctness, thereby allowing for tractable and efficient optimization. Hence, we select (.) as the accurate base classifier for the remaining analyses.
T 0 and use 9
1S(0)/
5

---

Published in Transactions on Machine Learning Research (08/2024)
4 MixedNUTS - -Nonlinearly Mixed Classifier
In contrast to the accurate base classifier, the robust base classifier (+)ss))s confidence property is benign. To achieve the best accuracy-robustness trade-off with the mixed classifier, we need to augment this benign properly as much as possible. While a similar temperat ure ure scaling operation can achieve some of the desired effects, its potential is limited by applying the same operation to confident and unconfident predictions, and is therefore suboptimal. To this end, we extend confidence modification beyond temperature scaling (which is linear) to allow nonlinear logit transformations. By introducing nonlinearities, we can treat low-confidence and high-confidence examples differently, significantly amplifying (.)sss)ss benign property and thereby considerably enhancing the mixed classifier's accuracy-robustues balance 2
4.1 Nonlinear Robust Base Classifier Transformation
We aim to build a nonlinearly mapped classifier (-) = (hrob (-), where M E M: IR + R is nonlinear transformation applied to the classifier (-yas()'s logits, and .M is the set of all possible transformations. The prediction probabilitics from this transformed robust base model are then mixed with those from Ietd ()) to form the mixed classifier (.) following
TS(0),
For the optimal pecuracy-robustnee trade-off, we select an M that maximizes the clean accuracy of falix (-) while maintaining the desired robust accuracy. Formally, this goal is described as the optimization problem
max MEM, aE(4/2.1] subject to Pix,ywD arg max Snix, (X + +0jar (X)) YJ mix
P(x,))D arg
arg max filx.(()) =Y]
(2)
2r Tmxx
where D is the distribution of data-label pairs, r is the desired robust accuracy of f ()), and 87m (x) is the minimum-mare margin perturbation of fiMx (.) at x. Note that (.) implicitly depends on M and a.
The problem (2) depends on the robustness behavior of the mixed classifier, which is expensive to probe. Ideally, the optimization should only need the base classifier properties, which can be evaluated beforehand. To allow such a simplification, we make the following two assumptions.
Assumption 4.1. On unattacked clean data, if h h (-) makes a correct prediction, then Geta(-) is also correct.
Assumption 4.1 allows to focus on examples correctly classified by the accurate base classifier Ietd (-) but not by the robust base model (-) when optimizing the transformation M()) to maximize the clean accuracy of the mixed classifier. Under Assumption 4.1 we can safely discard the opposite case of gstd being incorrect while (.) being correct on clean data. Assumption 4.1 makes sense because gatd clean accuracy should be considerably higher than (-)'s))s 's to justify mixing them together, and training standard classifiers that noticeably outperforms robust models clean data is usually possible in practice.
TT(0)
TS(O) (.)
TS(O) (!)'s
Assumption 4.2. The transformation (-) does not change the predicted class due to, e.g., monotonicity. Namely, it holds that arg max; M ((hrat(())): = arg max; (x)) for all x.
We make this assumption because we want the logit transformation to preserve the accuracy of hyab(.). While it is mathematically possible to obtain an (-) that further increases the accuracy of hrob(-), finding it could be as hard as training a new improved robust model. Hence, the best one would expect from a relatively simple nonlinear transformation is enhance (-)'s benign confidence margin property while not changing the predicted class. Later in this paper, we will propose Algorithm ① to find such transformation.
These two assumptions allow us to decouple the optimization of M(-) from the accurate basc classifier gat( This is because as proven in (Bai et al. 2024b Lemma 1), the mixed classifier is guaranteed to be robust when A. is robust with margin no smaller than 1-a (with the implicit assumption a 2 0.5. Hence, we can
-
solve the following problem as a surrogate for our goal formulation (2)
P M MEM, (E Px [maM. (X) 2 1ーa α
min
subject to
P Pz-xz
[m] M rob
(Z)
B,
(3)
2The same uonlincar logit transformation not applicd to the accurate base classificr because its confidence property is not benign. As explained in Section 3.1 eliminating Gesa detrimental confidence property by enforcing a constant margin with one-hot encoding is the hest, one can expect.
6

---

Published in Transactions on Machine Learning Research (08/2024)
where X clean is the distribution formed by clean examples incorrectly classified by (.), &a is the distribution formed by attacked examples correctly classified by 1.)b(")) X, Z are the random variables drawn from these distributions, and 3 E [0, I] controls the mixed classifier's desired level of robust accuracy with respect to the robust accuracy of h-ob(-).
Note that. (3) no longer depends on ()a()), allowing for replacing the standard base classifier without. re-solving for a new transformation (.). The following two theorems justify approximating (2) with by
characterizing the optimizers of (3]
Theorem 4.3. Suppose that Assumption and h.ob(.) respectively. If [ 2 "IM " Treas then a solution to (3) is feasible for (2)
4.2 holds. Let r ryar and Thon denote the robust accuracy of fm)(())
Theorem 4.4. Suppose that Assumption
4.1 holds. Furthermore, consider an input random variable X and suppose that the margin of (x,6()) is independent of whether (xta(X) is correct. Then, minimizing the objective of 3 is equivalent to marimizing the objective of (2)
The proofs of Theorem 4.3 and Theorem 4.4 are provided in Appendices A.1 and A.2 respectively. In
Appendix D.8. and Appendix D.8.2] we discuss the minor effects of slight violations to Assumption [1. and Assumption 1.2 respectively. Moreover, the independence assumption in Theorem 1.4 can be relaxed with minor changes to our method, which we discuss in Appendix [D.8.3] Also note that Theorems 4.3 and 4.4 rely
using T = 0 for (ata-))ss temperature scaling, justifying this temperature setting selected in Section 3.1
4.2 Parameterizing the Transformation M
Optimizing the nonlinear transformation M(.) requires representing with parameters. To avoid introducing additional training requirements vulnerable backdoors, the parameterization should be simple (i.e., not introducing yet another neural network). Thus, introduce manually designed transformation with only three parameters, along with an algorithm to efficiently optimize the three parameters.
Unlike linear scaling and the Softmax operation, which are shift-agnostic (i.e., adding a constant to all logits docs not change the predicted probabilitics), the desired nonlinear transformations' behavior heavily depends on the numerical range of the logits. Thus, to make the nonlinear transformation controllable and interpretable, we pre-process the logits by applying layer normalization (LN): for each input x, we standardize the logits (2) to have zero mean and identity variance. We observe that LN itself also slightly increases the margin difference between correct and incorrect examples, favoring overall formulation as shown in Figure 6. This phenomenon is further explained in Appendix DD.7
Among the PN logits, only those associated with confidently predicted classes can be large positive values. To take advantage of this property, we use a clamping function Clamp(-), such as ReLU, GELU, ELU, or SoftPlus, to bring the logits smaller than a threshold toward zero. This clamping operation can further suppress the confidence of small-margin predictions while preserving large-margin predictions. Since correct examples often enjoy larger margins, the clamping function enlarges the margin gap between correct and incorrect examples. We provide an ablation study over candidate clamping functions in Appendix D.2 and empirically select GELU for our experiments.
Finally, since the power functions with greater-than-one exponents diminish smaller inputs while amplifying larger ones, we exponentiate the clamping function outputs to a constant power and preserve the sign. Putting overything together, with the introduction of three scalars s, p, and e to parameterize M()), the combined nonlinearly transformed robust base classifier M[s..)) (-) becomes
M(e.p.c) (x) = 8 | 1i
Clamp(c) (x) sgn (h
Clamp(e)
(x)) where T
Clamp(e) = Clamp (x)) + c). (4)
Here, s E (0, +0o) is scaling constant, E +oo) is an exponent constant, and e E R is a bias constant that adjusts the cutoff location of the clamping function. With a slight abuse of notation, Ms,, c))-) denotes the transformation parameterized with s, p, and c. In [4] we apply the absolute value before the exponentiation to maintain compatibility with non-integer p values and use the sign function to preserve the sign. Note that when the clamping function is linear and p 1, [4) degenerates to temperature scaling with LN. Hence, an optimal combination of 8, P, and is guaranteed to be no worse than temperature scaling.
7

---

Published in Transactions on Machine Learning Research (08/2024)
Note that the nonlinear transformation ((.. (.) generally adheres to Assumption 4.2 While Assump-
tion 4.2 may be slightly violated if GELU is chosen as the clamping function due to its portion around zero being not monotonic, its effect is empirically very small according to our observation, partly because the negative slope is very shallow. We additionally note that the cortified robustness results presented in (Bai
et al.
2024b) also apply to the nonlinearly mixed classifiers in this work.
With the accurate base classifier's temperature scaling and the robust base classifie'ss nonlinear logit transformation in place, the overall formulation of MixedNUTS becomes
mix
( (x) log ((1 - (1-a)- Iard (x) + a 1yoh ( (x)),
TS(0)
(5)
as illustrated in Figure
!
4.3 Efficient Algorithm for Optimizing 8, p. G, and a
With the nonlinear transformation parameterization in place, the functional-space optimization problem reduces to the following algebraic oplimization formulation:
3
min
IP Px-xx clean
[m,
p.) Rob
(X) z
1-0]
s.p.caRR
a
(6)
Im ao subject to P PZ~xx
(Z) 2 l-a 2B 820, p20, 1/25a51.
rab
Exactly solving (6) involves evaluating mi mageome (x) for every x in the support of the distribution of correctly
predicted adversarial examples K This is intractable because the support is continuous set and the distributions X and Xy implicitly depend on the optimization variables s, p, and c. To this end, we clean and Aady with a small set of data. Consider the subset of clean examples incorrectly classified by 1to (.), denoted as Xx clean and the subset of attacked examples correctly classified by 1 (-),
approximate x-X x
denoted as 2 ady- Because we use N.N() (.) instead of Nob MM(xp)))) (.) to obtain X and X using them as X
surrogates to clean and X decouples the probability measures from the optimization variables.
Despite optimizing s, P, C, and a on a small set of data, overfitting is unlikely since there are only four parameters, and thus small number of images should suffice. Appendix D.3 analyzes the effect of the data subset size optimization quality and confirms the absence of overfitting.
The minimum margin m (rcppee) (I) also depends on the optimization variables P. c, and a, as its calculation 2oob
requires the maninn perturbation for M(s.nc))) (+) around z. Since finding m mi Meeee (x) for all 8, P. rob
and e combinations is intractable, we seek to use an approximation that does not depend on 8, P, and Specifically, the approximation is m, min. (z)), defined as
Mg Mo (x) = m ar e) (x + 8ALN (x)) x= (((aae)) (x + 8 mr (x)) = my p.c (z), rob
Proh
rob
(-) around x obtained from a strong where JALNN (x) an empirical Sinimum-margin perturbation of h A.N
adversarial attack. Note that calculating a (x) does not require attacking MM(aP.))) (.) and instead attacks ANN (.), which is independent of the optimization variables, ensuring optimization efficiency. To obtain m, sab on (xr) in Appendix B.1 we propose minimun-- AuloAllack (MMAA), an AutoAttack variant that keeps track of the minimun margin while generating perturbations. While some components of MMAA require I. (.)'s gradient information, Algorithm can still apply after some modifications even the base classifiers are black boxes with unavailable gradients, with the details discussed in Appendix D.5]
Since the probability measures and the perturbations are now both decoupled from s, P. C, a, only need to run MMAA once to estimate the worst-case perturbation, making this hyper-parameter search problem efficiently solvable. While using h (-) as a surrogate hyab M(s.se)) (-) introduces a distribution mismatch, we expect this mismatch to be benign. To understand this, observe that the nonlinear logit transformation (4) generally preserves the predicted class due to the (partially) monotonic characteristies of GELU and the sign-preserving power function. Consequently, we expect the accuracy and minimum-margin perturbations of 1 M((.p.))) (.) to be very similar to those of (.) Appendix D.9 empirically verifies this speculated proximity.
8

---

Published in Transactions on Machine Learning Research (08/2024)
Algorithm 1 Algorithm for optimizing 8, p, C, and a.
1: Given an image set, save the predicted logits associated with mispredicted clean images { (x) x E F Xdean)
2: Run MMAA on (-0)()) and save the logits of correctly classified perturbed inputs { (x) A'dr
3: Initialize candidate values 81,-..,81; Pi..... Pony Cha...g
4: for & for I..... do
for Ps for j = ..... do 6: for Cs for k = 1,...,
7 do Obtain mapped logits {hrob Mieersaee (x) Kaay.. Calculate the margins from the mapped logits {m, Mi (x) Aadu). "uob
7:
9: Store the bottom 3-quantile of the margins as v (corresponds 1-x (T)
10: Record the current objective 0lhh + XeXe
[m mo )) 2 qi Aob
11: end for
12: end for
13: end for
,,"",")) k²) )- = arg min, J.R. 0lsk 14: Find optimal indices (i",
15: Recover optimal mixing weight == 1/(1tjj 1 yea
return = p" p e* c a²
To simplify notations, let Addv far kaa (z) Eddr denote all correctly predicted minimum-margin perturbed images for 1 (.). Inherently, it holds that
P ZEA udl
[m, Mo (2) T-a Maa
zE odv
[mn o (Z) 1-9] ~ Pi Rooh
[m]] min
(Z)
1-a
The approximate hyper- parametere selection problem, which can be solved in surrogate
6 is then
min s.p.c
IP E clean
m Moc.p (X) 2 1-a
(7)
subject to ZE
[ MM
(z) 2 l-a 2 B 8 $20, p20, 1/25551.
Since (7) only has four optimization variables, it be solved via grid search algorithm. Furthermore, the ايي 2 should always be active at optimality Hence, we treat this constraint as equality, reducing the searching grid dimension to three. Specifically, we sweep over a range of 8, p, and c to form the grid, and calculate the a value that binds the chance constraint for cach combination. Among the grid, we then select an 8, p, e combination that minimizes
constraint IP z [m M
77)
objective.
The resulting algorithm is Algorithm I As discussed above, this algorithm only needs to query MMAA's APGD components once small set of validation data, and all other steps simple mathematical operations requiring minimal computation. Additionally, note that the optimization precision of Algorithm is governed by the discrete nature of the evaluation dataset. Le., with dataset consisting of 10,000 examples (such as the CIFAR-10 and CIFAR-100 evaluation sets), the finest optimization accuracy one can expect is 0.01% in terms of objective value (accuracy). Hence, it not necessary to solve (7) to a high accuracy. Moreover, as shown in Figure in Appendix D.1 which analyzes the sensitivity the formulation (7) objective value with respect to 8, p, and c, the optimization landscape relatively smooth. Therefore, a relatively coarse grid (512 combinations in our case) can find a satisfactory solution, and hence Algorithm is highly efficient despite the triply nested loop structure. Furthermore, the base classifier raw logits associated with A.,.(.) (-)'s minimum-margin perturbations do not depend on P, c and can be cached. Hence, the number of forward loops is agnostic to the search space size.
All of the above makes Algorithm I efficiently solvable. In practice, the triply-nested grid search loop can completed within ten seconds on a laptop computer, and performing MMAA on 1000 images requires 3752/1033 seconds for CIFAR-100/Imagee
with a single Nvidia RTX-8000 GPU.
TTo understand this, suppose that some combination of s, P. c, and o, this inequality satisfied strictly. Then, will be possible decrease (iee., increase 1ー without violating this constraint, and thereby further reduce the objective value.
9

---

Published in Transactions on Machine Learning Research (08/2024)
Raw Logits
Raw Probabilities
Transformed Probabilities
T=
T= 0.06
T= 0.003
Raw Probabilities
Class
Case 1 Large margin
0.8
Margin: 0.698
1.0
Margin:
0.93 (1)
B
-1.1
-1.0
0.097
0.023 0.023
Class A
B
C
Class
Class A
B
C
Case
Transformed Probabilities
Case
Small margin
1.0
Margin: 0.047
Margin: 0.0013
Case
0.49 0.45
0.33 0.33 0.33
-1.0
0.06
Class A
B
C
Class
Class A
B
Smaller more confident
Figure 3: The raw logits, the corresponding prediction probabilities, and the probabilities computed with the transformed logits. Our transformation augments the confidence margin difference between the two scenarios.
Figure 4: Probability trajectories on the probabil- ity simplex formed by temperature scaling, with or without the logit transformation. The transfor- mation reduces confidence when classes compete.
4.4 Visualization of the Nonlinear Logit Transformation (s.,p.))
To better understand the effects the proposed nonlinear logit transformation M(s, p, c)(-), we visualize how affects the base classifier prediction probabilities when coupled with the Softmax operation. Consider a three-class (A, B, and C) classification problem, with two example logit vectors. The first example simulates the case where class A is clearly preferred over the rest (large margin), while the second example illustrates competition between classes B and C (small margin). The raw logits, the corresponding prediction probabilities, and the probabilities computed with the transformed logits are visualized in Figure 3 Clearly, the margin is further increased for the large margin case and shrunk for the small margin case, which aligns with the goal of enlarging the benign confidence property of the base classifiers.
For further demonstration, we adjust the overall confidence level for the above two cases and compare how their prediction probabilities change with the confidence level. Specifically, by applying temperature scaling and varying the temperature 7, the prediction probability vectors form trajectories on the probability simplex, whose vertices represent the classes [ For example, small temperature increases the overall prediction confidence, moving the vector toward vertex. Conversely, a large temperature attracts the prediction probability to the simplex's centroid. By contimuuusly adjusting the temperature, we obtain trajectories that connect the centroid to the vertices. By comparing the trajectories formed with or without the nonlinear logit transformation (a (1/4))))))) and (1 ())))))))))))))) we can better understand the transformation's properties.
Figure shows the prediction probability vectors al three example temperature values, well the trajectories formed by continuously varying the temperature. We observe that the nonlinear logit transformation significantly slows down the movement of the small margin case from the centroid to the vertex. Moreover, the trajectory with the transformation is straighter and further from the edge BC, implying that the competition between classes and C has been reduced. In the context of mixed classifiers, the nonlinear transformation reduces the robust base classifier's relative authority in the mixture when it encounters competing classes, thereby improving the mixed classifier's 's accuracy-robusiie trade-off.
5 Experiments
We use extensive experiments to demonstrate the accuracy-robustaes balance of the MixedNUTS classifier f Mis" "p (²)(()) focusing on the effectiveness of the nonlinear logit transformation. Our evaluation uses CIFAR-
Krizhevsky 2009), CIFAR-100 (Krizhevsky 2009). and ImageNet (Deng et al. 2009) datasets. For
'Here, the purpose of temperature scaling different from Section In Section temperature scaling mitigates Seta())'s detrimental confidence property. Here, scaling with variable temperatures generates probability trajectories for visualization.
10

---

Published in Transactions on Machine Learning Research (08/2024)
CIFAR-10 (f.., +/ss))
CIFAR-100 (f... = +/ass)
ImageNet (f... 4/ios)
Robust Accuracy (AutoAttack)
MixedNUTS
MiXedNUTS
MS (ours) Robust 58, 50.5
Robust Base Classifier (RBC)
RBC
(ours) 6obos066.7755 71%
RBC Cleans 73 322%
(ours) Robwot:41 80%
RBC Cleset 7% Mart Rubbal
Clean: Us 23-27%
Rubust
Accurate Base
Classifier (ABC)
Mixed
Mixed Clean: 78.0356
Mixed
Clean: S44.6356
Cleam: 80 82ff
Mixed (Bai et al, 2023b)
Robustic8.co (64%
Rollsst:00. 1356
Rotuust DG 600%
MixedNUTS (ours)
ABC
ABC
ABC Clean: 160 Robust O D005
Ciean: 56, 2005 Robust 00.%
Clean: 10.06 Robust 0 0075
vallitation
specified RabustBench
Clean Accuracy
Clean Accuracy
Clean Accuracy
Figure 5: MixedNUTS balances the robustness from its robust base classifier and the accuracy from its standard base classifier. The nonlinear logit. transformation helps MiXedNUTS achieve a much better accuracy-robuse trade-off than a baseline mixed model without transformation. Appendix C.1 reports the basc model details and the optimal 8, P, c, a values.
CIFAR-10 10 (Boay 4/a0s)
CIFAR-100 (6x, = 9/ss)
ImageNet (fx.,e = 4/ms)
Clean Correct
1.0
.966
1.0
.973
.837
.980
Confidence Margin
Clean
0.8
0.8
0.8
.525
Incorrect
.534
0.6
.386
0.6
.376
.481
AutoAttacked
Correct
0.4
0.4
0.4
Auta/Attacked
0.2
0.2
0.2
Incorrect
0.0
0.0
0.0
Margin Gap (Hagher bollor)
Reabl)))
(.)
(-)
(ma(-)
(oob())
M2 of (e)
Bmal-)
beaul-) Wil Maah an (010)(
Figure 6: The median confidence margin of the nocurate/robust base classifier gsta(-)/ the layer-normen logits M..h() (-), and the nonlinearly transformed model Aoo M(="''" =0'=a)()) on clean and AutoAttacked data, grouped by prediction correctness. The number above each bar group is the "margin gap", defined as the difference between the medians on clean incorrect inputs and AutoAttacked correct ones. A higher margin gap signals more benign confidence property, and thus better aecuracy-robustnes trade-off for the mixed classifier.
each dataset, we select the model with the highest ro- bust accuracy verified on RobustBench (Croce et al.] 2021) as the robust base classifier Ayob and se-
Table 1: MixedNUTS's error rate changes relative Lo the robust base classifier (more negative is better).
lect a state-of-the-art standard (non-robust) model enhanced with extra training data the accurate base classifier (-). Detailed model information is reported in Appendix C.I.
Clean (4)
Robust (AutoAttack) (L)
CIFAR-10
28.53%
+1.70%
CIFAR-100
-31.72%
+1.52%
ImageNet
-12.14%
+2.62%
As an ensemble method, in addition to being training- free, MixodNUTS also highly efficient during infer-
ence time. Compared with a state-of-the-art robust, classifier, MixedNUTS's increase in inference FLOPs is low as 24.79%. A detailed comparison and discussion inference efficiency can be found in Appendix C.2
All mixed classifiers are evaluated with strengthened adaptive AutoAttack algorithms specialized in attacking MixedNUTS and do not manifest gradient obfuscation issues, with the details explained in Appendix B.2
5.1 Main Experiment Results
Figure compares MixedNUTS with its robust base classifier, its accurate base classifier, and the baseline method Mixed (Bai et al. 2024b) on three datasets. Specifically, Mixed is a mixed classifier without the nonlinear logit transformations. Figure 5 shows that MiXedNUTS consistently achieves higher clean accuracy and better robustness than this baseline, confirming that the proposed logit transformations mitigate the overall rccuracy trade-off.
Table I compares MixedNUTS's relative error rate change over its robust base classifier, showing that NUTS vastly reduces the clean error rate with only slight robust error rate increase. Specifically, the
11

---

Published in Transactions on Machine Learning Research (08/2024)
CIFAR-10 (foc)
CIFAR-10 (42)
CIFAR-10U (fx)
ImageNet (Foc
Clean
Confidence Margin
1.0
.710
.760
Correct
Clean
0.8
.486
Incorrect
0.6
3945
.400
AutoAttacked
192
.179
Correct
0.2
AutoAttackedd
Incorrect
0.0
Pertg
Pang al (2022)
Debonedetts an al, (2022) ACI
Reatlen la
No at (2020)
Wang et at. (2023) 18
Wang all (2023) vo
wotN TD to (E data) 68.3%
Swited
Lia et al. (20223)
Singh al. (2023) Nekehak dN / Aes
Margin Gap
BL.6% / 11.44
$0.1%% 18
99.0% 66
(higha betlor)
4%% 40.45
re-M5 (2022) )
5a.5% data))
$2.9%
(2025)
Figure 7: The median confidence margin of diverse set of robust models with the logits standardized via layer normalization. All models enjoy higher margins on correct predictions than on incorrect ones for clean and adversarial inputs alike. The percentages below each model name are the clean/AutoAtee AutoAttack accuracy.
relative clean error rate improvement 6 to 21 times more prominent than the relative robust error rate increase. Clearly, MixedNUTS balances accuracy and robustness without additional training.
Figure B compares the robust base classifier's confidence margins clean and attacked data with or without our nonlinear logit transformation [4] For each dataset, the transformation enlarges the margin gap between correct and incorrect predictions, especially in terms of the median which represents the margin majority. Using M MM*="."'' (.)) instead of h-r makes correct predictions more confident while keeping the mispredictions less confident, making the mixed classifier more accurate without losing robustness.
Figure 2 compares MixedNUTS with existing methods with the highest AutoAttack-validatee adversarial robustnessss, confirming that MixedNUTS noticeably improves clean accuracy while maintaining competitive robustness. Moreover, since MixedNUTS can use existing or future improved accurate or even robust models as base classificrs, the entries of Figure should not be regarded as pure competitors.
Existing models suffer from the most pronounced accuracy-robustnes trade-off CIFAR-100, where MixedNUTS offers the most prominent improvement. MixedNUTS boosts the clean accuracy by 7.86 percentage points over the state-of-the-art non-mixing robust model while reducing merely 0.87 points in robust accuracy. In comparison, the previous mixing method (Bai et al. 2024a) sacrifices 3.95 points of robustness (4.5x MixedNUTS's degradation) for a 9.99-point clean accuracy bump using the same base models. Moreover, (Bai et al. 2024a) requires training an additional mixing network component, whereas MixedNUTS is training-free (MixeNNUSS is also compatible with the mixing network for even better results). Clearly, MixedNUTS utilizes the robustness of (-) more effectively and efficiently.
On CIFAR-10 and ImageNet, achieving robustness against common attack budgets penalizes the clean accuracy less severely than on CIFAR-100. Nonetheless, MixedNUTS is still effective in these less suitable cases, reducing the clean егrоr rate by 18.53%//12.L (relative) while only sacrificing 1.91 1.91% 101%%0.98% (relative) robust accuracy on CIFAR-10/ 10AR--0/ImmaeNet compared to non-mixing methods. On CIFAR-10, MixedNUTS matches (Bai et al.
2024a 's clean accuracy while reducing the robust error rate by 5.17% (relative).
With the nonlinear transformation in place, is still possible to adjust the emphasis between clean and robust accuracy at inference time. This can be achieved by simply re-running Algorithm I with different B value. Note that the MMAA step in Algorithm does not depend on B, and hence can be cached to speed up re-runs. Mcanwhile, the computational cost of the rest of Algorithm 1 is marginal. Our experiments use 3 = 98.5% for CIFAR-10 and -100, and use 8 = 99.0% for ImageNet. The optimal p, values and the searching grid used in Algorithm I are discussed in Appendix C.1
5.2 Confidence Properties of Various Robust Models
MixedNUTS is built upon the observation that robust models are more confident in correct predictions than incorrect ones. Figure 7 confirms this property across existing models with diverse structures trained with different loss functions across various datasots, and hence MixodNUTS applicable for a wide range of base
12

---

Published in Transactions on Machine Learning Research (08/2024)
classifier combinations. For fair confidence margin
comparison, all logits are standardized to xero mean
Robust Accuracy (%)
50-
Mixed
MixedNUTS (ours)
and identity variance (corresponding to / (-) before
TRADES
45
converted into probabilitics. Appendix C. presents
TRADES (larger)
histograms to offer more margin distribution details.
40
Figure 7 illustrates that existing CIFAR-10 and -100 models have tiny confidence margins for mispredictions and moderate margins for correct predictions, implying that most mispredictions have a close runner-up class. ImageNet classifiers also have higher confidence in cor- rect predictions than incorrect ones, However, while the logits are always standardized before Softmax, the ImageNet models have higher overall margins than the CIFAR ones. This observation indicates that ImageNet models often do not have strong confounding class despite having more classes, and their non- con-predicted classes' probabilities spread more evenly.
20
0
84
86
88
90
94 96
Clean Accuracy (%)
Figure 8: Accuracy-robustness trado-off compari- son between MixedNUTS, mixed classifier without nonlinear transformation, and TRADES on 1000 CIFAR-10 images. TRADES (larger) denotes a larger TRADES model trained from scratch that has the same size as MixedNUTS.
5.3 Accuracy-Robustness Trade-Off Curves
In Figure we show MixedNUTS's robust accuracy as a function of its clean accuracy. We then compare this accuracy-robustnec trade-off curve with that of the mixed classifier without nonlinear logit transformation (Bai et al., 2024b) and that of TRADES (Zhang et al., 2019). a popular adjustable method that aims to improve the trade-off. Note that adjusting between accuracy and robustness with TRADES requires tuning its training loss hyper-parameter ßTR and training a new model, whereas the mixed classifiers are training-free and can be adjusted at inference time.
Specifically, select CIFAR-10 WideResNet-34-11 models trained with PIR = 0, 0.1, 0.3, and 6 as the baselines, where 0 corresponds to standard (non-robust) training and 6 is the default which optimizes robustness. For a fair comparison, we use the TRADES models with BTR 0 and 6 to assemble the mixed classifiers. For MixedNUTS, we adjust the level of robustness by tuning 3, the level-of-robustness hyperparameter of Algorithm ! specifically considering B values of 1, 0.96, 0.93, 0.8, and O.
Figure 8 confirms that training free mixod classifiers, MiXedNUTS and Mixed (Bai al., 2024b), achieve much more benign rccuracy- trade-offs than TRADES, with MixedNUTS attaining the best balance.
Since MixedNUTS is ensemble, it inevitably results in larger overall model than the TRADES baseline. To clarify that MixedNUTS's performance gain is not due to the increased size, we train a larger TRADES model (other training settings are unchanged) to match the parameter count, inference FLOPS,, and parallelizability. As shown in Figure S this larger TRADES model's clean and robust accuracy does not improve over the original, likely because the original training schedule is suboptimal for the increased size. This unsurprising, as has been shown that no effective one-size- site--iit-allll adversarial training parameter settings exist (Duesterwald let al. 2019). Hence, an increased inference computation does not guarantee better performance on its own. To make model benefit from larger size via training, neural architecture and training setting searches are likely required, which is highly cumbersome and unpredictable. In contrast, MixedNUTS is a training-free plug-and-play add-on, enjoying significantly superior practicality.
6 Conclusions
This work proposes MixedNUTS, versatile training-free method that combines the output probabilities of a robust classifier and an accurate classifier. By introducing nonlinear base model logit transformations, MixedNUTS more effectively exploits the benign confidence property of the robust base classifier, thereby achieving a balance between clean data classification accuracy and adversarial robustness. For performance- driven practitioners, this balance implies less to lose in using robust models, incentivizing the real-world deployment of safe deep learning systems. For researchers, as improving the rccuracysrobustness trade-off with
13

---

Published in Transactions on Machine Learning Research (08/2024)
single model becomes harder, MixedNUTS identifies building base models with better margin properties novel alternative direction to improve the trade-off in ensemble setting.
References
George Adam and Romain Speciel, Evaluating ensemble robustness against adversarial attacks. arXiv preprint arXiv:2005.057: 055550 2020.
Manaar Alam, Shubhajit Datta, Debdeep Mukhopadhyay, Arijit Mondal, and Partha Pratim Chakrabarte Resisting adversarial attacks in deep neural networks using diverse decision boundaries. arXiv preprint wwXii:2208.08697 2022.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: A query-efficienl black-box adversarial attack via random search. In Europeon Conference on Computer Vision, 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International Conference on Machine Learning, 2018a.
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. In International Conference on Machine Learning, 2018b.
Yatong Bai, Tanmay Gautam, Yu Gai, and Somayeh Sojoudi. Practical convex formulation of robust one-hidden-layer neural network training. In American Control Conference, 2022.
Yatong Bai, Tanmay Gautam, and Somayeh Sojoudi. Efficient global optimization of two-layer relu networks: Quadratic-tine time algorithms and adversarial training. SIAM Journal on Mathematics of Data Science, 5(2): 446-474, 2023.
Yatong Bai, Brendon G Anderson, Aerin Kim, and Somayeh Sojoudi. Improving the accuracy-robustiess trade-off of classifiers via adaptive smoothing. STAM Journal Mathematics of Data Science, 6(3): 788-814, 2024a.
Yatong Bai, Brendon Anderson, and Somaych Sojoudi. Mixing classifiers to alleviate the accuracy-robustnes trade-off. In Annual Learning for Dynamics and Control Conference, 2024b.
Yogesh Balaji, Tom Goldstein, and Judy Hoffman. Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets. arx preprint arXiv:1910.0 08051, 2019.
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXic:1908.3432 2013.
Jaydeep Borkar and Pin-Yu Chen. Simple transparent adversarial examples. arXiv preprinl ar Xxv::210550000 2021.
Ruisi Cai, Zhenyu Zhang, and Zhangyang Wang. Robust weight signatures: gaining robustness as easy as patching weights? In International Conference on Machine Learning, 2023.
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. In IEFE Symposium on Security and Privacy, 2017.
Tianlong Chen, Zhenyu Zhang, Sijia Lin, Shiyu Chang, and Zhangyang Wang. Robust overfitting may be mitigated by properly learned smoothening. In International Conference on Learning Representations, 2021.
Minhao Cheng, Qi Lei, Pin Yu Chen, Inderjit Dhillon, and Cho Jui Hsich. CAT: Customized adversarial training for improved robustness. In International Joinl Conference on Arlificial Intelligence, 2022.
Kenneth Co, David Martinez-Rego, Zhongyuan Hau, and Emil Lupu. Jacobian ensembles improve robustness trade-offs to adversarial attacks. In Artificial Neural Networks and Machine Learning, 2022.
14

---

Published in Transactions on Machine Learning Research (08/2024)
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference on Machine Learning, 2020-
Francesco Croce, Maksym Andriushchenko, Vikash Sehway, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. RobustBench: standardized adversarial robustness benchmark. In Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2021.
Jiequan Cui, Zhuotao Tian, Zhisheng Zhong, Xiaojuan Qi, Bei Yu, and Hanwang Zhang. Decoupled kullback-leibler divergence loss. ar Xrxiv preprinl arXiv:2305.13448 2023.
Edoardo Debenedetti, Vikash Sehwag, and Prateek Mittal. A light recipe to train robust vision transformers. In Conference Secure and Trustworthy Machine Learning. 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition, 2009.
Evelyn Duesterwald, Anupama Murthi, Ganesh Venkataraman, Mathicu Sinn, and Deepak Vijaykeerthy. Exploring the hyperparameter landscape of adversarial robustness. arXiv preprint orXiv:100: 1905.09988 2019.
Mudasir A Ganaie, Minghui Hu, AK Malik, M Tanveer, and PN Suganthan. Ensemble deep learning: A review. Engineering Applications of Artificial Intelligence, 115:105151, 2022.
Tan J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples.
In International Conference on Learning Representations, 2015.
Sven Gowal, Jonathan Uesato, Chongli Qin, Po-Sen Huang, Timothy Mann, and Pushmeet Kohli. An alternative surrogate loss for PGD-based adversarial testing. arXiv preprint arXiv:1910.00832 2019.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint 0TT:::2000005599 2020.
Sven Gowal, Sylvestre-Alvise Rebuffi, Olivia Wiles, Florian Stimberg, Dan Andrei Calian, and Timothy A Mann. Improving robustness using generated data. In Annual Conference on Neural Information Processing Systems, 2021.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. International Conference on Machine Learning, 2017.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXio preprint arXiv:15033 500.00231, 2015.
Gabriel Ilaaroo, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. ar preprint orXiv:2212.0.0 2.0989,, 2022.
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In International Conference on Machine Learning, 2018.
Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby. Big transfer (BiT): General visual representation learning. In European Conference on Computer Vision, 2020.
Alex Krizhevsky. Learning multiple layers of features from tiny images. https://www.css.trootto.ee htti://errinngfatuuuee--00---111
2009.
Alex Lamb, Vikas Verma, Juho Kannala, and Yoshua Bengio. Interpolated adversarial training: Achieving robust neural networks without sacrificing too much accuracy. In ACM Workshop Artificial Intelligence and Security, 2019.
Lin Li, Yifei Wang, Chawin Sitawarin, and Michael Spratling. OODRobustBench: benchmarking and analyzing adversarial robustness under distribution shift. arXio preprint axXi:23300.12798 2023.
15

---

Published in Transactions on Machine Learning Research (08/2024)
Chang Liu, Yinpeng Dong, Wenzhao Xiang. Xiao Yang, Hang Su, Jun Zhu, Yuefeng Chen, Yuan He, Hui Xue, and Shibao Zheng. A comprehensive study on robustness of image classification models: Benchmarking and rethinking, arXiv preprint arXiv:2302. 14301, 2023.
Faqiang Liu and Rong Zhao. Towards both accurate and robust neural networks without extra data. In International Conference on Artificial Neurol Networks, 2022.
Jeremiah Liu, John Paisley, Marianthi-Anma Kioumourtzoglou, and Brent Coull. Accurate uncertainty estimation and decomposition in ensemble learning. In Annual Conference on Newral Information Processing Systems, 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference Learning Representations, 2018.
Dongbin Na. Pytorch adversarial training on cifar-10.
http:://gittaab..comm/bb/3/
Pytorch- Adversarial -raptaaiiii--aiiinn-----aa CIFAR 2020.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robust via promoting ensemble diversity. In International Conference on Machine Learning, 2019.
Tianyu Pang, Min Lin, Xiao Yang, Jun Zhu, and Shuicheng Yan. Robustness and accuracy could be
reconcilable by (proper) definition. In International Conference on Machine Learning, 2022.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In ACM Asia Conference on Computer and Commu- nications Security, 2017.
Sheng Peng, Weilin Xu, Cory Cornelius, Matthew Hull, Kevin Li, Rahul Duggal, Mansi Phute, Jason Martin, and Duen Horng Chau. Robust principles: Architectural design principles for adversarially robust CNNs. Xrxiv preprinl ar 2rx: 16258, 2023.
Alcksandar Petrov, Francisco Eiras, Amartya Sanyal, Philip HS Torr, and Adel Bibi. Certifying ensembles: A general certification theory with S-Lipschituness. arXio preprint or uiv:23200 13019, 2023.
Samuel Pfrommer, Brendon G Anderson, and Somayeh Sojoudi. Projected randomized smoothing for certified adversarial robustness. Transactions on Machine Learning Research, 2023.
Samuel Pfrommer, Brendon Anderson, Julien Piet, and Somayeh Sojoudi. Asymmetric certified robustness via featuro-convex neural networks. Advances in Neural Information Processing Systems, 2024.
Rahul Rade and Seyed-Mohsen Moosavi-Dezfooli. Helper-based adversarial training: Reducing excessive margin to achieve a better accuracy vs. robustness trade-off. In ICML Workshop on Adversarial Machine Learning, 2021.
Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John C. Duchi, and Percy Liang. Understanding and mitigating the tradeoff between robustness and accuracy. In International Conference Machine Learning, 2020.
Sylvestre-Alvies Rebuffi, Sven Gowal, Dan A Calian, Florian Stimberg, Olivia Wiles, and Timothy Mann. Fixing data augmentation to improve adversarial robustness. arXiv preprint wxxi:21033.1946. 2021.
Vikash Sehwag, Saeed Mahloujifar, Tinashe Handina, Sihui Dai, Chong Xiang, Mung Chiang, and Prateek Mittal. Robust learning meets generative models: Can proxy distributions improve adversarial robustness? In International Conference on Learning Representations, 2022.
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Annual Conference on Neural Information Processing Systems, 2019.
16

---

Published in Transactions on Machine Learning Research (08/2024)
Naman D Singh, Francesco Croce, and Matthias Hein. Revisiting adversarial training for imagenet: Architec- tures, training and generalization across threat models. or preprint XXix:2333001870 2023.
Florian Tramér, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. In Annual Conference on Newral Information Processing Systems, 2020-
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In International Conference on Learning Representations, 2019.
Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Tn Annual Conference Neural Information Processing Systems, 2017.
Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. In International Conference on Machine Learning, 2023.
Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. ConvNeXt V2: Co-desiguing and scaling convnets with masked autoencoders. In IEEE/CVE Conference on Computer Vision and Pottern Recognition, 2023.
Zhuolin Yang, Linyi Li, Xiaojun Xu, Bhavya Kailkhura, Tao Xie, and Bo Li. On the certified robustness for ensemble models and beyond. In International Conference on Learning Representations, 2022.
Yaodong Yu, Stephen Bates, Yi Ma, and Michael Jordan. Robust calibration with multi-domain temperature scaling. In Annual Conference on Newral Information Processing Systems, 2022.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael Jordan. Theorctically principled trade-off between robustness and accuracy. In International Conference Machine Learning, 2019.
17

---

Published in Transactions on Machine Learning Research (08/2024)
Appendix
A Proofs
A.1 Proof to Theorem 4.3
Theorem 4.3 (restated). Suppose that Assumption of (+) and n-))-)) respectively. If B 2 /T then solution to (3) is feasible for (2)
4.2 holds. Let yyM and Thes denote the robust accuracy
Proof. Suppose that M-) is solution from 3
Since the mixed classifier f (-) is by construction
guaranteed to be correct and robust at some if h rob (.) is correct and robust with margin no smaller than
I-a at x, it holds that
P PxxYwD arg max & (X + 8,s (X)) Y] 2 P P(xy)> [M (X + 87M (X)) sobss
2 I-a H. (X)]
mix
mix
Pxxyyw [may (X + 8/
(x))
سي Hu (X)] P( (X)),
min
where (X) denotes the event of (.o being correct at X, i.e., arg max; Hrob, iX + 8 J
(X) Y.
Similarly, HM (X) denotes arg max; N. i + 87 (X)) Y. Under Assumption 4.2 Heor (X) is equivalent.
mni
to Hode (X). Therefore,
P ( arg max fixx, (X + 87 (X)) = Y] = P P(xr mmp rob.
(X + +6
pM (X)) 2 1-o
Heor (X)] P(x.y)
mixx
Throh P Px-xy
[m M bob.t
(X + +.0
(X)) 2 1-a
mix
2 Thet P Prixx
m
(2) 2 1-
2 Thoo B3 3270
TTM mix
which proves the statement.
A.2 Proof to Theorem 4.4
Thcorem 4.4 (restated). Suppose that Assumption 4.1 holds. Furthermore, consider an input random variable and suppose that the margin of M, (X) is independend of whelher gata(X) is correct. Then, minimizing the objective of 3 is equivalent to marimizing the objective of (2)
Proof. By the construction of the mixed classifier, for a clean input x incorrectly classified by (.).()) (i.e., x is in the support of ) the mixed classifier prediction fmix(x) is correct if and only if g (x) is correct and h (z)'s margin is no greater than 1-a
TS(())
Let Geor (X) denote the event of (X) being correct, i.e., arg max; Istd
TS(O) (X) = Furthermore, let Dic denote the data-label distribution formed by clean examples incorrectly predicted by 1.M (.). Then,
P(X,Y)DD arg max FM fit (X) = Y| =P Px
Mya (X) < L-a (x)
olean
My (X) < - Gooo ox)
(x) P PX (Goor (X))
X
for all Iransformations M and mixing weight a (recall that the mixed classifier (M) depends on a).
Suppose that the margin of M (.) is independent the accuracy of gata()), then the above probability further cquals to
(1 P Px-xx
mm (X) ma
1=a )=aa)= P Px-xx stean [Goo((X)
Xx cloam
18

---

Published in Transactions on Machine Learning Research (08/2024)
Since Px Xx dlean [X)] does not depend on M or a, it holds that
arg min MEM, aEp/2,11
P Px
MA (X)
arg max
((x,Y))D.. arg max f (X) Y]
MEM, aE|1/.11
P(XY))D arg max f (X) Y],
arg max MEM, oE[1/.1]
where the last equality holds because under Assumption 4.1 hy (x) being correct guarantees TS(()
Istd (a)'s IT(O)
correctness. Since the mixed classifier fixx (-) must be correct given that A (x)
(x) and Ista (x (x) are both
correct, (-) must be correct at clean examples correctly classified by M (t) Hence, maximizing (-)'ss)ss
clean accuracy on A (.)'s mispredictions equivalent to maximizing J /M (.)xs))ss
(-)'s overall clean accuracy.
B Custom Attack Algorithms
B.1 Minimum-Margin AutoAttack for Margin Estimation
Our margin-based hyper-parameter selection procedure (Algorithm 1 and confidence margin estimation experiments (Figures i and 7 require approximating the minimum-margin perturbations associaled with the robust base classifier in order to analyze the mixed classifier behavior. Approximating these perturbation roquires a strong attack algorithm. While AutoAttack is often regarded as a strong adversary that reliably evaluates model robustness, its original implementation released with [Croce & Hein 2020) does not return all perturbed examples. Specifically, traditional AutoAttack does not record the perturbation around some input if it deems the model to be robust at this input (i.e., model prediction does not change). While this is acceptable for estimating robust accuracy, it forbids the calculation of correctly predicted AutoAttacked examples' confidence margins, which are required by Algorithm D Figure i and Figure 7
To construct a strong attack algorithm compatible with margin estimation, we propose minimum-mmrriin AutoAttack (MMAA). Specifically, we modify the two APGD components of AutoAttack (untargeted APGD- CE and targeted APGD-DLR) to keep track of the margin at each attack step (the margin history is shared across the two components) and always return the perturbation achieving the smallest margin. The FAB and Square components of AutoAttack are much slower than the two APGD components, and for our base classifiers, FAB and Square rarely succeed in altering the model predictions for images that APGD attacks fail to attack. Therefore, we exclude them for the purpose of margin estimation (but include them for the robustness evaluation of MixedNUTS).
B.2 Adaptive Attacks for Nonlinearly Mixed Classifier Robustness Evaluation
When proposing a novel adversarially robust model, reliably measuring its robustness with strong adversaries is always a top priority. Hence, in addition to designing MMAA for the goal of margin estimation, we devise two adaptive attack algorithms to evaluate the robustness of the MixedNUTS and its nonlinearly mixed model defense mechanism. Both algorit hms are strengthened adaptive versions of AutoAttack. As is the original AutoAttack, both algorithms are ensembles of four attack methods, including black-box component. Hence, the reported accuracy numbers in this paper are lower bounds to the attacked accuracy associated with each of the components.
B.2.1 Transfer-Based Adaptive AutoAttack with Auxiliary Mixed Classifier
Following the guidelines for constructing adaptive attacks (Tramèr et al. 2020). our adversary maintains full access to the end-to-end gradient information of the mixed classifier fmix Nonetheless, when temperature scaling with 7 0 is applied to the accurate base classifier (sta(-) as discussed in Section 3.1 Istd TS(T() is no longer differentiable. While this is an advantage in practice since the mixed classifier becomes harder to attack, we need circumvent this obfuscated gradient issue in our evaluations to properly demonstrate white-box robustness. To this end, transfer attack comes to the rescue. We construct an auxiliary differentiable mixed classifier f.ix(()) by mixing (-ad-)'ss unmapped logits with M. (.).))) We allow our attacks to query the gradient.
19

---

Published in Transactions on Machine Learning Research (08/2024)
of [mix(+) to guide the gradient-based allak on f-ii((). Since (atd()) and Istd
(T) always produce the same prodictions, the transferability between fuix(.) and fmix()) should be high.
On the other hand, while 1oob (066))0))) (:)'s nonlinear logit transformation (4) is differentiable, it may also hinder gradient flow in certain cases, especially when the logits fall into the relatively flat near-zero portion of the clamping function Clamp(-). Hence, we also provide the raw logits of h+ob to our evaluation adversary for better gradient flow. To keep the adversary aware of the transformation (s, p, c)),), we still include it in the gradient (i.e., Ms p, p-) is only partially bypassed). The overall construction of the auxiliary differentiable mixed classifier f-ix(-) is then
(8) where Oa is the mixing weight and ra adjusts the level of contribution of Ms, p. (.) to the gradient. Our experiments fix ra to 0.9 and calculate ad using Algorithm [ with no clamping function, 8 and p fixed to 1, and è fixed to O. The gradient-based components (APGD and FAB) of our adaptive AutoAttack use VL.(mix(z)) as a surrogate for (I (x))) where L is the adversarial loss function. The gradient-free Square attack component remains unchanged. Please refer to our source code for details on implementation.
[mix(()) = log ((1 - aa) a o (.) + oaaaa + a hyab(() + aa(1 - ra) a Nob x' ()))
With the transfer- based gradient query in place, our adaptive AutoAttack does not suffer from gradient obfuscation, a phenomenon that leads to overestimated robustness. Specifically, we observe that the black- box Square component of our adaptive AutoAttack does not change the prediction of any images that white-box components fail to attack, confirming the effectiveness of querying the transfer-based auxiliary differentiable mixed classifier for the gradient. If we set ra to 0 (i.e., do not bypass M (s, p. c-) for gradient), the AutoAttacked accuracy of the CIFAR-100 model reported in Figure 5l becomes 42.97% instead of 41.80% and the black- box Square attack finds 12 vulnerable images. This comparison confirms that the proposed modifications on AutoAttack serengthen its effectiveness against MixedNUTS and eliminate the gradient flow issue, making it a reliable robustness evaluator.
B.2.2 Direct Gradient Bypass
An alternative method for circumventing the non-differentiability challenge introduced by our logil transfor- mations is to allow the gradient to bypass the corresponding non-differentiable operations. To achieve so, we again leverage the auxiliary differentiable mixed classifier defined in [8) and construct the overall output as .²" (0)(() -mix(x))
(m) + StopGrad MMs"
where StopGrad denotes the straight-through operation that passes the forward activation but stops the gradient (Bengio et al. 2013), for which PyTTorch realization is Tensor. detach The resulting mixed classifier retains the output values of the MixedNUTS classifier PM((' =p' (<)())) while using the gradient computation graph of the differentiable auxiliary classifier [mix(<)) In the literature, a similar technique is often used to train neural networks with non-differentiable components, such as VAES (Van Den Oord ct al. 2017).
This direct gradient bypass method is closely related to the transfer-based adaptive attack described in Appendix B.2.1 but has the following crucial differences:
Compatibility with existing attack codebases. The transfer-based attack relies on the outputs from both p x x²)(()) and fmi Since most existing attack codebases, such as AutoAttack, are implemented assuming that the neural network produces a single output, they need to be modified to accept two predictions. In contrast, direct gradient bypass does not introduce or require multiple network outputs, and is therefore compatible with existing attack frameworks without modifications. Hence, our submission to Bench uses the direct gradient bypass method.
Calculation of attack loss functions. From a mathematical perspective, the transfer-based atlack uses the auxiliary differentiable mixture to evaluate the attack objective function. In contrast, the direct gradient bypass method uses the original MixodNUTS's output for attack objective calculation, and then uses the gradient computation graph of fmix()) to perform back-propagation. Hence, the resulting gradient is slightly different between the two methods.
20

---

Published in Transactions on Machine Learning Research (08/2024)
Table 2: Details of the base classifies used in our main experiments.
Dataset
Robust Base Classifier Isaa(-)
Accurate Base Classifier h-ab(.)
CIFAR-10
ResNet-152
Kolesnikov et 2020
RaWideResNet-7O- (Peng et
2023
CTFAR-100
ResNet-152
Kolesnikov al.
2020
6 [Wang et al.
2023
ImageNet
ConvNeXt V2-L Woo et al.
2023
Swin-L LLu et al. 2023
Table 3: The optimal 8, p. e, a values returned by Algorithm used in our main experiments, presented along with the minimum and maximum candidate values in Algorithm searching grid.
p²
xx'
Pmin
Praxx
CIFAR-10
5.00
-1.10
4.00
.999
0.05
-1.1
0
1
4
C1EAR-100
.612
2.14
3.57
.986
0.05
-25
0.4
1
ImageNet
.0235
-.286
2.71
.997
0.01
0.2
-2
0
2
Table 4: The proposed nonlinear logit transformation M(s", ,p",'"")"" (.))()) has minimal effect on base classifier accuracy.
Clean (full dataset)
AutoAttack (1000 images)
Dataset
Avob
1oob (-)
hoob MM""""p""ee (-))(())))))))))))))))))))))))))
h-ob(-)
(-ob(-)
MM MM ()))
CIFAR-10 CIFAR-101 100
93.27%
93.27%
93.25%
71.4%
71.4%
71.4%
75.22%
75.22%
75.22%
43.0%
42.9%
43.3%
ImageNet
78.75%
78.75%
78.75%
57.5%
57.5%
57.5%
Our experiments show that when using direct gradient bypass, the original AutoAttack algorithm returns 70.08%, 41.91%, and 58. 58.62% for CIFAR-10, CLFAR-100, and ImageNet respectively with the MixedNUTS model used in Figure Compared with the transfer-based adaptive AutoAttack, which achieves 69.71%, 41.80%, and 58.50%, AutoAttack with direct gradient bypass consistently achieves lower success rate, but the difference is very small. Hence, use the transfer-based AutoAttack for Figure 5 but note that both methods can evaluate MixedNUTS reliably.
C MiXedNUTS Model Details
C.1 Base Classifier and Mixing Details
2 presents the sources and architectures of the base classifiers selected for main experiments (Figure Figure 2 Figure G and Table []. The robust base classifiers are the state-of-the-art models listed on Rench as of submission, and the accurate base classifiers are popular high-performanee models pre-trained on large datasets. Note that since MixedNUTS only queries the predicted classes from gata()) and is agnostic of its other details, (-ta(-) may be any classifier, including large-scale vision- laniunl models that currently see rapid development.
Table
Table presents the optimal s", et, and at values used in MixedNUTS's nonlinear logit transformation returned by Algorithm [ When optimizing p, and c, Algorithm I performs a grid search, selecting from provided set of candidate values. In our experiments, we generate uniform lincar intervals as the candidate values for the power coefficient p and the bias coefficient and use a log-scale interval for the scale coefficient. 8. Each interval has eight numbers, with the minimum and maximum values for the intervals listed in Table 3
Table shows that MixodNUSS's nonlinear logit transformation M(s pp, (-) has negligible effects on basc classifier accuracy, confirming that the improved accuracy-robustnes balance rooted in the improved base classifier confidence properties.
21

---

Published in Transactions on Machine Learning Research (08/2024)
Table 5: MixedNUTS's accuracy and inference efficiency compared with state-of-the-ar classifiers.
Model
Architecture
Parameters
GFL.Pss
Clean (1)
AutoAttack (1)
CIFAR-10
This work
Mixed (see Table
499.5M
151.02
95.19%
69.71%
Peng et al. 2023
Ra NeWieRReeNe-700116
267.2M
121.02
93.27%
71.07%
Bai al.
2024a
Mixed with Mixing Net
566.9M
117.31
95.23%
68.06%
Rebulli el al. 2021
WideResNNet-7--LE
266.8M
77.55
92.23%
66.58%
Kolesnikov et al. 2020
ResNet-152
232.3M
30.00
98.50%
0.00%
CIFAR-100
This work
Mixed (see Table
499.5M
107.56
83.08%
41.80%
Wang et al. 2023
WideResNet- 70-16
266.9M
77.56
75.22%
42.67%
Bai al.
2021 la
Mixed with Mixing Net
567.4M
117.31
85.21%
38.72%
Gowal et al. 2020
WideResNeet-O-LL
266.9M
77.55
69.15%
36.88%
Kolesmikov et
2020
ResNet-152
232.6M
30.00
91.38%
0.00%
ImageNet
This work
Mixed (see Table
394.5M
136.91
81.48%
58.50%
Liu al.
2023
Swin-L
198.0M
68.12
78.92%
59.56%
Singh et a. 2023
ConvNeXt-L ConvStem
198.
71.16
77 77.00%
57.70%
Peng al. 2023
ReWiidRessNet-0112
104.1M
51.14
73.44%
18.94%
Woo et al.
2023
ConvNeXt V2-I
196.5M
68.79
86.18%
0.00%
C.2
Model Inference Efficiency
In this section, we compare the performance and inference efficiency MixodNUTS with existing methods. As training-free ensemble method, MixedNUTS naturally trades inference efficiency for training efficiency. Nonetheless, since MixedNUTS only requires two base models and does not add new neural network components, it is among the most efficient ensemble methods. Specifically, the computational cost of MixedNUTS the sum of the computation of its two base classifiers, as the mixing operation itself is trivial from a conaatational standpoint.
In Table 5 the efficioncy of MixedNUTS, evaluated in terms of parameter count and floating- point operations (FLOPs), is compared with some other state-ot srate-of-the-art methods. Compared with the fellow mixed classifier method adaptive smoothing (Bai et al. 2024a), MixedNUTS more efficient when the base classifiers the same, is the case for CIFAR-100. This is because adaptive smoothing introduces additional mixing network, whereas MixedNUTS only introduces four additional parameters. On CIFAR-10, MixedNUTS uses a denser robust base classifier than adaptive sing, with similar number parameters but higher GFLOPs (121.02 vs 77.56). MixedNUTS's FLOPs count is thus also higher than
(ai et al. 2024a).
C.3 Base Classifier Confidence Margin Distribution
Table 6 displays the histograms of the confidence margins of the base classifiers used in the CIFAR-100 experiment Figure 5 We can observe the following conclusions, which support the design MixedNUTS:
(-na()) is more confident. when making mistakes under attack than when correctly predicting clean images. h-ob(-) more confident correct predictions than incorrect ones, required by MixedNUTS.. Note that when subject to strong AutoAttack, correct predictions still more confident than clean unperturbed incorrect predictions.
Layer normalization increases (-)ss))'s correct prediction margins while maintaining the incorrect margins.
MixodNUTS's nonlinear logit transformation significantly increases the correct prediction's confidence margins while keeping most of the incorrect margins small.
22

---

Published in Transactions on Machine Learning Research (08/2024)
N Table 6: Prediction confidence margin of Aroh()), LLN (+), and (s xp (-) used in the CIFAR-100 experiments
in Figure 6 The nonlinear logit transformation
(4) amplifies the margin advantage of correct predictions over incorrect ones. As in Figure G 10000 clean examples and 1000 AutoAttack examples are used.
Standard base model gatd
Robust base model hoab ())
With LN (-)
With transform hob
ot (e')()
5930
Corract Incorrect
1206
Correct
1000
Incorrect
1030
Incorrect
Incorrect
4030
1505
Clean
600
6:00
1030
1010
2.0
02
0.4
AutoAttack (APCD)
330
Correct Incorrect
350
Cornect Incorroct
Incorrect
Incorract
250
500
200
150
200
0.0
0.
0.4
1.5
2.5
3.5
0.0
0.2
7.4
3.6
0.2
3.5
3.0
2.2
0.4
0.8
3.5
1.0
0.7
0.2
0.6
0.8
1.0
Slice 1.000
Slice at 1.429
Slice 1.857
Slice 2.286
40
40
90.48
2.4
40.07
0.38
0.38
0.33
0.33
88.69
0.37
6.7
0.7
017
40.34
0.50
6.58
56.6
-3.8
-13
-1.
-07
-24
0.5
-18
-14
-13
-10
-07
-24
-25
-22
-18
-14
-13
-10
-07
-04
-25
-22
-1.5
-1.
-10
-07
-04
Slice at 2.714
Slice at p 3.143
Slice at 3.571
Slice at 4.000
75.7
0.33
033
0:33
0.33
65.0 03
25
22
29
1.
1.5
1.
07
-4
1.0
15
10
0.
04
22
1.
1.
1.5
10
07
144
25
22
1.3
1.5
1.5
100
0.7
0.4
Figure 9: Sensilivity analysis of the nonlinear logit transformation. Lower objective (darker color) is better.
D Ablation Studies and Additional Discussions
D.1 Sensitivity to 8, P. Values
In this section, we visualize the optimization landscape of the hyper-parameler optimization problem 7 in terms of the sensitivity with respect to 8, P, and c. To achieve so, we store the objective of [7) corresponding cach combination of s, p, e values our grid scarch as throo-dimensional tensor (recall that the value of
a can be determined via the constraint). We then visualize the tensor by displaying each slice of it as color
map. We use CIFAR-100 model as example, and present the result in Figure
23

---

Published in Transactions on Machine Learning Research (08/2024)
Table 8; The accuracy on images used for calculating s* p", and c* (marked as /in the "Seen" column) similar to that on images unseen by Algorithm (marked as X), confirming the absence of overfitting.
Table 7: Ablation study on clamping functions.
Clamp(-)
p
a²'
Obj (4)
CIFAR-10
Linear
.050
9.14
.963
.985
.671
Dataset
Seen
Clean
AutoAttack
ReLU
10.4
7500
4.00
.934
.985
.671
95.20%
69.20%
GELU
2.59
- -750
4.00
.997
.985
.671
CIFAR-10
x
95.18%
69.77%
CIFAR-100 100
/
82.80%
41.60%
CIFAR-100
Linear ReLU
00000s
10.5
.880
.985
.504
83.11% x
41.82%
.612
2.29
4.00
.972
.985
.500
82.60%
60.80%
GELU .612
-2.14
3.57
.986
.985
.500
ImageNet
81.20%
57.93%
The optimal hyper- and AutoAttack accuracy when AutoAttack accuracy on 1000- parameters are similar across similar 8, p, and are optimized us- example CIFAR-100 subsot with var- models, and hence can transfer be- ing different numbers of images. ious temperature scales for the stan- tween them. CIFAR-10 models are Evaluated with the CIFAR-100 dard base model (sta(-). The robust used, with (Peng et al. 2023) being base models from Figure on base classifier is Noob (²,))(()) with the model in the main experiments in 1000-example subset.
Table 9: MixedNUTS's clean Table 10: MixedNUTS's clean and Table 11:
the 8, P, e values reported in Ta- Figure 5
ble 3
# Images for Optimization
Auto
Robust
Clean
s*
c²
p
Attack
Accurate
Auto Attack
Base Model
Clean
Base Model
1000 (Default)
82.8%
41.6%
Peng et al.
2023
5.0
-1.1
4.0
300
83.0%
41.5%
9asd
TS(0) (-) (Default)
82.8%
41.6%
Pang et
2022
5.0
-1.1
4.0
100
85.1%
39.5%
9 2ad
(SS(0.5) (-)
82.8%
41.4%
Wang et al.
2023
5.0
1.1
2.71
9au ()) (.)
TS()
82.8%
41.3%
As shown in Figure 9 while the optimization landscape is non-convex, it is relatively smooth and bemign, with multiple combinations achioving similar, relatively low objective values. When the exponent parameter p is small, the other two parameters, 8 and 6, have be within a smaller range for optimal performance. When p larger, a wide range of values for 8 and can work. Nonetheless, an excessively large p may potentially cause numerical instabilities and should be avoided if possible. For the same consideration, we do not recommend using the exponentiation function in the nonlinear logit transformation.
For further illustration, construct a mixed CIFAR-100 classifier with simple GELU the nonlinear logit transformation (still using Istd (.) is 77.9%/40.4%%
TS(0) (-) as the standard base classifier) The resulting clean dreusrobuust. accuracy 1000-example data subset. While this result is slightly better than the 77.6% /3..9% accuracy of the bascline mixed classifier without nonlincarity, it is noticcably worse than MixedNUTS's 82.8%/41.6%. We can thus conclude that selecting good combination of s, and e is crucial for achieving optimal performance.
D.2 Selecting the Clamping Function
This section performs ablation study on the clamping function in the nonlinear logit transformation defined in [4) Specifically, we compare using GELU ReLU Clamp(-) to bypassing Clamp(-) (i.e. use lincar function). Here, we sclect CIFAR-10 ResNet-18 model (aa 2020) and CIFAR-100 WideRosNet- 70-16 model (Wang et al.] (022)) as two examples of h-o and compare the imal objective ret urnud by Algorithm I using each of the clamping function settings. As shown in Table 7 while the optimal objective is similar for all three options, the returned hyper-pane paro-ppaaacterss s", p², c², and a* is the most "modest" for GELU, which translates to the best numerical stability. In comparison, using a linear clamping function requires applying a power of 9.14 to the logits, whereas using the ReL.U clamping function requires scaling
24

---

Published in Transactions on Machine Learning Research (08/2024)
the logits up by factor of 10.4 for CIFAR-10, potentially resulting in significant numerical instabilities. Therefore, we select GELU as the default clamping function and use it for all other experiments.
D.3 Effect of Optimization Dataset Size and Absence of Overfitting
Since the MMAA step has the dominant computational time, reducing the number of images used in Algorithm [ can greatly accelerate it. Analyzing the effect of this data size also helps understand whether optimizing 8, p, and c on validation images introduces overfitting. Table Shows that on CIFAR-100, reducing the number images used in the optimization from 1000 to 300 (3 images per class) has minimal effect on the resulting mixed classifier performance. Further reducing the optimized subset size to 100 still allows for an accuracy-robustress balance, but shifts the balance towards clean accuracy.
To further demonstrate the absence of overfitting, Table reports that under the default setting of optimizing 8, P. c on 1000 images, the accuracy on these 1000 images is similar to that on the rest of the validation images unseen during optimization. The CIFAR-10 and -100 models, in fact, perform slightly better on unseen images. The ImageNet model's accuracy on unseen images is marginally lower than seen ones, likely due the scarcity of validation images per class (only per class in total since ImageNet has 1000 classos) and the resulting performance variance across the validation set.
D.4 Temperature Scaling for (-td(-)
This section verifies that scaling up the logits of Ista(-) improves the accuracy-robustnet tradc-off of the mixed classifier. We select the pair of CIFAR-100 base classifiers used in Figure 5 By jointly adjusting the temperature of (-ta()) and the mixing weight a, we keep the clean accuracy of the mixed model to approximately 84 percent and compare the APGD accuracy. In Table 10 we consider two temperature constants: 0.5 and 0. Note that as defined in Section 2. when the temperature is zero, the resulting prediction probabilities a o Gaa(-) is the one-hot vector associated with the predicted class. As demonstrated by the CIFAR-100 example in Table 10, when we fix the clean accuracy to 82.8%, using T = 0.5 and T = 0 produces higher AutoAttacked accuracy than T (no scaling), with T producing the best accuracy-robustness balance.
D.5 Algorithm 1 for Black-Box hrob(.) without Gradient Access
When optimizing the hyper-parameters 8, P, and c, Step 2 of Algorithm | requires running MMAA on the robust base classifier. While MMAA does not explicitly require access to base model parameters, its gradient-based components query the robust base classifier gradient (the standard base classifier gad(-) can be a black box).
However, even the gradient of (-) is also unavailable, then 8, P. c, and a can be selected with one of the following options:
Black-box minimum-margin attack. Existing gradient-free black-box attacks, such Square (Andriushchenko et al. 2020) and BPDA (Athalye et al.] 2018a)), can be modified into minimum-margin attack algorithms. As are gradient-based based methods, these gradient-free algorithms are iterative, and the only required modification is to record the margin at each iteration to keep track of the minimum margin.
Transfer from another model. Since the robust base classifiers share the property of being more
confident in correct examples than in incorrect ones (as shown in Figure 5), an optimal set of s, p, c values for one model likely also suits another model. So, one may opt to rum MMAA on a robust classifier whose gradient is available, and transfer the 8, p, e values back to the black-box model.
Educated guess. Since each component of our parameterization of the nonlinear logit transformation is intuitively motivated, a generic selection of s, p, c values should also perform better than mixing lincarly. In fact, when we devised this project, we used hand-sclected p, values for prototyping and idea verification, and later designed Algorithm for a more principled selection.
To empirically verify the feasibility of transferring aaper-parameters across robust base classifiers, we show that the optimal hyper-parameters are similar across similar models. Consider the CIFAR-10 robust base
25

---

Published in Transactions on Machine Learning Research (08/2024)
classifier used in our main results, which is from (Peng et al. 2023 Suppose that this model is black box, and the gradient-based components of MMAA cannot be performed. Then, we can seek some similar robust models whose gradients are visible. We use two models, one from (Pang et al. 2022), , and the other from Waann et al. 2023 as examples. As shown in Table 11 the optimal 8, p. C values calculated via Algorithm are highly similar for these three models. Hence, if we have access to the gradients of one of (Peng el al. 2023; Pang et al. 2022 Wang et al.) 2023), then we can use Algorithm 1 to select the hyper-parameter combinations for all three models.
Since other parts of MixedNUTS do not require access to base model weights gradients, MixedNUTS can be applicd to model zoo even when all base classifiers are black boxes.
D.6 Selecting the Base Classifiers
This section provides guidelines on how to select the accurate and robust base classifiers for the best mixed classifier performance. For the accurate classifier, since MixedNUTS only considers its predicted class and does not depend on its confidence (recall that MixedNUTS uses I (.)) the classifier with the best-known clean accuracy should be selected. Meanwhile, for the robust base classifier, since MixedNUTS relies on its margin properties, one should select model that has high robust accuracy well benign margin characteristics (i.e., is significantly more confident in correct predictions than incorrect ones). As shown in Figure 7 most bigh-performance robust models share this benign property, and the correlation between robust accuracy and margins is insignificant. Hence, state-of-tho-art robust modcls are usually safe to use.
That being said, consider the hypothetical scenario that between a pair of robust base classifiers, one has higher robust accuracy and the other has more benign margin properties. Here, one should compare the percentages of data for which the two models are robust with certain non-zero margin. The model with higher "robust accuracy with margin" should be used.
D.7 Behavior of Logit Normalization
The LN operation on the model logits makes the margin agnostic to the overall scale of the logits. Consider two example logit vectors in R3, namely (0.9, 1, 1.1) and (-2, 1, 1.1). The first vector corresponds to the case where the classifier prefers the third class but is relatively unconfident. The second vector reflects the scenario where the classifier is generally more confident, but the second and third classes compete with each other. The LN operation will scale up the first vector and scale down the second. It is likely that the competing scenario is more common when the prediction is incorrect, and therefore the LN operation, which comparatively decreases the margin under the competing scenario, makes incorrect examples less confident compared with correct ones. As a result, the LN operation itself can slightly enlarge the margin difference between incorrect and correct examples.
For ImageNet, instead of performing LN on the logits based on the mean and variance of all 1000 classes, we normalize using the statistics of the top 250 classes. The intuition of doing so is that the predicted probabilities of bottom classes are extremely small and likely have negligible influence on model prediction and robustness. However, they considerably influence the mean and variance statistics of logits. Excluding these least-related classes makes the LN operation less noisy.
D.8
Further Discussions on Assumptions
D.8.1
When Assumption 4.1 is Slightly Violated
If Assumption
4.1 is slightly violated, then there is a slight mismatch between the objective functions of 3 and (2) due to discarding the case of 92dd
TS(O) (.) being incorrect while hoop M(sss))()) being correct clcan data. As a result, the reformulations in this section become slightly suboptimal. However, note that the constraint in [2) which enforces the level of robustness of the mixed classifier, is not compromised. Furthermore, as
mentioned above, the amount of clean examples correctly classified by hiob ).M(s.ee)))) but not by Iatd TT(() (-) is
usually exceedingly rare, and hence the degree of suboptimality is extremely small.
26

---

Published in Transactions on Machine Learning Research (08/2024)
Also note that with slight violation of Assumption 4.1 while our algorithm may become slightly suboptimal, the mixed classifier outperfomms our expectation, because it can now correctly classify additional clean the only theoretical result dependent on Assumption 4.1
examples than suggested by Theorem 4.4
D.8.2 When Assumption 4.2 is Slightly Violated
Assumption 4.2 assumess that the nonlinear logit transformation applied to h-ra(-) does not affect its predicted class and hence inherits (-os accuracy, When Assumption 4.2 is violated, consider the following two cases: 1) the logit transformation M Mss.p.c)( corrects mispredictions; 2) M(s.p.c)()) contaminates correct predictions.
Consider the first scenario, i.e, h moo only theoretical result dependent on Assumption 4.2 still holds, and the mixed classifier can correctly classify even more clean examples than Theorem 4.3 suggests.
(.) is correct whereas h-ab(-) is not. In this case, Theorem 4.3 (the
Conversely, consider the second case, where Mrab (.) is incorrect whereas (ro (.) is correct. In this case, Theorem 4.3 may not hold. However, this is the best one can expect. In this worst-case scenario, although the nonlinear logit transformation improves h-os(-)'s confidence property, it also harms hrob standalone accuracy, which in turn negalively affects the MixedNUTS model. Fortunately, this worst case is easily avoidable in practice by checking (166))))))))3 (-)'s standalone clean accuracy. If (ooo (M)))))))) clean accuracy deteriorates, the search space for 8, p, and e can be adjusted accordingly before re-running Algorithm 1.
D.8.3 Relaxing the Independence Assumption in Theorem 4.4
Theorem 4.4 assumees that the margin of h hyb (X) and the correctness of (xta(X) are independent. Suppose that (X) 2 1-a] may not be [may, "oob
such an assumption does not hold for pair of base classifiers. Then, Px clenn Px-xx_
equal to P Px~xx_. alean [m (X) 2 1-a (Fco(())] In this case, we need to minímize the latter quantity in order wul
to effectively optimize change the objective value assignment step in Line 10 of Algorithm to oták + IP
Hence, need to modify the objective functions of (3) and
(6) accordingly, and Mto (X) 2
X
sikk 01-8 (x)r With such modification, the optimization of P. is no longer decoupled from gaa(-), but
the resulting algorithm is still efficiently solvable and Theorem 4.4 still holds.
D.9 Approximation Quality of 77)
Algorithm I solves a surrogate of (6) for efficiency. One of the approximations of (7) to use the minimum -animum-margin perturbation against I (.) instead of that associated with (.). While h hob (-) and AIN (-) are expected to have similar standalone accuracy and robustness, their confidence properties are different, and therefore the Sinimum--margi perturbation associated with M (-) can be different from that associated with A.N (-), inducing distribution mismatch. To analyze the influence of this mismatch on the effectiveness of Algorithm I. we record the values of s* an,and c², compute the minimum waatenaaaannnniiie AutoAttacked examples of Mi(" hob x=" ee)) (.) and re-run Algorithm 1 with the new examples. the objective value calculated via the new examples and 8" p²; e* is close to the optimal objective returned from the original Algorithm | then the mismatch is small and benign and Algorithm is capable of indirectly optimizing (6)
We use the CIFAR-100 model from Figure 5 as example to perform this analysis. The original optimal objective returned by Algorithm | is 50.0%. The re-computed objective based on hyob
M (')'ss minimum-
margin perturbations, where s* .612, pr = 3.57, c* 22144 is 66.7%. While there is gap between the two objective values and therefore the approximation-indueed distribution mismatch exists, Algorithm I can still effectively decrease the objective value of (6)
27


Reviews:
[{"review_id": "5AurY3ihNF", "reviewer": "Reviewer_oyU1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The authors do a very good job of explaining the motivation of why this problem is an important one. The paper has a good logical flow, with the authors introducing the problem and giving a high level flowchart of the proposed solution in Figure 1. The authors have also done a comprehensive job is mentioning related papers and prior art. In section 2, the authors do great job of motivating the intuition of using a heterogeneous ensemble classifier is clear, with a clear explanation on the benefits of mixing the outputs of the 2 types of classifiers i.e. the base and the robust. Section 4 is also well written, with the assumptions for the robust classifier clearly stated. The assumptions for the bounds of $\\alpha$ are based on a cited paper, when seems reasonable. The definition of the non-linear transform $M$ is also well written, with some issues with the notation here and there. The empirical results and experiments all look reasonable to me. \n\n#### Questions/Clarifications\n- In section 3, the authors introduce an explanation for the modifications to the base classifier using temperature scaling. It is a little unclear on why the temperature for the scaled base classifier ($g_i^{TS,T}$)is set to 0. The motivation given for this is  \"... to mitigate $g(\\cdot)$'s detrimental property of being more confident when mispredicting under attack than when correctly predicting clean data ... \", but it's still unclear why one-hot encoding the outputs would help in this scenario, apart from making the math in section 4 easier. \n- In section 4.3, the algorithm for finding $s, p, c$ and $\\alpha$ and with the MMAA algorithm maintaining the minimum margin, it seems like the algorithm would be reasonably fast to run as a grid search. It was not clear to me from the section if the search for the hyper parameters would be reasonably efficient based on the size of the search space. \n- Also in section 4.3, the search algorithm depends on generating clean examples that are incorrectly classified $\\tilde{\\chi}_{ic}$ and attacked examples correctly classified $\\tilde{\\chi}_{ca}$ , both for $h^{LN}(\\cdot)$. Would this lead to a cold start problem for the ensemble classifier?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors of the paper propose a novel method to improve the adversarial robustness of a neural classifier, while not losing out on it's prediction accuracy. Their  model is an heterogeneous ensemble of 2 classifiers\n- a base classifier ($g(\\cdot)$) trained for the attack-free scenario \n- robust classifier ($h(\\cdot)$) trained to be robust against adversarial attacks \n\nAnd the ensemble is given by: \n\n$$f_i(x) := log ((1 −\\alpha) \\cdot \\sigma \\circ g(x)_i + \\alpha \\cdot \\sigma \\circ h(x)_i$$\n\nwhere $\\alpha \\in [1/2,1]$ adjusts the mixing weight (given my a previous paper) and $i \\in [C]$ for $C$ classes. The base classifier's ($g(\\cdot)$) is scaled using temperature scaling\n\n$$g_i^{TS,T} := \\frac{g_i(x}{T}$$ \n\nand $T$ is set to 0 i.e. the base classifier becomes $g_i^{TS,0}$. To simplify the optimization, the authors make a few assumptions regarding the nature of $h^M(\\cdot)$ , where $M$ is the non-linear transform applied on the logits of $h(\\cdot)$. The transformed $h(\\cdot)$ is defined to be $h^{M}\\substack{s\\\\p\\\\c}(x)$ where $s \\in (0, +\\infty)$ is a scaling constant, $p \\in (0, +\\infty)$ is the exponent constant and $c \\in \\mathbb{R}$ is a bias constant with the choice of $Clamp$ function being the GELU.  The authors then define an algorithm to optimize for $s, p, c$ and $\\alpha$ while also providing a method to calculate a minimum margin for the optimization using the AutoAttack algorithm. The authors also provide extensive empirical studies to back up their claim of an efficient and robust Mixed model.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- (Minor) In Section 3: Is is stated that the robust classifier $h(\\cdot)$ enjoys the property of having more confidence in correct predictions than in mistakes ... this sentence is a little unclear with the definitions and could use an example or definition in context. \n- (Minor) In various sections, the base and robust classifier are defined by $g(\\cdot)$ and $h(\\cdot)$ respectively. Changing the variable names to something more semantic might help with the readability and referencing of the models in later sections. I would always get confused as to which one was the robust and which one was the base.\n- (Minor) In section 4 , when defining the distributions $X_{ic}$ for the distribution formed by clean examples incorrectly classified by $h^M(\\cdot)$ and  $X_{ca}$ for the distribution formed by attacked examples correctly classified by $h^M(\\cdot)$, it would be a bit more clear to use more descriptive subscripts for the, just to make it easier to refer to later.", "broader_impact_concerns": "There are no broader ethical impacts from this paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "qPAEVbJ6Qz", "reviewer": "Reviewer_C4np", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n\n1. The paper presents a \"training-free\" method, which is cost-effective once two basic models are available. Since it does not require re-training the entire model, this approach is relatively inexpensive. Moreover, the ensemble method involves a careful combination of the logits/outputs from two trained models, allowing flexibility as the basic models do not need to have the same structure. This makes the method lightweight and empirically easy to use.\n\n2. The paper demonstrates the performance of the proposed method. Experimental results show that MixedNUTS achieves higher clean data accuracy compared to other robust methods while maintaining a reasonable level of robustness.\n\n3. The paper provides extensive discussion and clarity in its writing.\n\nWeaknesses:\n\n1.  A main drawback is the intractability of optimizing parameters for the nonlinear transformation. The paper resorts to an empirical method, grid search, which lack strong theoretical guarantees. For instance, it is challenging to theoretically justify that $h^{LN}$ serves as an effective surrogate for $h^{M}_{s,p,c}$.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces MixedNUTS,  a robust model by ensembling a standard model and a robust model. The motivation stems from the observation that a robust model exhibits greater confidence in correct predictions compared to incorrect ones. Specifically, the ensemble method involves a convex combination of the softmax outputs from the standard model, combined with temperature scaling, and the robust model, combined with a nonlinear transformation.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Questions I Seek to Have Answered:\n\n1. Why is the nonlinear transformation applied only to the robust base model and not to the non-robust model as well?\n\n2. In Section 4.4, second paragraph, why do we compare $\\sigma(h/T)$ with $\\sigma(h^M/T)$, considering that the temperature scale $h^M/T$ is not directly used in $f^M$?\n\n3. In Section 5.1, how is \"Mixed\" chosen? What is $\\alpha$ in this combination?\n\n4. Regarding Figure 8, how are the clean accuracy and robust accuracy controlled to produce this plot? Is it by adjusting $\\beta$ for MixedNUTS?\n\nMinor Suggestions:\n\n1. I recommend changing the color scheme for the white and grey bars in the bar plots to improve clarity and visual distinction.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZdVEjI7OCA", "reviewer": "Reviewer_CcFH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: Overall, this paper has lots of strengths. The most noticeable ones are listed below:\n1) The paper provides good intuitive explanations of its various high-level ideas.\n2) Under reasonable and almost realistic assumptions, the paper provides rigorous proofs to show its framework is theoretically solid.\n3) The paper provides good numerical results achieving best-known accuracy-robustness tradeoff, with significant improvement on this metric over any existing classifier.\n4) The method is training-free, and can be applied on one accurate model and one robust model with quite general properties. The handful of parameters can be pre-configured on a small set of data by another algorithm (Algorithm 1), thus don't need much manual tuning.\n\nWeaknesses:\n1) Algorithmic novelty: Compared to the existing Mixed method [Bai et al. (2024)], most algorithm change in the proposed MixedNUTS method is through adding a nonlinear logit transformation to the robust classifier. Thus, the algorithm didn't change much, even though the result improved a lot.\n2) Most numerical experiments are only tested against two or less attack methods, according to the main paper and appendix B (if I misunderstood, please let me know). The results could be more convincing if more than 3 attacks are included in most experiments. \n3) (minor) Theoretically, some assumptions not exactly satisfied in practice. e.g. GELU is not monotonic [assuption 4.2]; clean classifier may not be strictly better than robust classifier on clean data [assumption 4.1]. It would be interesting to touch on the theoretical guarantees when the assumptions are slightly violated, such as when the assumed monotonic activation is replaced by GELU.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a novel general training-free framework called MixedNUTS, for building a classifier that is both accurate and robust, by combining two general existing models: one with very good accuracy, another with good robustness and reasonable accuracy. The paper proposes nonlinear base model logit transformations on the robust base classifier to exploit its benign confidence property. The paper achieves good numerical results on common image datasets under strong adaptive attacks: MixedNUTS achieved vastly improved accuracy and near-SOTA robustness – it boosts CIFAR-100 clean accuracy by 7.86 points, sacrificing merely 0.87 points in robust accuracy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "This work would be better and more convincing if (one or two) more attacks are tested in numerical experiments.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposed a simple yet effective method to improve adversarial robustness, by combining the logits of robust and non-robust classifiers in the inference stage. The initial submission had several weaknesses, as pointed out by the reviewers. The authors did a successful rebuttal and revision, which led to a unanimous consensus on accepting this submission from all reviewers.
Decision: Accept (poster)