With False Friends Like These, Who Can Have Self-Knowledge?
Authors: Lue Tao
Songcan Chen
Venue:  (2021)
Keywords: Robustness
Adversarial Risk
Neural Networks
Machine Learning Security

Abstract: Adversarial examples arise from excessive sensitivity of a model. Commonly studied adversarial examples are malicious inputs, crafted by an adversary from correctly classified examples, to induce misclassification. This paper studies an intriguing, yet far overlooked consequence of the excessive sensitivity, that is, a misclassified example can be easily perturbed to help the model to produce correct output. Such perturbed examples look harmless, but actually can be maliciously utilized by a false friend to make the model self-satisfied. Thus we name them hypocritical examples. With false friends like these, a poorly performed model could behave like a state-of-the-art one. Once a deployer trusts the hypocritical performance and uses the "well-performed" model in real-world applications, potential security concerns appear even in benign environments. In this paper, we formalize the hypocritical risk for the first time and propose a defense method specialized for hypocritical examples by minimizing the tradeoff between natural risk and an upper bound of hypocritical risk. Moreover, our theoretical analysis reveals connections between adversarial risk and hypocritical risk. Extensive experiments verify the theoretical results and the effectiveness of our proposed methods.

Under review as a conference paper at ICLR 2021
WITH FALSE FRIENDS LIKE THESE, WHO CAN HAVE SELF-KNOWLEDGE?
Anonymous authors Paper under double-blind review
ABSTRACT
Adversarial examples arise from excessive sensitivity of a model. Commonly studied adversarial examples are malicious inputs, crafted by an adversary from correctly classified examples, to induce misclassification. This paper studies an intriguing, yet far overlooked consequence of the excessive sensitivity, that is, a misclassified example can be easily perturbed to help the model to produce correct output. Such perturbed examples look harmless, but actually can be maliciously utilized by false friend to make the model salf-satisfied. Thus name them hypocritical examples. With false friends like these, a poorly performed model could behave like state-of-the-un one. Once deployer trusts the hypocritical performance and uses the "well-performed" model in real-world applications, po- tential security concerns appear even in benign environments. In this paper, we formalize the hypocritical risk for the first time and propose a defense method spe- cialized for hypocritical examples by minimizing the tradeoff between natural risk and an upper bound of hypocritical risk. Moreover, our theoretical analysis reveals connections between adversarial risk and hypocritical risk. Extensive experiments verify the theoretical results and the effectiveness of our proposed methods.
INTRODUCTION
Deep neural networks (DNNs) have achieved breakthroughs in a variety of challenging problems such as image understanding (Krizhevsky et al. 2012), speech recognition (Graves et al.] 2013).
and automatic game playing (Mnih et al. (015)). Despite these remarkable successes, their perva-
sive failures in adversarial settings, the phenomenon of adversarial examples (Biggio et al. 2013 SSzeedyy et al. 2014), have attracted significant attention in recent years (Athalye et al. 2018 Carlini jct 2019 Tramer et al.) 2020). Such small perturbations on inputs crafted by adversaries are ca- pable of causing well-trained models to make big mistakes, which indicates that there is still large gap between machine and human perception, thus posing potential security concerns for practical machine learning (ML) applications (Kurakin et al. 2016 Qin et al. 2019 Wu et al.) 2020b)..
An adversarial example is "an input to a ML model that intentionally designed by an attacker to fool the model into producing incorrect output" (Goodfellow & Papernot, 2017). Following
the definition of adversarial examples on classification problems (Goodfellow et al. [015 Papernot
al. 2016 Elsayed al. 2018 Carlini al. 2019 Zhang al. [2019; Wang et al. [2220b] Zhang
et al. 2020 Tramèr et al. 2020), given a DNN classifier f and a correctly classified example æ y). an adversarial example Ta generated by perturbing such
with class label y (i.e., [(x
that f( * y and Ladv E (z). The neighborhood B. (x) denotes the set of points within fixed distance e O of æ, as measured by some metric (e.g., the Lp distance), so that Fadv is visually the "same" for human observers. Then, an imperfection of the classifier is highlighted by Padv = (cc()))cc(( the performance gap between the accuracy (denoted by Acc(-)) evaluated on clean set sampled from data distribution D and adversarially perturbed set A.
An adversary could construct such a perturbed set A that looks no different from D but can severely degrade the performance of even sfate-of-the-ar DNN models. From direct attacks in the digital space [Goodfellow et al.] 2015 Carlini & Wagner 2017) to robust attacks in the physical world (Kurakin et al. 2016; Xu et al.) 2020), from toy classification problems (Chen et al. 2020 Dobriban ctt aal,][220) to complicated perception tasks (Zhang & Wang 2019 Wang et al. 2020a). from the
high dimensional nature of the input space (Goodfellow al. 2015 Gilmer al. 2018) to the

---

Under review as a conference paper at ICLR 2021
(x)
Fadv
Wadv misclessiiiet fied as "tennis ball!"
commenty classified B8 "pandia"
(a) Illustration of adversarial examples
Zhyo
B (ar)
37yyp correclly clessified as "panda"
misclessified
as "tripod"
(b) Illustration of hypocritical examples
Figure 1: Comparison between adversarial examples and hypocritical examples. Left: Conceptual diagrams for the generation of adversarial example Ladv and hypocritical example Thyp. The input space (ground-truth) classified into the orange lined region (e.g.. class "not panda") " and the blue dotted region (e.g., class "panda"). The black solid line is the decision boundary of non- robust model, which classifies the region above the boundary as "panda" and the region below the boundary as "not panda". Red shadow and black shadow in the ball (xx denote that the points in there are misclassified and correctly classified, respectively. As we can see, Tadv or Zhyp can be easily found by perturbing correctly classified boundary. Right: A demonstration of adversarial examples and hypocritical examples on real data. Here we choose ResNet50 (He et al., 2016a) trained on ImageNet (Russakovsky et al. 2015) as the victim model. In (a) the correctly classified "panda" can be stealthily perturbed to be misclassified as "tennis ball". In (b) the "panda" (misclassified as "tripod") can be stealthily perturbed to be correctly classified. Perturbations are rescaled for display.
misclassified æ across the model's 's decision
framework of (non)-robust features (Jelley et al.) 2018 Tlas al. 2019), many efforts have been devoted to understanding and mitigating the risk raised by adversarial examples, thus closing the gap Gadv- Previous works mainly concern the adversarial risk on correctly classified examples. However, they typically neglect a risk on misclassified examples themselves which will be formalized in this work.
In this paper, we first investigate an intriguing, yet far overlooked phenomenon, where given a DNN classifier f and a misclassified example with class label y (i.e., f(x) * y). we can easily perturb to Thyp such that /((hhy)) = and Thyp (xr). Such an example Thyp looks harmless, but actually can maliciously utilized by a false friend to fool model be self-satisfied. Thus we name them hypocritical examples (see Figure I for comparison with adversarial examples).
Adversarial examples and hypocritical examples are two sides of the same coin. On the one side, well-performed but sensitive model becomes unreliable in the existence of adversaries. On the other side, a poorly performed but sensitive model behaves well with the help friends. With false friends like these, naturally trained suboptimal model could have of-th-----e-art performance, and even worse, a randomly initialized model could behave like well-trained one (see Section 2.1).
It is natural then to wonder: Why should we care about hypocritical examples? Here give two main reasons:
1. This is of scientific interest. Hypocritical examples are the opposite of adversarial exam- ples. While adversarial examples are hard test data to a model, hypocritical examples aim make it easy do correct classification. Hypocritical examples warn MI researchers
2

---

Under review as a conference paper at ICLR 2021
think carefully about high test accuracy: Does our model truly achieve human- like intelli- gence, or is it just simply because the test data prefers the model?
2. There are practical threats. A variety of nefarious ends may be achievable if the mistakes of MI. systems can be covered up by hypocritical attackers. For instance, before allowing autonomous vehicles to drive on public roads, manufacturers must first pass tests in specific environments (closed or open roads) to obtain a license (Administration et al.) 2016 Briefs 2015 Lei 2018). An attacker may add imperceptible perturbations on the test examples (e.g., the "stop sign" on the road) stealthily without human notice, to hypocritically help ML-based autonomous vehicle to pass the tests that might otherwise fail. However, the high performance can not be maintained on public roads without the help of the attacker. Thus, the potential risk is underestimated and traffic accidents might happen unexpectedly when the vehicle driving on public roads.
In such a case, if the examples used to evaluate a model are falsified by a false friend, the model will manifest like a perfect one (on hypocritical examples), but actually may not be well performed even on clean examples, not to mention adversarial examples. Thus a new imperfection of the classi- fier can be found in Gnyp Acc(F) - Acc(D), the performance gap between the accuracy evaluated on clean set sampled from D and hypocritically perturbed set F. Still. F looks no different from D but can stealthily upgrade the performance. Once deployer trusts the hypocritical performance carefully designed by false friend and uses the "well-performed"" model in real-world applications, potential security concerns appear even in benign environments. Thus we need methods defend our models from false friends, that is, making our models have self-knowledge.
We propose a defense method by improving model robustness against hypocritical perturbations. Specifically, we formalize the hypocritical risk and minimize it via differentiable surrogate loss (Section 3). Experimentally, we verify the effectiveness of our proposed attack (Section 2.1) and defense (Section 4.1 Further, we study the transferability of hypocritical examples across mod- els trained with various methods (Section 4.2). Finally, we conclude our paper by discussing and summarizing our results (Section 5 and Section 6p. Our main contributions are:
We give formal definition of hypocritical examples. We demonstrate the unreliability of standard evaluation process in the existence of false friends and show the potential security risk on the deployment of model with high hypocritical performance.
We formalize the hypocritical risk and analyze its relation with natural risk and adversarial risk. We propose the first defense method specialized for hypocritical examples by mini- mizing the tradeoff between the natural risk and an upper bound of hypocritical risk.
Extensive experiments verify the effectiveness of our proposed methods. We also examine the transferability of hypocritical examples. We show that the transferability not always desired by the attackers, which depends on their purpose.
FALSE FRIENDS AND ADVERSARIES
Better an open enemy than a false friend! Only by being aware of the potential risk of the false friend can we prevent it. In this section, we expose kind of false friends, who are capable of ma- nipulating model performance stealthily during the evaluation process, thus making the evaluation results unreliable.
We consider a classification task with data (I, y) E Rd x (l,..., C} from a distribution D. Denote by f Rd + {l,..., C}} the classifier which predicts the class of an input example æ: f(x)
arg max; px(x), where P& (z) is the kth component of p(x) Rd A (e.g., the output after
softmax activation). in which C {u RC 1"
2 0} is the probabilistic simplex.
Adversarial examples are malicious inputs crafted by an ad adversary to induce misclassification. We first give the commonly accepted definitonn of adversarial examples as follows:
Definition (Adversarial Examples). Given a classifier I and a correctly classified imput (ar y) ~ D (i.e,, fx) = y). an c-bounded adversarial example is an input x² E Rdd such that:
f(x") ) # and x² Bx
3

---

Under review as a conference paper at ICLR 2021
The assumption underlying this definition is that inputs satisfying x² E (x)x) preserve the label y of the original input a. The reason for the existence of adversarial examples is that a model overly sensitive to non-semantic changes. Next, formalize complementary phenomenon to adversarial examples, called hypocritical examples. Hypocritical examples are malicious inputs crafted by false friend to stealthily correct the prediction of model:
Definition 2 (Hypocritical Examples). Given classifier I and misclassified input (a. ~ DD (i.e., f(x) # y), an e-bounded hypocritical example an input x E IR such that:
f(x") y and x² (x).
The same as adversarial examples, hypocritical examples are bounded to preserve the label of the original input, and are another consequence that arises from excessive sensitivity classifier.
As false friend, hypocritical example can be generated from misclassified example by maxi- mizing
1(J(x') v):
a'E.((m)
which is equivalent to minimizing
min ar (m.
1(f(x') + v):
where I (-) is the indicator function. Similar to Madry et al. (2018): Wang al. (2020b), practice, we leverage the commonly used cross entropy (CE) loss as the surrogate loss of (J(x')) o y) and minimize it by projected gradient descent (PGD).
Note that Equation 2 looks similar to but conceptually differs from the known targeted adversarial at- tack (Carlini Wagner 2017), which generates kind of adversarial examples defined on correctly classified clean inputs and targeted to wrong classes. The hypocritical examples here are defined on misclassified inputs and are targeted to their right classes.
2.1
ATTACK RESULTS
In this subsection, we demonstrate the power of our proposed hypocritical attack on three bench- mark datasets: MNIST (LeCun et al. 1998). CIFAR-10 (Krizhevsky et al. 2009) and ImageNet (Russakovsky al. 2015).
Table 1: Accuracy (%) evaluated on MNIST.
Table Accuracy (%) evaluated on ImageNet. Attacks are bounded with 16/255.
Attacks are bounded with c = 0.2.
Model
F
D
Model
F
D
Naive (MLP)
100.0
10.4
0.0
Naive (VGG16)
100.0
0.1
0.0
Naive (LeNet)
79.2
10.1
0.0
Naive (ResNet50)
12.6
0.1
0.0
Standard (MLP)
100.0
97.8
29.8
Standard (VGG16)
99.9
71.6
0.3
Standard (LeNet)
100.0
99.4
0.1
Standard (ResNet50)
99.9
76.1
0.0
We attack models trained with standard approach using clean examples (Standard) and models that randomly initialized without training (Naive). For MNIST, the hypocritically perturbed F and the adversarially perturbed set A are constructed by attacking every example in the clean test set sampled from D. Both attacks are bounded by a loc ball with radius 0.2. For ImageNet, A are constructed based on its validation set sampled from D. Both attacks are bounded by a loc. ball with radius e = 16/255. For each experiment, we conduct trials with different random seeds and report the averaged result to reduce the impact of random variations. Appendix A.2 describes further experimental details about DNN architecture, training procedure and more results.
and
Results on MNIST and ImageNet are summarized in Table and Table respectively. First, we find that the naturally trained models are extremely sensitive to hypocritical perturbations (e.g., Standard (MLP) and Standard (LeNet) achieve 100% accuracy hypocritically perturbed MNIST test set, and Standard (VGG16) and Standard (ResNet50) achieve 99% accuracy on hypocritically perturbed ImageNet validation set). Second, we find that part of randomly initialized models is extremely
4

---

Under review as a conference paper at ICLR 2021
sensitive (e.g., Naive (MLP) and Naive (VGG16) achieve 100% accuracy on F of MNIST and ImageNet). These results demonstrate the unreliability of standard evaluation process in the exis- tence of false friends. Once "well-performed" model (such Naive (MLP) or Naive (VGG16)) is permitted to be deployed in real-world applications due to that the deployer has a false sense of performance, potential security concems appear even in benign environments.
seems that the Naive (ResNet50) model relatively robust to hypocritical examples on ImageNet. But that is just trivial defense, it simply predicts most of the points in input region as a certain class because of the poor scaling of network weights at initialization (He et al., 2016b Elsayed al., 2019). More discussions are in Appendix A.2 Therefore, it is not enough blindly pursue robustness against hypocritical perturbations but ignore the performance on clean examples.
3 HYPOCRITICAL RISK
In this section, we formalize the hypocritical risk and analyze the relation between natural risk, adversarial risk. and hypocritical risk. We propose defense method specialized for hypocritical examples by minimizing the tradcoff between natural risk and an upper bound of the hypocritical risk. Moreover, by decomposing existing method designed for adversarial defense (TRADES (Zhang et al., 2019)). we find that, surprisingly, TRADES minimizes not only the adversarial risk on correctly classified examples, but also looser upper bound of the hypocritical risk. Our theoretical analysis suggests that TRADES can be another candidate defense method for hypocritical examples.
To characterize the adversarial robustness of classifier S. Madry al. (0188 Uesato et al. (2018);
Cullina et al. (2018) defined the adversarial risk under the threat model of bounded ball:
Rad (z.,/~~D
E max 1(f(2') + v) (2)
(3)
The standard measure of classifier performance, known as natural risk, is denoted (f) E(m.g)>> [1((f)) * v)]. Let q( y) be the probability density function of data distribution D. We denote by S the conditional data distribution on correctly classified examples w.r.t. f. with a conditional density function g(ær y E) q( v) Z(E) if E is true (otherwise q(æ, E) 0), where the event E f(x)) = and E) = Sa.y 1 (f(x) = y)dq((r, y) is a normalizing constant.
We denote by Sj the conditional data distribution on misclassified examples with the conditional density function q(v) y E) and f(x) * y as the event E. Then we have the following relation between the natural risk and the adversarial risk:
Proposition 1. Denote the adversarial risk on correctly classified examples by
Rad
I
max (x)
((fxx') # v)
(m.y)~s}
then we have
Rad (f) Reat + (1 R (f).
Proposition D shows that we can view the adversarial risk RJ) as the tradeoff between Rnat (f) and Radv (f) f) with the scaling parameter
Rnaa(f). The adversarial risk on correctly classified examples Radv(f) (f) is in sharp contrast to the hypocritical risk defined on misclassified examples formalized follows:
Definition 3 (Hypocritical Risk). The hypocritical risk on misclassifeed examples of a classifier f under the threat model of bounded ball is defined as
1(f(x')= = v)
Rfy
E
max 20 (B)
(m
The hypocritical risk Rhyp (f) is the proportion of perturbed examples (originally misclassified) that can be successfully correctly classified by the classifier after a false friend's attack. When considering the existence of false friends, good model should have not only low natural risk but also low hypocritical risk, to be robust against hypocritical perturbations.
5

---

Under review as a conference paper at ICLR 2021
3.1
TRADEOFF BETWEEN NATURAL AND HYPOCRITICAL RISKS
((()
1/2
Figure 2: Counterexample given by Equation
Motivated by the tradeoff between natural and adversarial risks Tsipras et (2019) ; Zhang et al. (2019), we notice that there may also exist an inherent tension between the goal of natural risk minimization and hypocritical risk minimization. To illustrate the phenomenon, we provide toy example here, which is modified from the example in Zhang et al. (2019), and its risk minimization solutions can analytically found.
Consider the case (x, v) IR x (-1,+1 from distribution D, where the marginal distribution over the instance space is uniform distribution over [0, 1], and for i 0, 0,
2
(zz) Pr(y +1
{{ 1/4,
I E 22k, (2k 1)e), 1 x ((2k + 1)e, (2k + 2)e].
(4)
See Figure 2 for visualization of sf(x). In this problem, we consider two classifiers: a) the Bayes optimal classifier sign(2n(z) b) the all-one classifier which always outputs "positive". Table displays the trade-off between natural and hypocritical risks: the minimal natural risk 1 is achieved by the Bayes optimal classifier with large hypocritical risk, while the optimal hypocritical risk is achieved by the all-one classifier with large natural risk.
Table 3: Comparison of Bayes optimal classifier and all-one classifier.
Bayes Optimal Classifier
All-One Classifier
Rnat Rnyp
1/8 (optimal)
3/8
(optimal)
3.2 UPPER BOUNDS OF HYPOCRITICAL RISK
is natural then to optimize our models to minimize natural and hypocritical risks at the same time. However, it's hard to do optimization over Rfyp( To ease the optimization obstacles in there, we derive the following upper bounds.
Theorem 1. For any data distribution D and its corresponding conditional distribution on misclas- sified examples S w.E.t. a classifier f. we have
(Xhhy) =) <
E (..))~;
1(f(D)))) + f(x))
(f # f(x)),
(m.))~57
(m.3))~77
Rfy(f)
Rayg(f)
where Thyp
arg max (f((x') y) and Drev arg max 1 (f((x') + f(x)). E'E (m))
Here Brev means that it pursues to reverse a clean example to a different class, from the point of view of the model. The upper bounds found in Theorem allow to optimize the hypocritical risk using

---

Under review as a conference paper at ICLR 2021
proper surrogate loss functions which are both physical meaningful and computaionally tractable. Before moving forward to algorithmic design, we state a useful proposition below, which reveals the internal mechanism behind in TRADES.
Proposition 2. (f) (1 nat Rf) (f)+ Reat (nat( Rny))))
(ff( + f())).
(0.a)//
Proposition 2 shows a connection between adversarial risk and hypocritical risk: the adversarial risk on correctly classified examples Radv (f) and the looser upper bound of the hypocritical risk on misclassified examples (J) can be seamlessly united to new risk on all examples (f). We name it reversible risk since minimizing it pursues the model whose predictions can't 't be reversed by small perturbations.
3.3 ALGORITHMIC DESIGN
Now we are ready to design objective functions that improve model robustness against hypocritical examples while keeping model accuracy on clean examples.
Similar to Zhang et al. 2201); Wang al. (2020b); Tramer et al. (2020) Raghunathan et al. 22020,
we propose a defense objective by minimizing the tradeoff between the natural risk and the tighter upper bound of the hypocritical risk:
RTHRM(f) = (f) + (f).
(5)
where À > O is a tunable scaling parameter balancing the importance of natural risk and hypocritical risk. We name our method THRM (Tradeoff for Hypocritical Risk Minimization).
0-1 loss in THRM is still intractable. In practice. for the indicator function (f(x)) # y) in (f), we adopt the commonly used CE loss as surrogate loss. Observed that Rny = (f) ((a.y)>11 (f((n((()))))))))) + f(=)), we absorb the Raat(f) term into À and use KL divergence as the surrogate loss of the indicator function 1 (Thyp) * f f=)) (Zheng et al. 2016 Zhang et al., [2019] Wang et al. 2020b), since (Xnnp) * f(x) implies that the perturbed examples have different output distributions to that of clean examples. Our final objective function for THRM becomes
Optimization
LTHRM
E (a..y~~D
(CCE(p()))))
v) ACKL(((Z); p(#ng)))
Intuition behind the objective LTHRM: the first term in Equation 6 enncouraees the natural risk to be
optimized, while the second regularization term encourages the output to be stable against hypocrit- ical perturbations, that is, the classifier should not be overly confident in its predictions especially when a false friend wants it to be.
To derive the objective function for TRADES, we can minimize the tradeoff between the natural risk and the reversible risk:
RTRADES = (f) + (f)
Similar to THRM, we use CE loss and KL divergence as the surrogate loss of 1(f(x) * y) and ((ee)) # f(x)). respectively. The final objective function becomes
LTRADES (w.y)~>D
(CCE(p(x)).)) y) + ALKL (p(=), p(#rev))),
which is exactly the multi-class classification objective function first proposed in Zhang et al. (2019) for adversarial defense. From the perspective of the hypocritical risk, our Proposition reveals an advantage behind it, that is, TRADES is capable of minimizing the upper bound of hypocritical
risk Rnyp( thus can considered as a candidate defense method for hypocritical examples. Proposition 2 also implies that there may be deeper connection between adversarial robustness and hypocritical robustness. We will discuss more and compare our proposed THRM with TRADES in next section.
4 EXPERIMENTS
In this section, to verify the effectiveness of the methods (THRM and TRADES) suggested Sec- tion 3.3 we conduct experiments real-world datasets including MNIST and CIFAR-10.
7

---

Under review as a conference paper at ICLR 2021
10
Trendine of TRADES Trendine of THIM
Trendline TRADES
90-
Trenclir THIM
TRAEES
TRAUES
THIM
THRM
Reyp (%)
Roop
25
Reat (%)
Row (%)
(a) On MNIST. Perturbations are bounded by loo norm with = 0.2.
(b) On CIFAR-10. Perturbations are
bounded by (sc norm with e 2/255.
Figure 3: Tradeoff between natural risk and hypocritical risk on real-world datasets.
4.1 WHITE- BOX ANALYSIS
For the wide range of the scaling parameter X, we conduct experiments in parallel over multiple NVIDIA Tesla V100 GPUs. On MNIST, perturbations are bounded by Ix norm with e 0.2. On CIFAR-10, models are trained against different hypocritical attackers bounded by 1s norm with = 1/255, 2/255 and 8/255, respectively. Each experiment is conducted 3 times with different random seeds. The hypocritical risk reported here is actually an approximation of the real value, since the optimization problem in it is NP-hard and we approximately solve it using surrogate loss and PGD on test set. Further details about model architecture and training procedure are in Appendix AA.3] Note that these experiments are extensive. It takes over 230 GPU days to completely train the models considered in this section. We believe that these experiments are beneficial to the ML community to further understand the tradeoffs and relative merits in THRM and TRADES.
Results on MNIST (e = 0.2) and CIFAR10 10 (e = 2/255) are shown in Figure 3 Each data point rep- resents a model trained with different A. More results including comparison with Madry's 's defense (Madry et al. 2018) are provided in Appendix [A.3] due to the limited space. First, we observe that, on both datasets, as the regularization parameter increases, the natural risk Rnat increases while the hypocritical risk Rhyp decreases, which verifies the effectiveness of our proposed method and the theoretical analysis in Proposition where we reveal that TRADES is capable of minimizing looser upper bound of hypocritical risk. Second, we show that THRM achieves better tradeoff on MNIST since it optimizes tighter upper bound than TRADES. However, the situation becomes nuanced on CIFAR-IO. As we can see in Figure 3(6) THRM seems to behave better in the begin- ning when À is small but is surpassed by TRADES when À increases. Overall, optimizing only tighter upper bound of hypocritical risk achieves better tradeoff on test set when the task relatively simple (e.g., on MNIST with 0.2), while simultaneously optimizing hypocritical risk and ad- versarial risk achieves better tradeoff on test set when the task tends to be hard (e.g., on CIFAR-10 with e 2/255 and 8/255).
Above phenomenon shows that, when dealing with finite sample size and finite-time gradient- descent trained classifiers, better adversarial robustness may help the generalization of hypocritical robustness, which conforms our intuition that they are two sides of the same coin. Interestingly, contemporary work claims that, on CIFAR-10, TRADES achieves better adversarial robustness than Madry's 's defense in fair hyperparameter settings (Anonymous) [202]) Thus there may be potential mutual benefits between adversarial robustness and hypocritical robustness. After all, robust train- ing objectives force DNNs to be invariant to signals that humans are invariant to, which may lead to feature representations that are more similar to what humans use (Salman et al. 2020). A rigor- ous treatment of the synergism is beyond the scope of the current paper but is important future direction.
4.2 TRANSFERABILITY ANALYSIS
Transferability of adversarial examples across models is well known TTamerr et al. 2017 Papernot al. 2017b: Ilyas al. 2019) and here we examine the transferability of hypocritical examples
8

---

Under review as a conference paper at ICLR 2021
on MNIST and CIFAR-10. We observe that hypocritical examples, i) can transfer easily between naturally trained models, ii) are hard to transfer from randomly initialized models to other models (and vise versa), iti) are hard to transfer from standard models to defended models, iv) generated from THRM models usually have high transferability. Experimental details are in Appendix [A.4] Better transferabilityy is beneficial for black-box attacks but is not always desired by hypocritical attackers. A hypocritical attacker only expects high transferability on the targeted model the attacker chose to help. If there are other competing models available to the deployer, the attacker actually does not want the hypocritical examples to be successfully transferred to those competing models. Thus fine-grate ane-grained attack methods are required. We leave this to future work.
DISCUSSION
The false friends considered in this paper as powerful as typical adversaries. They all know the ground truth labels of clean examples. Such powerful friends actually can help a model to not only correctly classify misclassified clean example but also correctly classify an adversarial example crafted by an adversary. One may expect to rely on true friends against adversaries. Unfortunately, an omniscient and faithful friend unachievable in practical tasks, so far at least. Once it achieved, the problem of robustness disappears immediately. What we can do at present is using a relatively more robust model as surrogate of the true friend to improve the robustness of a weak model. This induces a promising general method in practice, that is, nigh-performance models can be employed as true friends to help a weak model without exposing training data and model weights for the purpose of privacy protection and knowledge transfer (Abadi et al. 2016 Papernot et al. 2017a). Additional discussions are Appendix
CONCLUSION
In this work, we expose new risk arising from excessive sensitivity. Model performance becomes hypocritical in the existence of false friends. By formalizing the hypocritical risk and analyzing its relation with natural risk and adversarial risk, we propose to use THRM and TRADES as de- fense methods against hypocritical perturbations. Extensive experiments verify the effectiveness of methods. These findings open new avenues for mitigating and exploiting model sensitivity.
REFERENCES
Martin Abadi, Andy Chu, Ian Goodfellow, Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308-318, 2016
National Highway Traffic Safety Administration et al. Federal automated vehicles policy: Acceler- ating the next revolution in roadway safety. US Department of Transportation, 2016.
Sergio A Alvarez. An exact analytical relation among recall, precision, and classification accuracy in information retrieval. Boston College, Boston, Technical Report BCCS-02-01 pp. 1-22, 2002.
Anonymous. Bag of tricks for adversarial training. In Submitted to International Confer- ence on Learning Representations, 2021. URL XbčxvrtB8Ce under review.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of se- curity: Circumventing defenses to adversarial examples. In International Conference on Machine Learning (ICML), 2018.
Battista Biggio, Igino Corona, Davide Maiorca, B Nelson, N Smdic, P Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European confer- ence on machine learning and knowledge discovery in databases (ECML PKDD), 2013.
UMTRI Briefs. Mcity grand opening. Research Review, 46(3), 2015.
Michael Buckland and Fredric Gey. The relationship between recall and precision. Journal of the American society for information science, 45(1):12-19 1994.

---

Under review as a conference paper at ICLR 2021
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE symposium on security and privacy (sp), 2017.
Nicholas Carlini, Anish Athalye, Nicolas Papemot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. arXiv preprint arXiv: Setiv:900.. /902.06705 2019.
Lin Chen, Yifei Min, Mingrui Zhang, and Amin Karbasi. More data can expand the generalization gap between adversarially robust and standard models. In International Conference Machine Learning (ICML), 2020.
Daniel Cullina, Arjun Nitin Bhagoji, and Prateek Mittal. Pac-leamning in the presence of adversaries. In Advances in Neural Information Processing Systems (NeurIPS), 2018.
Edgar Dobriban, Hamed Hassani, David Hong, and Alexander Robey. Provable tradeoff's in adver- sarially robust classification. arXiv preprint arXiv:2006.05161, 2020.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. In Proceedings of the IEFE conference computer vision and pattern recognition (CVPR), 2018.
Gamaleldin Elsayed, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alexey Kurakin, lan Good- fellow, and Jascha Sohl-Dickstein. Adversarial examples that fool both computer vision and time-limited humans. In Advances in Neural Information Processing Systems (NeurIPS), 2018.
Gamaleldin F. Elsayed, lan Goodfellow, and Jascha Sohl-Dickstein. Adversarial reprogramming of neural networks. In International Conference Learning Representations (ICLR), 2019.
Justin Gilmer, Luke Metz, Fartash Faghri, Samuel S Schoenbolz, Maithra Raghu, Martin Watten- berg, and Ian Goodfellow. Adversarial spheres. arXiv preprint arXiv: XXX1::801.0077/ 2018.
Ian Goodfellow and Nicolas Papernot. Is attacking machine learning easier than defending it. Blog post Feb, 15:2017, 2017.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015.
Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. Speech recognition with deep recur- rent neural networks. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2013.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016a.
Kun He, Yan Wang, and John Hoperoft. A powerful generative model using random weights for deep image representation. In Advances in Neural Information Processing Systems (NeurIPS), 2016b.
Andrew lyass, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Saumya Jetley, Nicholas Lord, and Philip Torr. With friends like these, who needs adversaries? In Advances in Neural Information Processing Systems (NeurIPS), 2018.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
10

---

Under review as a conference paper at ICLR 2021
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep volutional neural networks. In Advances in neural information processing systems (NeurIPS), 2012.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In International Conference on Learning Representations (ICLR) Workshops, 2016.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-23 -3224,, 1998.
Le On the establishment of autonomous vehicles regulatory system in china. Journal of Beijing Institute of Technology (Social Sciences Edition), (2):17, 2018.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial ex- amples and black-box attacks. In International Conference on Learning Reprexentations (ICLR), 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare Belle- mare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 51 8(754)) 2015.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In 2016 IEEE European sympo- sium on security and privacy (EuroS&P), 2016.
Nicolas Papernot, Martín Abadi, Ulfar Erlingsson, Ian Goodfellow, and Kunal Talwar. Semi- supervised knowledge transfer for deep learning from private training data. In International Conference on Learning Representations (ICLR), 2017a.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference computer and communications security (ASIA CCS), 2017b.
Yao Qin, Nicholas Carlini, Garrison Cottrell, Ian Goodfellow, and Colin Raffel. Imperceptible, bust, and targeted adversarial examples for automatic speech recognition. In International Con- ference Machine Learning (ICML), 2019.
Aditi Raghunathan. Sang Michael Xie, Fanny Yang. John Duchi, and Percy Liang. Understanding and mitigating the tradeoff between robustness and accuracy. In International Conference on Machine Learning (ICML), 2020.
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In International Conference Machine Learning (ICMI.) 2020.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision (IJCV), 115(3):211-255) 2015.
Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adver- sarially robust imagenet models transfer better? arXiv preprint arXiv:2007.08489 2020.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations (ICLR), 2014.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. In International Conference Learning Representations (ICLR), 2014.
Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Bonch, and Patrick McDaniel. The space of transferable adversarial examples. arXiv preprint arXiv: arXiv:/704.03:: 1704.0345, 2017.

---

Under review as a conference paper at ICLR 2021
Florian Tramèr, Jens Behrmann, Nicholas Carlini, Nicolas Papernot, and Momn-Herii Jacobsen. Fundamental tradeoffs between invariance and sensitivity to adversarial perturbations. In Inter- national Conference Machine Learning (ICML), 2020.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. arXiv preprint arXiv:2002.0833 083447 2020.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be odds with accuracy. In International Conference on Learning Representa- lions (ICLR), 2019.
Jonathan Uesato, Brendan O' O'Donoghue, Pushmeet Kohli, and Aaron Oord. Adversarial risk and the dangers of evaluating against weak attacks. In International Conference on Machine Learning (ICML), 2018.
Hongjun Wang, Guangrun Wang. Ya Dongyu Zhang. and Liang Lin. Transferable, control- lable, and inconspicuous adversarial attacks on person re-identification with deep mis-ranking, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020a.
Yisen Wang, Difan Zou, Jinfeng Yi. James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learming Representations (ICLR), 2020b.
Dongxian Wu, Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip connections matter: On the transferability of adversarial examples generated with resnets. In International Conference on Learning Representations (ICLR), 2020a.
Tong Wu, Liang Tong, and Yevgeniy Vorobeychik. Defending against physically realizable attacks on image classification. In International Conference on Learning Representations (ICLR), 2020b.
Kaidi Xu, Gaoyuan Zhang, Sijia Liu, Quanfu Fan, Mengshu Sun, Hongge Chen, Pin- Yu Chen, Yanzhi Wang, and Xue Lin. Adversarial t-shirt! evading person detectors in a physical world. In
Proceedings of the European Conference on Computer Vision (ECCV), 2020.
Sergey Zagoruyko and Nikos Komodakis.
Wide residual networks.
arXiv preprint
arXiv:1605.07146. 2016.
Haichao Zhang and Jianyu Wang. Towards adversarially robust object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning (ICML), 2019.
Jingfeng Zhang. Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In International Conference on Machine Learning (ICML), 2020.
Stephan Zheng, Yang Song, Thomas Leung, and Ian Goodfellow. Improving the robustness of deep neural networks via stability training. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
12

---

Under review as a conference paper at ICLR 2021
A EXPERIMENTAL DETAILS
A.1 DETAILS IN FIGURE 1
Attack procedure. In adversarial attacks, we perturb clean inputs maximize the surrogate loss using PGD. In hypocritical attacks, we perturb clean inputs to minimize the surrogate loss using PGD. In both attacks, for the purpose of imperceptibility, we execute PGD attack 100 steps (step size is e/50) with early stopping ImageNet and the budget here is 2/255.
More examples. More adversarial examples and hypocritical examples generated on ImageNet using our methods are shown in Figure 5 More hypocritical examples generated on MNIST and CIFAR-10 10 are shown in Figure 4(a) and Figure 4(b) The victim models are LeNet (Standard) and Wide ResNet (Standard) for MNIST and CIFAR-IO, respectively. They are trained with the same procedures described in Appendix A.. In both attacks, for the purpose of imperceptibility, we execute 100 steps PGD attacks (step size is c/50) with early stopping MNIST and CIFAR-10.
The budget for MNIST here is 0.2. The budget for CIFAR-10 here is 8/255.
1
1
2
3
3
4
4
3
3
Frgg
B
B
Y
(a) On MNIST
(b) On CIFAR-10
Figure 4: Hypocritical examples. In each subfigure, the first cloumn represents the clean examples sampled from original data distribution, the second cloumn represents the generated perturbations, the third cloumn represents the perturbed examples. Perturbations are rescaled for display. Red labels and black labels below images denote misclassification and correct classification, respectively.
13

---

Under review as a conference paper at ICLR 2021
goose
black
oystertatcher
goose
lampstade
Epspessrr
Fox
screwddtvver
forkllift
w wardrobe
washing machine
washing machine
strethher
photocopker
washing machine
hald eagle
kite
remote control
screwdrriver er
sydinge
ballpoint pen
grocary store
restaurant
shovel
wbeelbarrow
paimbrush
lacile
greenhouse
library
tiger shark
snoek
(a) Adversarial examples
(b) Hypocritical examples
Figure 5: More examples on ImageNet. In each subfigure, the first cloumn represents the clean ex- amples sampled from original data distribution, the second cloumn represents the generated pertur- bations, the third cloumn represents the perturbed examples. Perturbations are rescaled for display. The model predictions of these images are shown below each image. Red labels and black labels below images denote misclassification and correct classification, respectively.
14

---

Under review as a conference paper at ICLR 2021
A.2 DETAILS IN SECTION 2.1
Architecture. For MNIST, four-layer multilayer perception (MLP) (2 hidden layers, 768 neu- rons in each) with ReLU activations and variant of LeNet model (2 convolutional layers of sizes 32 and 64, and a fully connected layer of size 1024) are adopted. For CIFAR-10 10, four-layer MLP (2 hidden layers, 3072 neurons in each) with ReLU activations, ResNet18 (He et al. (2016a) and Wide ResNet (Zagoruyko & Komodakis (0166) (with depth 28 and width factor 10) are adopted. For ImageNet, VGGI6 (Simonyan & Zisserman 2014) and a ResNet50 (He et al. 2016a) are adopted.
Training procedure. i) Models trained with standard approach using clean examples (Standard). For MNIST, models trained for 80 epochs with Adam optimizer with batch size 128 and learning rate of 0.001. Early stopping is done with holding out 1000 examples from the MNIST training set. For CIFAR-IO, models are trained for 150 epochs with SGD optimizer with batch size 128 and the learning rate starts with 0.1, and is divided it by 10 at 90 and 125 epochs. We apply weight decay of 2e-4 and momentum of 0.9, Early stopping done with holding out 1000 examples from the CIFAR-I0 training set. For ImageNet, we use the pretrained standard models available within Py PyTorch
prrchvision.models. ii) Models that randomly initialized without training (Naive). For all models, we use the default PyTorch initialization, except that we initialize the convolutional weights in Wide ResNet with He initialization (He al., 2015). We conduct all the experiments using single NVIDIA Tesla V100 GPU. Each experiment is conducted 3 times with different random seeds, except the standard models trained on ImageNet, in which we use the pretrained standard models available within Py PyTorc.
Attack procedure. loss using PGD. In hypocritical attacks, we perturb clean inputs to minimize the surrogate loss using PGD. In both attacks, we execute 50 steps PGD attacks (step size is e/10) with 20 times of random restart on MNIST and CIFAR-10, and ImageNet. Other hyperparameter choices didn't it offer a significant change in accuracy. On MNIST, the hypocritical perturbed set F and the adversarially perturbed set A are constructed by attacking every example in the clean test set sampled from D. Both attacks are bounded by a (so ball with radius e 0.2. On CIFAR-1O,, both attacks are bounded by loo ball with radius 8/255, On ImageNet, F and A are constructed based on its validation set sampled from D. Both attacks are bounded by a &o ball with radius 16/255.
adversarial attacks, we perturb clean inputs to maximize the surrogate
50 steps PGD attacks (step size is e/8) on
Numerical results. The attack results on CIFAR- 10 are shown in Table 4 Full results of Table I. Table 2 and Table 4 are shown in Table 5 Table o and Table 7 respectively. Moreover, we show the attack results of 9 Naive models evaluated on ImageNet in Table O We find that all the Naive models in VGG family achieve high accuracy on F and all the Naive models in ResNet family have relatively poor performance on F. Especially, the Naive (ResNet152) model in Trial I is invariant to hypocritical perturbations. Even in the existence of a strong false friend, the hypocritical performance is still low the clean performance (only 0.1%). We carefully examined the Naive (ResNet152)) model and find that it's actually a trivial classifier, which purely classifies almost all the points in input region [0, 1]d as certain class for some simple reasons, such as poor scaling of network weights at initialization. Therefore, it is not enough to blindly pursue robustness against hypocritical perturbations but ignore the performance clean examples. Once train a Naive model with clean examples, the model becomes vulnerable immediately (see Standard (ResNet50)), whereas the trained weights are better conditioned (Elsayed et al. 2019).
Table 4: Accuracy (%) of models evaluated on CIFAR-10 10. Attacks are bounded with 8/255.
Model
F
D
A
Naive (MLP)
92.3
9.9
0.0
Naive (ResNet18)
20.8
8.7
0.3
Naive (Wide ResNet)
13.8
10.0
6.9
Standard (MLP)
88.6
45.1
3.9
Standard (ResNet19)
100.0
94.1
0.0
Standard (Wide ResNet)
100.0
95.1
0.0
15

---

Under review as a conference paper at ICLR 2021
Table 5: Full results of accuracy (%) evaluated MNIST. Attacks are bounded with 0.2.
Trial 1
Trial 2
Trial 3
Model
F
D
A
F
D
A
F
D
A
Naive (MLP)
100.0
11.2
0.0
100.0
10.8
0.0
100.0
9.4
0.0
Naive (LeNet)
69.1
9.7
0.0
77.5
10.6
0.0
91.0
10.1
0.0
Standard (MLP)
100.0
98.0
31.1
100.0
97.7
30.4
100.0
97.7
28.1
Standard (LeNet)
100.0
99.4
0.1
100.0
99.4
0.1
100.0
99.3
0.0
Table 6: Full results of accuracy (%) evaluated on ImageNet. Attacks are bounded with c 16/255.
Trial
Trial
Trial
Model
A
F
D
D
A
Naive (VGG11)
100.0
0.1
0.0
100.0
0.1
0.0
100.0
0.1
0.0
Naive (VGG13)
100.0
0.1
0.0
100.0
0.1
0.0
100.0
0.1
0.0
Naive (VGG16)
100.0
0.1
0.0
100.0
0.1
0.0
100.0
0.1
0.0
Naive (VGG19)
100.0
0.1
0.0
100.0
0.1
0.0
100.0
0.1
0.0
Naive (ResNet18)
58.4
0.1
0.0
83.2
0.1
0.0
57.6
0.1
0.0
Naive (ResNet34)
7.4
0.1
0.0
12.5
0.0
10.4
0.0
Naive (ResNet50)
10.6
0.1
0.0
14.7
0.1
0.0
12.6
0.1
0.0
Naive (ResNet101)
0.3
0.1
0.1
0.2
0.1
0.3
0.1
Naive (ResNet152)
0.1
0.1
0.1
0.3
0.1
0.1
0.2
0.1
0.1
Standard (VGG16)
99.9
71.6
0.3
N/A
N/A
N/A
N/A
N/A
N/A
Standard (ResNet50)
99.9
76.1
0.0
N/A
N/A
N/A
N/A
N/A
N/A
A.3 DETAILS IN SECTION
Architecture. For MNIST, variant of LeNet model (2 convolutional layers sizes 32 and 64, and fully connected layer of size 1024) is adopted. For CIFAR-10, a Wide ResNet (with depth 28 and width factor 10) adopted.
Training procedure. For the wide range of the scaling parameter X, we conduct experiments in parallel over multiple NVIDIA Tesla V100 GPUs. Each experiment conducted 3 times with different random seeds. For MNIST, all models (including Standard, Madry, TRADES, THRM) are trained for 80 epochs with Adam optimizer with batch size 128 and a learning rate of 0.001. Early stopping is done with holding 1000 examples from the MNIST training
suggested in Rice et al. (2020). For CIFAR-10, all models are trained for 150 epochs with SGD optimizer with batch size 128 and the learning rate starts with 0.1, and is divided it by 10 90 and 125 epochs. We apply weight decay of 2c-4 and momentum of 0.9. Early stopping is done with holding out 1000 examples from the CIFAR-I0 training set as suggested in Rice et (2020).
Attack procedure. For the inner maximization in the objective function of THRM, we perturb clean inputs to minimize the CE loss the surrogate loss. For the inner maximization in TRADES. we maximize the KL divergence as the surrogate loss. For the inner maximization in Madry, we maximize the CE loss as the surrogate loss. On MNIST, the training attack is PGD with random start and 10 iterations (step size e/4). On CIFAR-10 10, the training attack is PGD with random start 8/255, and the training attack PGD with random start
and 10 iterations (step size e/4) when 1/ /255 and = 2/255. On all experiments, the test attack 50 steps PGD (step size is e/10) with 20 times random restart. Other hyperparameter choices didn't 't offer significant change in accuracy.
and 7 iterations (step size e/3) when
Numerical results. The natural risk reported here is estimated on test set. The hypocritical risk reported here is estimated test set and is actually an approximation of the real value since we approximately solve the optimization problem by PGD on examples from test set. Results on MNIST (c 0.2) and CIFAR-10 (e 1/255, 2/255 and 8/255) shown in Figure
16

---

Under review as a conference paper at ICLR 2021
Table Full results of accuracy (%) evaluated on CIFAR-10. Attacks are bounded with 8/255.
Trial 1
Trial 2
Trial 3
Model
F
D
A
FF
D
A
F
D
A
Naive (MLP)
98.2
9.0
0.0
91.4
10.3
0.0
87.4
10.3
0.0
Naive (ResNet18)
14.8
10.0
0.6
20.7
9.2
0.2
27.0
6.9
0.0
Naive (Wide ResNet)
12.2
10.0
7.7
10.4
10.0
9.6
18.7
10.0
3.3
Standard (MLP)
89.1
46.2
3.6
88.4
44.8
3.8
88.3
44.5
4.2
Standard (ResNet18)
100.0
93.9
0.0
100.0
94.3
0.0
100.0
94.0
0.0
Standard (Wide ResNet)
100.0
95.0
0.0
100.0
95.1
0.0
100.0
95.3
0.0
6 Each point in Figure O represents one model trained with a certain x. Full numerical results on MNIST (E 0.2) and CIFAR-10 (e = 1/255, = 2/255 and E 8/255) can found in Table 9 Table T0 and Table I respectively. On MNIST (e 0.2), THRM has better tradeoff than TRADES. However, when the task becomes hard, TRADES performs as well as or better than THRM. On CIFAR-10. 10, as the task becomes harder (the larger the radius the harder the task), the gap between TRADES and THRM becomes larger. This phenomenon shows that better adversarial robustness may help the generalization of hypocritical robustness, especially when the task is hard. Moreover, we compare our methods with Madry et al. (2018)ss defense designed for adversarial robustness (denoted Madry)
Table 8
and and standard training method (denoted as Standard), We summarize results in Table [2] For direct comparison, we pick a certain À for each model trained by TRADES and THRM in each task. We observed that, in all tasks. Madry's 's defense has nonnegligible robustness on hypocritical examples, although there is no hypocritical risk or its upper bound in the objective function. This phenomenon indicates that optimizing only adversarial risk could bring certain degree of robustness against hypocritical examples. While this experimental results partly support our hypothesis (i.c., the potential mutual benefits between robustness against adversarial perturbations and hypocritical perturbations), further exploration needed. We note that the standard deviation becomes larger when is bigger in TRADES and THRM, which is attributed to optimization difficulty and result in more significant difference among different trials. Reducing the initial learning rate may mitigate this phenomenon.
do not take the evidence as an ultimate one and
For completeness, further evaluate the adversarial risk correctly classified examples of the models trained by THRM and TRADES Results on MNIST (e = 0.2) and CIFAR-10 (c = 2/255) summarized in Table T3 and Table respectively. One interesting finding is that models trained with THRM manifest noteworthy adversarial robustness, especially on CIFAR-10, although there is no adversarial risk in the objective function of THRM. These facts also support the hypothesis (i.e., the potential mutual benefits between robustness against adversarial perturbations and hypocritical perturbations).
'They actually optimize the adversarial risk in Equation via surrogate loss.
17

---

Under review as a conference paper at ICLR 2021
Trendline Trendline of THRM
Trendline TRADES
53
Trencline THRM
THRM
THRM
Rhyp (%6
Rhyp (%6
42-
nat
nat
(a) On MNIST. Perturbations are bounded by Lo norm with 0.2.
(b) On CIFAR-10 10. Perturbations are bounded by loo norm with 1/255.
200
183
Trendline of TRADES
TRADES
Rayo (%)
Ryy (9)
75-
Trendline of TRADES
Trendline THRM
TRADES
THRM
Rat (%)
Rat
(c) On CIFAR-10. Perturbations bounded by Loo norm with 2/255.
(d) On CTFAR-10. Perturbations bounded
by L. norm 8/255.
Figure 6: Tradeoff between natural risk and hypocritical risk on real-worth datasets.
Table 8: Full results of natural risk (%) and hypocritical risk (%) MNIST, Attacks bounded by loc norm with 0.2.
(a) For TRADES.
(b) For THRM.
Trial
Trial 2
Trial 3
Trial
Trial 2
Trial
Run
Ray
R
Rooo
Rua
Ropp
R.s.
Rhpp
Rea:
Ropp
R.w.
Rup
1.0
94,1
0.8
91.5
0.8
92.8
10
0.7
100.0
0.8
100.0
0.7
100.0
86.7
1.0
91.8
1.1
95.6
0.7
100.0
98.6
0.7
100.0
10
84.7
1.1
85.3
1.1
84.3
200
0.8
96.3
0.7
97.2
0.7
98.5
82.8
1.4
78.6
1.3
79.4
1.0
79.6
1.2
72.9
0.9
$1.8
$3.9
78.2
60
4.4
41.8
4.5
34,1
3.6
43.7
800
2.7
46.1
2.6
63.4
2.4
44.4
120
6,1
31.1
6.9
31.1
5.9
31.6
2000
5.8
34,1
4,8
26.4
3.8
28.3
160
28.7
8.6
31,3
6.4
31.7
7.3
31.6
14.0
17.8
7.5
20.4
200
31.2
7.7
31.3
7.1
30.2
4000
6.7
25.7
5,5
23.3
9.0
18.2
240
9.9
26.8
8.8
36.3
9.1
28.6
5000
10.4
20.9
5.7
29.3
11.6
17.7
26.9
500
11.4
21.6
14.0
24.7
11.5
25.2
7000
13.3
12.6
6.3
31.4
9.3
20.3
18

---

Under review as a conference paper at ICLR 2021
Table 9: Full results of natural risk (%) and hypocritical risk (%) on CIFAR-10. Attacks are bounded by loc norm with 1/255.
(a) For TRADES.
(b) For THRM.
Trial
Trial 2
Trial
Trial
Trial 2
Trial
Rust
Rhyp
Rest
Ryy
Rut
Rhyp
Rest
Rat
Rhip
Rux
Rayp
0.1
5.2
84.5
5.1
85.5
4.8
80.1
4.6
91.8
4.6
90.7
4.5
89.7
63.1
5.5
65.1
64.4
74.9
72.6
72.6
5
6.3
$2.0
5.9
50.4
5.8
53.2
50
5.3
62.6
5.5
62.0
5.4
60.4
10
48.3
6.9
45.0
100
5.6
59.1
58.2
58.7
30
9.0
38.0
8.9
41.7
36,9
200
6.6
52.7
6.5
49.2
6.2
51.6
50
8.5
38.9
8.4
36.8
38.4
300
6.9
44.8
6.5
45.5
7.0
46.4
70
9.0
34.5
8.8
32.0
400
7.3
47.6
42.7
7.9
40.6
90
9.7
32.0
10.4
28.0
9.3
31.1
600
8.4
37.7
8.6
36.8
8.6
37,9
120
34.4
150
10.2
30.9
10.1
28.0
10.1
31.
1000
9.9
30.7
9.8
33.5
10.5
31.4
180
10.6
29.2
10.4
31.8
10.5
31.0
1500
11.6
28.7
11.5
29.4
27.3
210
11.2
27.9
11.4
30.8
11.0
28.0
2000
13.2
24.7
12.2
24.4
12.8
24.1
Table 10: Full results of natural risk (%) and hypocritical risk (%) on CIFAR-I0. Attacks are bounded by loo norm with 2/255.
(a) For TRADES.
(b) For THRM.
Trial
Trial 2
Trial
Trial
Trial
Trial
Riat
Knyp
Rast
Rayp
Rot
Reyp
Raxt
R
Rest
Rhyp
Roat
Rayp
0.1
5.4
91.3
5.3
90.6
5.4
90.0
1
4.8
99.6
4.5
99.3
4.7
99.6
5
8.5
64.4
7.7
64.6
8.7
65.7
50
6.3
85.7
5.8
84.1
5.2
85.4
9.0
6.7
78.4
30
10.2
54.6
10.1
51.9
10.1
54.9
150
7.1
71.7
69.9
7.5
72.8
50
11.1
46.5
11.2
48.9
11.0
48.5
200
8.4
65.6
7.9
65.0
8.5
66.7
70
44.7
11.6
250
8.8
62.4
62.3
90
12.1
42.1
12.4
46.1
12.4
43.8
300
9.6
59.4
9.0
60.2
9.7
60.0
150
13.6
37.3
13.9
39.3
13.4
35.0
600
11.8
50.5
11.4
51.1
11.8
50.3
180
14.1
34.8
14,2
14.4
37.7
12.5
45.6
12.9
45.2
13.3
46,2
210
14.7
32.9
14.6
32.4
14.5
32.9
1000
14.2
42.3
13.8
42.3
14.0
42,4
Table 11: Full results of natural risk (%) and hypocritical risk (%) on CIFAR- 10. Attacks are bounded by Ioo norm with 8/255.
For TRADES.
(b) For THRM.
Trial
Trial 2
Trial
Trial
Trial
Trial 3
R.su:
Rays
Ruat
Rhy
Ruut
Resp
R
Rays
R
Rays
R
Rays
0.1
16.6
94.0
20.5
95.3
13.0
99.4
1
4.8
100.0
9.0
100.0
5.4
100.0
7.7
15.1
89.4
15.2
87.7
15.6
89.6
50
12.0
99.3
11.3
99.6
11.8
99.0
10
17.0
85.0
17.6
86.0
16.7
86.1
100
98.0
13.6
98.1
97.6
19.8
74.1
20.2
74.3
20.5
74.8
150
16.7
96.4
15.8
97.3
16.5
97.2
50
23.0
67.9
22.8
65.7
22.6
66.7
200
19.3
93.0
20.2
93.8
22.3
95.0
23.9
90
26.6
56.5
27.1
57.1
26.2
56.0
300
25.7
92.1
33.0
73.2
26.2
85.8
19

---

Under review as a conference paper at ICLR 2021
Table 12: Comparison of natural risk (% over 3 random trials) and hypocritical risk (%dstd over random trials) between methods on real-world datasets. Attacks are bounded by lec norm.
(a) On MNIST (c = 0.2).
(b) On CIFAR-10 1/255).
Method
Root
Rhyp
Method
Rnat
R Rhyp
Standard
0.6 0.1
100.0
0.0
Standard
4.9 0.2
99.3
0.3
Madry
0.9 +
93.8
1.5
Madry
5.3 -
62.6 +
TRADES (A = 240)
9.3 + 0.6
30.6
5.1
TRADES (A = 150)
10.1
0.1
30,0 + 1.7
THRM (A = 5000)
9.2 3.1
22.6 + 6.0
THRM (A = 1000)
10.1 - 0.3
31.9 1 1.4
(c) On CIFAR-I0 (c 2/255).
(d) On CIFAR-10 8/255).
Method
Rnat
Raya
Method
Reat
Rhyp
Standard
4.8 + 0.2
100
0.0
Standard
4.7 1 0.3
100.0 + 0.0
Madry
0.2
74.3
Madry
14.2 0.1
1 1.0
TRADES (A = 150)
13.6 1 0.2
37.2
2.2
TRADES (A = 50)
22.8
1 0.2
66.8 1 1.1
THRM (A = 1000)
14.0 + 0.2
42.3 = 0.0
THRM (A = 250)
24.2 = 0.4
90.0 + 3.4
Evaluated results natural risk (%) and adversarial risk (%) on MNIST. Attacks are bounded by no norm with 0.2.
Table 13:
(a) For TRADES.
(b) For THRM.
Trial
Trial 2
Trial
Trial
Trial
Trial
R
Rdv
Raxt
Ravv
Rus
Rathy
Reat
Ras
Rest
Rade
Rus
Radh
0.1
0.7
5.9
0.9
6.9
0.9
6.2
0.5
100.0
0.6
99.8
0.7
100.0
0.8
5.2
0.7
100.0
5
0.8
4.4
1.0
5.4
1.1
5.0
100
0.7
92.2
0.7
94.8
0.7
91.7
20
1.5
3.6
1.4
4.7
1.3
4.3
400
1.0
67.1
1.2
$9.1
0.9
82.7
40
2.7
2.7
3.2
2.6
3,2
600
2.1
39.9
37.2
1.2
73.6
60
4,4
4.5
3.0
3.5
2.7
35.3
28.8
2.4
61.0
80
5.2
299
5.0
3.3
4.8
3,4
1000
3.3
23.6
3.1
24.7
2.3
$1.5
160
7.8
4.7
8.6
4.9
6.4
4.0
3000
7.3
31.4
14.0
30.0
7.5
29.4
7.7
240
9.9
4.7
8.8
5.2
9.1
4.7
5000
10.4
29.5
5.7
25.6
11.6
20.8
300
8.3
5.1
14.0
7.1
9.5
4.7
6000
14.1
53.4
16.1
21.1
9.5
23.9
500
11.4
14.0
5.8
11.5
5.6
7000
13.3
31.9
6.3
22.8
29.0
Table 14: Evaluated results of natural risk (%) and adversarial risk (%) on CIFAR-10. Attacks are bounded by loo norm with = 2/255.
(a) For TRADES.
(b) For THRM.
A
Trial
Trial 2
Trial 3
À
Trial I
Trial 2
Trial
R.s.
Rids
Rust
Rads
Run
Rui
Rus.
R
Rade
R adv
21.7
1
6.3
13.3
6.3
13.6
6.4
13.3
10
5.4
26,8
5.4
27.6
5.2
28.2
12.1
7.7
6.3
19.2
10
9.7
11.7
9.0
10.7
9.5
11.1
100
6.7
15.1
6.4
14.8
6.5
15.3
30
10.2
10.1
9.5
10.1
9.7
150
7.1
13,9
7.0
14.0
14.1
50
11.0
8.4
13.4
13.3
70
11.6
9.0
11.6
8.9
11.6
8.6
250
8.8
12.2
8.6
12.1
9.0
12.8
120
13.2
8.3
13.0
7.8
12.9
8.0
400
10.0
11.3
10.0
11.6
10.7
11.0
150
13.9
8.4
13.4
7.7
11.4
11.2
10.4
180
14.1
14.2
7.4
14.4
7.8
800
12.5
10.5
12.9
10.7
13.3
10.3
210
14.7
14.6
7.5
14.5
7.8
1000
14.2
10.7
13.8
10.5
14.0
10.1
20

---

Under review as a conference paper at ICLR 2021
A.4 DETAILS IN SECTION
Hypocritical attacks here are executed by 50 steps PGD (step size is e/10) on source models. Note that the optimization method used here not pursue state-of-the-art transferability, but to examine the transferability of hypocritical examples. There are many methods designed to improve the transferability of adversarial examples may be extended to hypocritical examples (Liu et al., 2017 Dong et al. 2018 Wu et al. (2200a. Figure 7 shows the transferability heatmap of hypocritical attack over models trained on MNIST. Figure shows the transferability heatmap of hypocritical attack over 7 models trained on CIFAR-10. The value in the i-th row and j-th cloumn of each heatmap matrix
the proportion of the hypocritical examples successfully transferred target model j out of all hypocritical examples generated by source model i (including both successful and failed attacks on the source model).
Success rato (S)
No ve (YLP)
100.0
28.8
16.1
0.3
12.3
11.4
Stunders (MLP)
1.0
99.5
51.8
0.0
47.4
27.3
4.0
Kalve ekkrt)
Standard (Lewer)
3.4
43.4
0.4
100.0
0.1
93.0
51.1
8.7
2.4
90
Merve (Letee1))
66.9
Ghodertt (illee))
3.4
41.5
0.4
94.6
0.2
100.0
45.5
9.0
2.0
40
Viddy (Labet)
90.9
20
TRADES (Lahiet)
3.2
29.3
0.3
41.1
0.3
35.1
29.5
23.1
2.3
(LaNut)
Nav (4.3)
Sundert
Sandde (ohee
Nare (abaet
TRGES (AaNe)))
Target
Figure 7: Transferability of hypocritical examples on MNIST. Attacks are bounded by loo norm with 0.2. "(LeNet*)" means that it is the same architecture with "(LeNet)" but different random initialization.
100
Molve (I
19.9
3.3
Sowvarrd (91
1.0
79.3
0.0
27.0
16.6
12.1
14.8
Marve (Alide Redive
0.8
1.2
2.1
22.5
4.3
3.4
3.6
100.0
4.6
Madry (Wide Rediee
56.9
92.5
32.6
TRACES (iide Reslet
0.7
10.4
0.0
49.1
58.1
67.1
39.4
THEM (Aide Reahvet
1.0
21.6
0.0
74.8
39.4
22.0
85.5
Surcars (WNCe NasMetl
Modyy (WNS ReMMe
Target Model
Figure 8: Transfcrability of hypocritical examples on CIFAR-10. Attacks are bounded by 4. norm with 8/255.
21

---

Under review as a conference paper at ICLR 2021
B PROOFS OF MAIN RESULTS
In this section, we provide the proofs of our main results.
B.1 A PROOF OF PROPOSITION I
Proposition Denote the adversarial risk on correctly classified examples by
Rad
E
max at (x)
1(f(xr') + v)
(a.y))y)
then we have
Radv (f) = (fa + (1 - Rna((f))Radi (f).
Proof.
1(f(x') # (x)
Rad
= E
max
(a.g)m/
E (a.0)~
(f(xz) v)
max 1([(xe') + &
o' (m)
E (2.1)~D
1(f(x) Fy)
max 1(f(x') $ $3)
of (.))
1f(x') # w)
(f)
max (1)
(
max 1(f(xx) + w)
+ (1 - (f)
(x.8)~
w (B)
1(f(=') + v)
(f)
[1((x) + v)I + (1 (f)) E
max
(2.8)~077
(00.))mss}
x (m)
=Raa((]) + (1 R/aa(()))
E
1(f(x') + 41 (m)
max
(0,,)~=;}
=faat(f) + (f)
B.2 A PROOF OF THEOREM 1
Theorem 1. For any data distribution D and its corresponding conditional distribution on misclas- sified examples S w.r.t. a classifier f. we have
E ((((F)))) = v) < (m.y)~s
E
1(f(3)ny)) # f(x))
E
(I(rev)) # (x)),
(m.
(x.g)~~;
Rm)())
Reyp(()
Rmn(f)
where Bhyp = arg max 1 (f(x')= = y) and Drev
= arg max 1f((x') + f(x))) (x)
xt (
Proof. To prove the first inequality, we have
Rf)
E
max 1(f(x') v)
(7
w (()
E
1 (2( = y)
(0.
E (m.
1 (((w))))) # f(x)),
22

---

Under review as a conference paper at ICLR 2021
where the above inequality involves two conditions:
1 = 1 ([hwp))) + f(x)),
if (2hyp) 1/-
1 (f = y) o 1 (hypp) # f(z)),
if (ahyp) # y.
To prove the second inequality, we have
Rhyp (f) = E
(1f(2)))) f(x))
(#.w)~o}
S E 1 ((e( + f(x)).
(a.))~s;
Since (ae y) ~ Sj. we have f() 4 y. If there exists Zhyp such that (2ivp) = then ((hhyp) of f(x).. Now let Frev Thyp: then S [dTrev # f(x) is true. Otherwise, if we couldn't find Phyp such that (ahpp) = y, there still exists a posibility to find a rev such that f((rrev) # y but I (TTre) of f(x is true. Therefore, the above inequalities holds.
B.3 A PROOF OF PROPOSITION 2
Proposition 2. (se = (1 - Rnat (f)) (f) + Roat (F)ypl)))) ((( # f(x)).
(m.
Proof.
=(1 (f) =(1 - -Rnan (f) + Rnat [1(f(I + y)] + (fa
(f)
RaatRR (f))
E.
E [1(f( + f(x))]
(2.,))u0}
(a.,))>s
E
[1((()) = y) (f # y)l +
E
[1((() + v) 1 (Z)r))) + f(x))
(x.
(a.
E (..0)~
[(((x)) = -y) max x (x)
1([(x') v)
+
E
1(f(x) + y)
max 1(f(x') + f(x)))
(a.n)~
x'EB. (x)
E (x
1(f(x) = y) max 1(f(x') # f(x)) x.)
+
E (a.))DD
((((x) + v)
max 1(f(x') + S(a))
(x)
E
max 1(f(x') + f(+)) (m)
(=
x'
E (m
(f * f(x)))
C ADDITIONAL DISCUSSIONS
We showed that correctly classified examples (hypocritical examples) could be easily found in the vicinity of misclassified clean examples. As result, a hypocritically perturbed set could be con- structed with these hypocritical examples. The victim model's standard accuracy evaluated on the hypocritically perturbed set becomes higher than that on the clean set. It is natural then to wonder: How about adversarially robust accuracy (i.e., accuracy under adversarial perturbations) of the victim model on hypocritical examples? It's easy to see that, if the adversary bounded by the same c ball as the false friend, the model's 's adversarial accuracy evaluated on hypocritically perturbed set is zero, since a misclassified example exists in the ball of a hypocritical example (by definition). However, if the adversary's 's power is restricted by another ó ball such that ó < E, then robust hypocritical example may exist in the vicinity of a clean example so that a 5-bounded adversary can not change the model's 's prediction on the robust hypocritical example. In such case, the model's 's adversarial accuracy evaluated on the robustly hypocritically perturbed set could be higher than that on the clean set. New attack and defense methods are required to further explore this phenomenon.
23

---

Under review as a conference paper at ICLR 2021
32
Positives
Model decision boundary
True Positives
Oracle decision boundary
Correctly classified region
Misclassified region
Non-robust region
Correctly classified non-robust region
False Negatives
Misclassified non-robust region
True Negatives
I1
Figure 9: A visualization of the toy example to illustrate the phenomenon of the tradeoff between adversarial and hypocritical risks. Oracle decision boundary is the circle given by Equation Model decision boundary the line given by Equation I0 with the threshold green shadow region denote that the points in there misclassified and correctly classified by the model, respectively. The gray lined region denotes that the points in there can be perturbed with little perturbations to reverse the prediction of the model.
0.5. Red shadow and
D TRADEOFF BETWEEN ADVERSARIAL AND HYPOCRITICAL RISKS
Despite the experiments in Section 4.1 and Appendix A.3 showed that, when dealing with finite sam- ple size and finite-time gradient-descent trained classifiers, there may be mutual benefits between adversarial robustness and hypocritical robustness in real-world datasets, we note that in general, this synergism does not necessarily exist. We illustrate the phenomenon by providing another toy example here, which inspired by the precision-recet tradeoff [uccklnd & Gey (1994); Alvarez 20022)
Consider the case (a, y) E R [-1,+1] from a distribution D, where the marginal distribution over the instance space is uniform distribution over [0, 1J2. We assume that the decision boundary of the oracle (ground truth) a circle:
(ax) sign - ||:r cl|2),
where the centre e (0.5, T.5) and the radius as belonging to the positive class, otherwise they are labeled as belonging to the negative class. We consider the linear classifier with fixed w = (0, 1)T and a tunable threshold b:
0.4. The points inside the circle are labeled
f(æ) sign(w (w b) = sign(rz b).
(10)
See Figure for visualization of the oracle and the linear classifier over the instance space. this problem, we can show the tradeoffs by tuning the threshold of the linear classifier over [0, 1]. The precision is the number of true positives (i.e. the number of examples correctly classified as positive class) divided by the the sum of true positives and false positives (i.e. the number of examples misclassified as positive class). The recall the number of true positives divided by the sum of true positives and false negatives (i.e. the number of examples misclassified as negative class). We compare the adversarial risk correctly classified examples Raav(] defined in Proposition | and the hypocritical risk misclassified examples Rf) defined in Definition
The computing
24

---

Under review as a conference paper at ICLR 2021
Precision
Recall
Radv
Ray
Figure 10: Visualization of the computing formulas of precision, recall, adversarial risk, and hypo- critical risk in the toy example. These values can be viewed as the proportion of the areas of different regions.
Precision
02
Natural risk
Recall
Adversarial risk correctly classified examples
Precision or recall (%)
Hypociiiical misclassified examples
Risk
40
62
06
1.0
02
Threshold value
Threshold value
Figure 1I: The tradeoff' between precision and recall in the toy example.
Figure 12: The tradeoff between adversarial and hypocritical risks in the toy example.
formulas of these values are visualized in Figure 10
Here we choose the bounded la ball (xr)) (a' E IR? ||xo æ||2 < }} with 0.1 as the threat model.
Figure 11 plots the curve of precision and recall versus threshold b. We can see that there is obvious precision-recall tradeoff' between the two gray dotted lines. Similarly, Figure plots the curve of Radv (f)) and Rhyp (f) versus threshold We can see that the tradcoff exists almost everywhere: as the adversarial risk increases, the hypocritical risk decreases, and vise versa.
25


Reviews:
[{"review_id": "wUT5NFpxre", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary\nThis paper presents a new kind of adversarial attacks, named hypocritical attack. It is a reverse version of the original adversarial attack. It tricks a model into classifying data correctly with a perturbation. This can be a problem since it can make people satisfy the model performance, but the model is not robust on the real test dataset. The authors review the adversarial attack and define the new hypocritical examples and risk. The authors also show the simple results why hypocritical attach is a critical issue by a Naive model that is initialized randomly. It shows high performance on the hypocritical examples but low on the clean test data. They also investigate the algorithms that improve model robustness, THRM, and TRADES. Experiments show a trade-off between original classification loss and hypocritical risks, and THRM is a tight upper bound against the TRADES.\n\nThe main strength of this paper is suggesting a sound and straightforward attack named hypocritical attack. I think almost machine learning (ML) researchers cheer with joy when they see the high-performance results on their model. But sometimes, we need to think about our mistakes or test data characteristics (i.e., easy test data). This paper suggests another consideration to the ML community. The authors also investigate minimization algorithms on the attack risk and show future directions such as transferability.\n\nI have several questions about this paper that I want to listen to the author's response.\n\n\nQuestions and Comments\nIs there any trade-off between adversarial risk and hypocritical risk like precision-recall trade-off?\nWhat if we apply the technique to make hypocritical examples in the training data? We can measure the training accuracy then we can make the examples from the training data. Do these examples boost the classifier?\nIt would be better to describe the toy example since it is hard to understand the text's experiment in this paper. Even this experiment is borrowed from Zhang et al. 2019, readers want to understand the experiment with this paper only.\nTypo: with large hypocritical hypocritical risk, while the optimal on page 7.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "jf4rPjEG16a", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "1. The premise of the paper is that the adversary can perturb the *test* set so that the model is shown to perform better that it really is capable of. And in Section 7 (Conclusion) the paper claims that it exposes this new risk. However, remember that this risk is already mitigated in practice by keeping the test data *independent* of the model/classifier (e.g., see Kaggle competitions where the test set is hidden). Therefore, the perceived risk is not even present. In the context that the technique has been introduced, it seems like the [malicious] actor would only be fooling him/her self rather than fooling the model/classifier.\n\nIt is hard to come up with an application for manipulating the *test* data in the manner proposed and I am curious to hear what the authors feel.\n\n2. What if instead of manipulating the images to conform to the classifier's predictions, we manipulate the labels, i.e., relabel the test data as per the classifier's predictions? Would that result in the same final objective being achieved?\n\n3. Equations 1 and 2: The technique is based on generating adversarial training examples and does not make any fundamental technical contributions.\n\n4. Abstract: \"Extensive experiments verify the theoretical results and the effectiveness of our proposed methods.\" -- Just a couple of standard simple image datasets (MNIST, CIFAR-10) have been employed. Harder and more number of datasets should be employed in order to justify the statement in the Abstract.\n\n5. Section 6 Discussion: \"What we can do at present is using a relatively more robust model as a surrogate of the true friend to improve the robustness of a weak model.\" -- It is not clear how that would help. The attacker might mislead the 'relatively more robust model' as well. Moreover, why not use the 'relatively more robust model' instead of the weaker model?\n\nOne simple mitigation strategy could be to digitally sign the test set to check for tampering. Why has that not been suggested?", "extra_scores": {}, "extra_text": {}}, {"review_id": "bvMARVU4LEE", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper considers attacks that flip the label for those data points that are incorrectly classified. It proposes a risk metric to capture this and uses prior attacks for this type of attack.\n\nPositive points:\nThe main idea in the paper is, at first look, interesting but then on more thinking I have doubts (see below):\n\nNegative points:\n- It is not very surprising that mis-classified data points can be perturbed to be properly classified using exactly the same kind of attack as prior work (not sure why authors call the attack conceptually different).\n- I think the work is missing an important comparison to R_bdy (and R_rob) in Zhang et al. Is that in Propositon 2? Why call it R_rev?\n- Also, why is equation (4) a natural risk to minimize? In Zhang et al., the term to minimize is nicely motivated by bounding R_rob - R^*_nat, which naturally leads to the tradeoff. I do not see how the tradeoff arises here.\n- Also, the fact that TRADES would help here is obvious, isn't it? As TRADES handles two sided errors, it naturally also takes care of the one side of the error the authors care about. And that naturally raises the question that will only caring about one side of the error (in Eq 4) increase the other side (R_adv)? This R_adv is not shown in experiments. Also, shouldn't the experiment be done with larger and more variety of datasets - particularly, when for more complex datasets TRADES is doing reasonably well - so might be better off using TRADES for complex and real image datasets.\n- A lot of the technically machinery is not very novel with same attack as before.\n\nMinor points:\nAs a writing style, false friends is a good analogy but please do not use it throughout the paper in place of technical terms such as this line \"Better transferability is beneficial for black-box attacks but is not always desired by false friends\".\n\n----------------Post rebuttal-----------\nThe issues I raised still persist in my mind. Also, other reviewers have similar issues and also more issues other than what I point out, which seem valid issues to me. I will keep my score as is.", "extra_scores": {}, "extra_text": {}}, {"review_id": "GJB6Cq0p2aL", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "#### Problem Statement\n\nThe research on adversarial examples typically focuses on labels which are correct but non-robust: eg. an image of a bus that is classified correctly but with a few changes to pixels can be misclassified. This paper suggests that we should also pay attention to robustness in misclassification. If a classifier is not robust, even on images where it makes a mistake, changing a few pixels might fool the model into giving it the correct label, but presumably  for the wrong reasons. \n\nThe authors envision a scenario where a \"false friend\" examines the output of a network, and attacks the mislabeled examples to make it look like they are labeled correctly. In such a scenario, not much training happens even though the network is far from optimal on noiseless examples. They suggest certain mitigations against such attacks by favoring networks whose labels on mistakes are robust. \n\nThey propose a new objective function called THM and compare it to TRADES. \n\n#### Pros\n\nThe paper makes a good point that (non)robustness for classifications sometimes goes hand in hand with (non)robustness for misclassifications.  \n\n#### Cons\n\n* The attack scenario which the paper envisions does not strike me as very realistic. \n\n* From what I can tell, TRADES would aim for robustness on all examples, whether rightly or wrongly misclassified, whereas THRM seems to focus only on robustness for misclassified instances.    It appears that adversarial risk on correctly classified examples is not being weighted. This might be a better choice for the task of avoiding the type of attacks proposed in this paper,  but overall I am not convinced it is a better objective than THRM, which aims for robustness everywhere. \n\n* One could imagine that in the real world, when one is not concerned about false friend attacks, one would like robustness for the correct labels, but perhaps not so much for the wrong labels, as this could be a sign that the network is unsure of its prediction, and could change its output, given more data or training. \n\n#### Summary of recommendation\n\nThe observation that non-robustness applies to correct and wrong labels is interesting. But I didn't find the attack scenario built on top of this observation and the proposed mitigations compelling. \n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper the flip-side of an adversarial "attack" in that data may be perturbed to make it look like a model was performing well rather than the standard notion of adversarial attacks. The reviewers found this notion interesting and potentially worthy of investigation. However as it stands, the proposed applications and methods do not seem developed well enough as would be expected at a conference like this.
Decision: Reject