Does the Half Adversarial Robustness Represent the Whole? It Depends... A Theoretical Perspective of Subnetwork Robustness
Authors: Jovon Craig
Joshua Andle
Theodore Stein Nowak
Salimeh Yasaei Sekeh
Venue: Submitted to ICLR 2023 (2023)
Keywords: Adversarial Learning
Adversarial Robustness
Subnetworks
Semirobustness
Information-Theoretic Measures
Mutual Dependency

Abstract: Adversarial robustness of deep neural networks has been studied extensively and can bring security against adversarial attacks/examples. However, adversarially robust training approaches require a training mechanism on the entire deep network which can come at the cost of efficiency and computational complexity such as runtime. As a pilot study, we develop in this paper a novel theoretical framework that aims to answer the question of how can we make a whole model robust to adversarial examples by making part of a model robust? Toward promoting subnetwork robustness, we propose for the first time a new concept of semirobustness, which indicates adversarial robustness of a part of the network. We provide a theoretical analysis to show that if a subnetwork is robust and highly correlated with the rest of the network, then the remaining layers are also guaranteed to be robust. To guide the empirical investigation of our theoretical findings, we implemented our method at multiple layer depths and across multiple common image classification datasets. Experiments demonstrate that our method, with sufficient dependency between subnetworks, successfully utilizes subnetwork robustness to match fully-robust models' performance across AlexNet, VGG16, and ResNet50 benchmarks, for attack types FGSM, I-FGSM, PGD, and C&W.

Under review as a conference paper at ICLR 2023
DOES THE HALF ADVERSARIAL ROBUSTNESS REPRE- SENT THE WHOLE? IT DEPENDS ... A THEORETICAL PERSPECTIVE OF SUBNETWORK ROBUSTNESS
Anonymous authors Paper under double-blind review
ABSTRACT
Adversarial robustness of deep neural networks has been studied extensively and can bring security against adversarial attacks/examples. However, adversarially robust training approaches require training mechanism on the entire deep network which can come at the cost of efficiency and computational complexity such as runtime. As a pilot study, develop in this paper novel theoretical framework that aims to answer the question of how can we make whole model robust to adver- sarial examples by making part of a model robust? Toward promoting subnetwork robustness, we propose for the first time new concept of semirobustress, which indicates adversarial robustness of part of the network. We provide a theoretical analysis to show that if a subnetwork is robust and highly dependent to the rest of the network, then the remaining layers are also guaranteed to robust. To guide the empirical investigation of our theoretical findings, we implemented our method at multiple layer depths and across multiple common image classification datasets. Experiments demonstrate that our method, with sufficient dependency between subnetworks, successfully utilizes subnetwork robustness to match fully- fully-robust models' performance across AlexNet, VGG16, and ResNet50 benchmarks, for attack types FGSM, I-FGSM, PGD, C, and AutoAttack.
INTRODUCTION
Deep neural networks (DNNs) have been highly successful in computer vision, particularly in image classification tasks, speech recognition, and natural language processing where they can often outperform human abilities Mnih et al. (2015): Radford et al. (213)); Goodfellow et al. (016)). Despite this, the reliability deep learning algorithms is fundamentally challenged by the existence of the phenomenon of "adversarial examples", which are typically natural images that are perturbed with random noise such that the networks misclassify them. In the context of image classification an extremely small perturbation can change the label of correctly classified image Szegedy et al.) (2014); Goodfellow et al. (2014). For this reason, adversarial examples present a major threat to the security of deep-learning systems; however, a robust classifier can correctly label adversarially perturbed images. For example, an adversary could alter images of the road to fool a self-driving car's neural network into misclassifying traffic signs Papernot al. (2016a). reducing the car's safety, but a robust network would detect and reject the adversarial inputs Ma et al. (2018); Biggio et al. (2013). The problem of finding perturbed inputs, known as adversarial attacks, has been studied extensively Kurakin et al. (2017): Sharif et al. (2016); Brown et al. (2017) Eykholt et al. 2018).
To handle adversarial attacks, two major solutions have been studied: (1) Efficient methods find adversarial examples Su et al. (2019) : Laidlaw & Feizi (2019); Athalye et al. (2018): [iu et al. (2016)
Xie et al. (2017]: Akhtar Mian (2018), 3), (2) Adversarial training to make deep neural networks more
robust against adversarial attacks Madry et al. (2018; Tsipras et al. (2019); Gilmer al. 2011 Ilyas
ctt al. 2019); Papernot et al. (2016b).
The adversarial perturbations may be applied to the input to the network's hidden layers et al. (2014); Szegedy et al. (2014) and it has been show that this strategy is effective at improving
network's 's robustness Goodffllow et al. (2014). Several theories have been developed to explain
the phenomenon of adversarial examples Raghunathan et al. (2018): Xiao et al. (2019)/Cohen
Cohen et al.
(20T)); (hamir et al. (209)) Fawzi et al. (2rlii & Wagner (017
Weng et al. (2018) Ma

---

Under review as a conference paper at ICLR 2023
et al. (018). Previously Ilyas et al. (2019) investigated adversarial robustness from theoretical perspective. The authors address "useful, non non-robust features": useful because they help a network improve its accuracy, and non-robust because they imperceptible to humans and thus not intended to be used for classification. Normally, model considers robust features to he about as important as non-robust ones, yet adversarial examples encourage it to rely on only non-robust features. [[ayas et al.) (2019) introduces a framework to explain the phenomenon of adversarial vulnerability A feature is considered "p-useful feature" if is correlated with the true label in the dataset. Similarly, "y-robustly useful features" are p-useful for set of adversarial perturbations. While Ilyas et al. (2019) constitutes fundamental advance in the theoretical understanding of adversarial examples, and opens the way to thorough theoretical characterization of the relation between network architecture and robustness to adversarial perturbations, little attention has been paid to how robustness throughout the network is guaranteed and whether adversarial training must be applied to the entire network.
In this paper, we develop a new theoretical framework that monitors the robustness across the layers in DNN and explains that if the early layers are adversarially trained and are sufficiently connected with the rest the network, then adversarial robustness of the latter layers is obtained, here by connectivity we mean the early layers are highly dependent to the latter layers. All of these findings raise a fundamental question: How can we make a whole model robust to adversarial inputs by making part of the model robust? In addition, the vulnerability of models trained using standard methods to adversarial perturbations makes it clear that the paradigm of adversarially robust learning different from the classic learning setting, In particular, we already know that robustness comes at the cost of computationally expensive training methods (more training time) Zhang et al. (2019),
as well as the potential need for more training data and memory capacity (Schmidt et al. (018))
Hence, one notable challenge in adversarially robust learning is computational complexity while maintaining desired performance. To this end, by exploiting the possibility that subnetworks can be robust to adversarial attacks, we propose a novel approach that aims to theoretically analyze adversarial robustness guarantees in a network by adversarially training only subset of layers. This work will also pioncer the new concept of "semirobustness" which indicates adversarial robustness of a part of the network. This includes a new perspective of adversarial perturbations and a novel theoretical framework that explains theories for the following claim:
Ifa subnetwork is robust and highly dependent to the rest of the network and passes sufficient connectivity towardd the last leyer, then the remaining layers are also guaranteed to be robust.
Contributions
To summarize, our contributions in this paper are: (1) We introduce novel concept of semirobustness in subnetworks. We show that a subnetwork is semirobust if and only if all layers within the subnetwork are semirobust. (2) For the first time we provide theoretical framework and prove that under some assumptions if the first part of the network is semirobust then the second part of the network's robustness is guaranteed. (3) Experimentally,
demonstrate that given sufficient mutual dependency between subnetworks, our method displays the same adversarial robustness of a network as compared to regular adversarial training.
SUBNETWORK ROBUSTNESS
n layers where, F(") is a function mapping Notations We assume that a given DNN has a total of the input space X' a set of classes y, iee. F()) X' + y f(() is the I-th layer of F(n); F(i5) = f(5) o ... f()) is a subnetwork which is a group of consecutive layers fit), ,..,,.., f6); f6);
F() F(.)) ,() o... (1) is the first part of the network up to layer j. Denote g(0) the activation function in layer / and TTy) the prior probability of class label y E V. Let f(() be the 1-th layer of F("), (x)-1) o (w0000)) + b(l),, where o()) is the activation function. In this section, we define the notion of a Semirobust Subnetwork. We discuss semirobustness more in Section 2.1
Definition 1 (Semirobust Subnetwork) Suppose input X. and label y are samples from joint distri- bution D. For a given distribution D, a subnetwork F(3 is called 7 semirrobtttt if there exists mapping function G, L, r y such that
E Ex inf y. G. F - 6)] 2 7/
(I)
BES
2

---

Under review as a conference paper at ICLR 2023
for appropriately defined set of perturbations Sz. In 17 G3 is non-unique function mapping layer f()) to class set V, and "(j is constant denoting the correlation between y and F())
Note that Gj is necessary if the dimensionality of FDD does not match that of y. but if FG) = F(n) the semirobust definition becomes standard 7-robustness defined in Iyas et al. (209)). To define semirobustness for single layer f() in (1) we simply replace f()) in FG) and Kj-1 (X + 6) in X + 6, where K is mapping function K X' + Lj-1- In this paper to avoid confusion, we use + for layer semirobustness as input as well. Throughout this paper, we assume that the network F(a) is useful network i.e. for given distribution D, the correlation between F and true label () y (())(X)) is highest in expectation in optimal performance. Intuitively, highly useful network F(n) minimizes the classification loss E(x,))<D| ((X,,)) that
- -E V (b+
Σ
WF( F( (
(2)
F())
where WFr() the weight vector and [[n] is the set of n-th layer networks. Definition raises valid questions regarding the relationship between subnetwork and its associated layers' robustness. We show this relationship under the following thoerem.
Theorem The subnetwork FG) is Is-semirobust if and only if every layer of FU), i.e. ,6), /o.1))...,,J is also semirobust with bound parameters "ij.-..>7!! respectively.
Theorem D is key point used to support our main claims on the relationship between layer-wise and subnetwork robustness, and its proof is provided as supplementary materials (SM). Next, we show that under strong dependency assumption between layers the robustness of subnetworks are guaranteed.
2.1 SEMIROBUSTNESS GUARANTEES
In this section, we provide theoretical analysis to explain how dependency between layers of subnet- works promotes semirobustness and eliminates the entire-network adversarial training requirement.
Non-linear Probabilistic Dependency (Mutual Information): Among various probabilistic depen- measure called mutual information
dency measures, this paper, we adopt an information-thewn (MI): a measure of the reduction in uncertainty about one random variable by knowing about another. Formally, it defined as follows: Let X' and Z Euclidean spaces, and let Pxz probability measure in the space X' x Z. Here, Px and Pz define the marginal probability measures. The mutual information (MI), denoted by I [(X; 2), is defined as,
dP dPx2 dPx Pz
((X;Y) = E Px
9
where dPxz- is the Radon-Nikodym derivative, (0, oc) + R is convex function, and g(1) 0. Px dPxy Note that when PPxPy + 1, then -+ Using 3 the MI measure between two layers f(() and f(i with joint distribution P. and marginal distributions P., Pj respectively is given
dP d...P.
E P.P,
9
The concept of MI is integral to the most important theory in theoretical framework through the assumptions below.
Assumptions: Let G. : C. + y be a function mapping layer f(a) to label y E V, and let
Gy C, + y be a function mapping layer f(5) to a label y y. Let gs f f()) (x + 5) and
hg.j = fX) + +)) for 8 E S. (perturbation set). Note that gs = -hsa--
A1: The class-conditional MI between ha.j-1 and hss is at least hyperparameter P5 2 0, i.c.
Σ n (v) (ho.5-1: h6. 2 P3
(5)
3

---

Under review as a conference paper at ICLR 2023
A2: There exists constant Uj 2 0 such that for all 5 E S:
p p(hs.5-1, h1, (y) (p(bag--lu)o(has)
E Bp(hsa-nihssi
- U, and
ly- (G3 ho.j G5-1 hā.j 2 + J,
where (1, y) is the joint probability of random triple (hs.j-1, A.....1 y))
Theorem 2 Let fo be -e-semirobust subnetwork equivalent to F(o), and let fo be the subnetwork F(a+1.n) and for ) = a ++,..... ......I.. n. assumptions A1 and A2 holds true. Then fo -y-semirobust.
In Theorem 76 S Ja + +Ejoat Pj. Note that the constant Uj does not depend on "a. Tbe and pj- This theorem is an extension of the following lemma, and the proofs of both are found in the SM.
Lemma Let p(n-1) be -n-1--mmirobuut subnetwork. Let g8 f( + 8) and hi f(a for δ E Sg- Let Gn-1 fn-1 + y be a fanction mapping layer g the network's output y E y. Under the following assumptions f(n) is -emmirrrobs:::
B1: The MI between f(x-1) and f((n) is at least hyperparameter 2 0, i.e.
Σ T (95 holu) 2
9
B2: There exists a constant U 2 such that for all E S:
plgs,hsly) ((gsl))p((hs)
] S U, and Eplgeshe.)) ly (hs Gn-1 o g6)) 2 1
Note that in Lemma A2, when a n Intuition:
I'm S 7 + and assumptions B1 and B2 are particular cases of A1 and
Let IF(() determine the information flow passing through layers in the network F(x), Intuitions from the I_F literature would advocate that information is preserved up to a given layer, one can utilize knowledge of this information flow in the next consecutive layer's learning process due to principle F(iJ) f() F((J-1), and consequently IF((,) = IF() o ( This is desirable as in practice training the subnetwork requires less computation and memory usage. This explains that under the assumption of the strong connection between j-th and j 1-th layers, the information automatically passes throughout the later layers, and subnetwork training returns sufficient solutions for task decision- makinn.. To better characterize the measure of information flow, employ non-linear& and probabilistic dependency measure that determines the mutual relationship between layers and how much one layer tells us about the other 2 (and Lemma 1 is that strong non-linear mutual
seed-forward network the learning
one. An important takeaway from Theorem connectivity between subnetworks guarantees that securing only the robustness of the first subnetwork ensures information flow throughout the entire network.
Linear Connectivity:
To provably show that our theoretical study in Theorem 2 is satisfied for the linear connectivity assumption between subnetworks, we provide a theory that investigates the scenario when the layers in the second half of the network are linear combination of the layers in the first subnetwork.
Let Ja be semirrobbss subnetwork equivalent to F(a), and let fo be the subnetwork Theorem 3
p(a+1.), ffr j a aa+1 ........,,...................... 1,..., r()= = 1 where Aiÿ is map Li + L.j and matrix of an ()/a))
dimensionality L; L,j then fo is "u-semirobust where % = =7a (
This theorem shows that when the connectivity between layers in fa and fo is linear, we achieve the semirobustness property for the subnetwork fú. Importantly, note that linear combination multipliers determine the Pearson correlation between layers given the constant variance of the layers. This is because if f6) = f then Corr(f() f()) = )(year(f(i), Theorem 3 is an extension of the lemma 2 Detailed proof and accompanying experiments provided in the SM.
4

---

Under review as a conference paper at ICLR 2023
Lemma 2 Let the last layer f(() be linear combination of f(x=1),...,, ......f)), expressed as f(
X f where X is map Li > f.n and matrix of dimensionality Li x L. If F(n-1) is
y-semirobust, then f(r) is "n-semiiobuust where "nn
%
At this point, valid argument could be how the performance of a network differs under optimal full-network robustness, [g) and subnetwork robustness (J fo). Does the difference between performance have any relationship with the weight difference of subnetworks f and f&? This question is investigated in the next section by analyzing the difference between loss function of the networks (fa+ ft) and (fa fo).
Question:
2.2 FURTHER THEORETICAL INSIGHTS
Let w be the convergent parameters after training has been finished for the network F=(n) (fā. ft), that is adversarially robust against a given attack. Let W+ be the convergent parameters for network (Ja Jb), that is adversarially semirobust against the attack. This means that only the first half of the network is robust against attacks. Let ws. Wy, and w be weights of networks f fo. and fa, respectively, Recall the loss function (2). and remove offset b without loss of generality.
Define [(w) :- Σ FEJ
UF P(n) (X),
(X),
(6)
therefore the loss function in ② becomes (LX,)) (x), Y)} E(x,Y) (Y [(w)) and {Y
w² := argminn,
(l(w))}, where E is defined in (6)
Definition (Performance Difference) Suppose input X and task Y have joins distribution DD Let ((m) == fo) E F be the network with n layers when the subnetwork fa is semirobust. The performance difference between robust F=(n) == ft) and semirobust F(n) is defined as
(0r((),F(tn)) == EXXYY~D ((F (X),)) - ((F(()) (X).))
(7)
Let ((") = flw²) - -((()). The performance difference is the average 5: d( ((F+(n), F(m))= = E(XY)~ (Y- 5(w" ())][=]]]-- E E(XY)> [Y (e(w") (0(=-(())) ))]
(8)
Using Taylor approximation of l around w ((75) ~ ((ww) + (3 w -"""- ( 1 ww"),
(9)
where Ve(w") and (2((w") are gradient and Hessian for loss l at w". Since wt is the convergent points (fa f7). then Vl(w*) = 0, this implies
((3 f(ww) (w w (lw - w-) ma w011?
(10)
where maz is the maximum eigenvalue of 2 ((ww) In 11 write | w 112 10ح w 11?
holds because ~ht = (wa, (.) and w = (wa, w Note that here the weight matrices wt and 25* are reshaped. Using the loss function (x {Y {(w)), we have
1
E EXXYYDD ( - [(w"))}
Exx.r (Y- (Amar wi 1²))
(11)
This explains that the performance difference (8) between networks F=(1)) and F(n) is upper bounded by the L2 norm of weight difference of f and fo i.e. We wg. Alternatively, using Cauchy-Schwarz
Schwarz inequality, we have
E E(x,y>~ (Y-()())) ((w"))) E E(x.v (11lf(")((; (x;
(12)
5

---

Under review as a conference paper at ICLR 2023
where f(() is the last layer of the network. Recall (8) from Lee al. (0021) As Wt and w" are the weights of network (fa fE) and (fa fo), we have
Ilf (m) x)) f(²) (x: w " & |ا - wollello (fa(x, w.)) 1l2.
(13)
next, we assume the activation function o is Lipschitz continous i.e. for any u and v there exist constant C s.t. [o - [v)] C I v/. Next, assume the activation function is satisfied in
(0)) = 0. Further by assuming that ||x|| is bounded by Cz and by using peeling procedure, we get:
1x;"")(;;") - ((r) (x; " ))2 S Cx,rllig - -w II
(1000)) |F,
(14)
jEa
here (() the weight matrix of layer j-th in fa and Cx = CxCa. Combining provide the upper bound:
15) and (4) we
EX {Y- (e((²) {(w"))] E(XY)) (wa) (1
(15)
where (x,o Cx.a IIjea || ||.. This alternative approach validates the result shown in (TT) and aligns with the conclusion that the performance difference between robust and semirobust networks is highly related to their weight differences. In this section we proved two bounds for performance difference defined in (8)
EXPERIMENTS AND ANALYSES
To confirm our theoretical findings and experimentally validate Theorems 1 3 we test our method multiple layer depths, and across multiple common image classification networks trained CIFAR- 10 Krizhevsky et al. (2009), CIFAR-100) Krizhevsky et al. (2009), and Imagenette Howard Deng
al. (2009) datasets.
3.1
EXPERIMENTAL SETUP
To guide the empirical investigation of our theoretical findings, we consider attack models, MI estimator, and adversarial training settings as follows.
The most common threat model used when generating adversarial examples is the threat model,
Attack Models:
additive threat model. Let X xxa) where each X; E X' is feature of X. In an additive
assume adversarial example X, (X, + 51, Xxa 52), i.e., X, X @ 8, X, = -X+ X + where = (ō1,.... .......,)) 52). Under this attack model, perceptual similarity is usually enforced by bound on the norm of 6, ||ō]| e. Note that a small is usually necessary because
otherwise, the noise on the input could be visible.
We some of the common additive attack models: the Fast Gradient Sign Method (FGSM) Goodfellow et al. (2014); Szegedy et al. (2014), iterative FGSM (I-GGSM) Kurakin et al.) (2017). [PGD]
Progressive Gradient Descent (PGD) Madry al. (2018), Carlini & Wagner (C))Carlin & Wagner
(2017), and AutoAttack Croce & Hein (2020). We use 255 255
& 16 and 32
255 For iterative attacks
we use e_step of 253 and number of iterations equal to menn (4
1.25 e) for 10, 20, and 36
iterations for the respective e values, as suggested by Kurakin et al. (2018). Attacks use an -nom with the exception of C&W, which uses L Lg-norm. Additional details can be found in the SM.
MI Estimation: We use a reduced-complexity MI estimator called the ensemble dependency graph estimator (EDGE) Noshad et al.] (2019). The estimator combines randomized locality-sensitive hashing (LSH), dependency graphs, and ensemble riauttin methods. We chose EDGE because has been shown that achieves optimal computational complexity (n)). where is the sample size. It is thus significantly faster than its plug-in competitors Kraskov et al. (2004); Moon et al. (2017): : Noshad et al. (2017). addition fast execution, EDGE has an optimal parametric MSE rate of O()n)) under specific condition.
Adversarial Training: Adversarial training is an approach to making models more robust to adversarial attacks by producing adversarial examples and inserting them into the training data. Given

---

Under review as a conference paper at ICLR 2023
adversarial examples in the original imput, we focus the min-max formulation of adversarial training that uses standard training on a classifier by minimizing a loss function that decreases with the correlation between the weighted combination of the features and the label Goodfellow et al.
2015): : Madry et al. (2018), ming (1.))) maxş Lo(x + 8.
3.2 LEARNING HYPERPARAMETER P
A key point in the claim of Theorem 2 is to determine the hyperparameter Pa+1 that bound the dependency between last layer in subnetwork fa := fx(a) and first layer in subnetwork fo := f((+1,n) and hyperparameters Pa+2: ..... Pn. that bound dependencies between consecutive layers in fa. Within the experimental results we denote these values as ........ Pa+11 where Pa corresponds to the last pair of layers in fo. We have devised a novel adversarial training algorithm to determine these p-values that learns hyperparameters and supports that subnetwork robustness guarantees network robustness.
This procedure labeled Algorithm I assumes that the mutual dependency between the two parts of a network F(x) is based on their MI mea- sure. To retrieve baseline results, this method first performs standard ("regular") training of the whole network with the original dataset, and then the same training is done with adversarial examples of that set. The network's two halves are denoted fa and So if regularly trained, or Ja and S if adversarially trained. In the next stage, the algorithm runs T trials, each of which does adversarial training on fo for E epochs while fa is frozen. The second part F(() after being trained for an epoch is labeled fa. Ideally, the current training accuracy of the net- work Acct should approach Acc" within a small value of k at which point the training current trial ends. Next, the class conditional MI, 1j.t = E,, (f)))( fO))y) {())) between each pair of consecutive layers from f(a) to f(x), is calculated. As the trials progress the largest testing accuracy achieved (Acc) is up- dated, along with the corresponding trials' I5₁ values (Po+1 pr)). After adversarial training ends, these results are reported for the trial which achieves the highest adversarial testing accuracy Acc. We provide the hyperparameter settings for Algorithm in the SM.
Algorithm 1 Learning Hyperparameter p
Do regular and adversarial training of F(*) as (fa- fo) and (fa- respectively
Store test accuracy of adversarial training (fa= as Ace
Set k to be as small possible
Initialize Pa+1...... Pn = ,.,...,00
for f = 1,
T do
Load fo: freeze f for e = I....,. do
Do one epoch of adversarial training of
fo to get fo
Store test accuracy of (fä, Fo) as Acef if -Acç k then
Break out of epoch loop and store Acc;
the
end end for
a aa+11..... do Compute 1,, given in (5) for all con-
secutive layers in (fa, fo). then store I5,
end end
Ace= largest Acc;
P) smallest Lj,t for j =a a I,...,
Report Patly..., Pn and Ace
3.3 PIECE-WISE ADVERSARIAL ROBUSTNESS GUARANTEED
The experimental results support our claims in Theorem 2 The tests span AlexNet, VGG16, and ResNet50 architectures on CIFAR- 10, CIFAR-100, and Imagenette datasets. As the network always undergoes the same procedure for standard training, the regular test accuracies are the same for all fe sizes. Theorem 2 is correct, then despite ft being frozen when training fo, the network should still be robust adversarial examples due to the mutual dependencies within it. We see in Table that 2% of Acc" across varying combinations
the fs network training frequently approaches within of networks and datasets. For this experiment the number of trainable (e.g. convolutional or linear) layers in fo varies by network with values of 4, 12, and 16, to ensure that fo comprises a large portion of the respective networks. For this table all data was attacked with AutoAttack using 255 We report the adversarial test accuracies of the fully robust model (Acc*), the semirobust network (Ja fo) denoted (Accsr), and the the network (f²; Jo) denoted Acc.
7

---

Under review as a conference paper at ICLR 2023
Table 1: Subnetwork training with AutoAttack on varying setups
Model
Dataset
fo layers
Acc
AcCar
Ace
Diff.
Pn
Pn-3
Pn-7
Pn-11 Pn-15
AlexNet
CIFARIO
4
64.7
19.7
64.5
-0.2
1.92
5.16
CIFAR100
4
33.6
16.9
32.3
-1.3
2.79
3.55
Imagenette
4
75.3
67.3
74.0
-1.2
2.17
6.24
VGG16
CIFARIO
12
79.0
63.9
76.6
-2.5
3.22
5.27
6.83
7.60
CIFAR100
12
54.2
38.5
51.8
-2.4
3.05
3.59
4.11
4.28
Imagenette
91.0
26.1
86.0
-5.1
2.35
6.82
7.07
7.11
ResNet50
CIFARIO
75.8
46.7
74.7
-1.1
3.22
5,93
6.69
6.70
6.48
CIFAR100
16
56.7
25.8
55.9
-0.8
3.26
3.92
4.14
3.91
3.95
Imagenette
16
89.2
9.5
82.3
-6.9
3.05
6.17
6.61
6.58
0.00
Value
CIFARIC CIFARICO Imagenette
6
10
12
14
10
15
falayer fjp-a)
layer f(r-a)
layer fyv-))
Figure 1: Connectivity values of layers in fo on multiple datasets at large relative sizes of fo
Guarantees for Multiple Layers Robustness: We report the behavior of across various sizes of for models, and datasets in Figs and 2 Both experiments were run data perturbed with AutoAttack 8 Starting from the output layer f(a), each prior layer of f(n-a) (where x is the x-axis using = 255 255 value) tends to show higher p values, leveling off at a certain depth. An exception for this tends to occur when training fo fails to converge, sometimes resulting in values close to 0 in the early layers of fo. This can be seen in Fig. [ for ResNet50 on Imagenette. The accompanying data table in the SM reflects that this particular run of ResNet50 failed to achieve an Ace similar to Acc". Such occurrences support the idea that sufficient Pat1 1 required to achieve subnetwork robustness of fo-
Effects of Dataset, Network, and Attack Type on In order to investigate the effects of dataset, network type, and attack type the observed p values, we ran series of experiments for Algorithm [ with certain hyperparameters held constant which are found in the SM along with additional analysis. We observe that attack type and network depth lack readily apparent trends with the values of p for each layer. We do observe a clear trend where the range of values of p obtained across the layers of fo is smallest for CIFAR-100 and largest for Imagenette.
Experimental Analysis We observe in our experiments that changes in the dataset impact the values of p. CIFAR-100 consistently reported the lowest values of p for given layer while Imagenette reported the highest, reflecting the network's 's accuracy on these datasets. A likely reason for this that for task which the network has accurately learned, it displays high MI between each layer to facilitate this high performance. Similarly, we show that for deeper layers in the network within
fo. p tends to take higher values. This may indicate that deeper networks provide a better flow of information which enables fa+1 to readily learn to utilize the features in fa- Our results indicate that when subnetwork training fails to reproduce Acc*, Pa+ is often Rs O, indicating that the network isn t properly learning to pass information from the subnetwork fö- We report no clear trends between and any of the attack types magnitudes used here. This, coupled with the frequent matching of performance when compared to Acc", indicates that our method largely orthogonal to each attack type, resulting in comparable performance while leveraging the robustness of the first subnetwork.
8

---

Under review as a conference paper at ICLR 2023
4 RELATED WORK
An important paper that studies adversarial ro- bustness from a theoretical perspective by Ilyas et al.] (2019), who claim that adversarial examples are "features" rather than bugs. The authors state that network's being vulnera- ble to adversarial attacks "is a direct result of [its] sensitivity to well-generalizing features in the data". Specifically, deep neural networks learning what they call "useful, non-robust features": useful because they help a network improve its accuracy, and non-robust because they are imperceptible to humans and thus not intended to be used for classification. Conse- quently, a model considers robust features to be about as important as non-robust ones, yet ad- versarial examples encourage it to rely on only non-robust features. [yas et al. (2019) introduces a framework to explain the phenomenon of adver- sarial vulnerability. Rather than focusing on proving a probabilistic close-for form solution to determine the minimal subnetwork which needs to be adversarially trained in order to confer full-network adversarial robustness.
P for Varying fo Size for Rasnet50
7.0
6.5
Value
5.5
5.0
4.5
3.5
fb layer f(n-s)
Figure 2: Connectivity values of ResNet50 on CIFAR-10 perturbed by AutoAttack
which features the model is learning, method's focus
More recently some attention has been given to the adversarially robust subnetworks through methods following the concept from Frankle & Carbin 2018) including Peng al. (2022) and Fu et al. (2021). Although these works are also interested in robust subnetworks, the focus is often more empirical, or focuses on the robustness of the subnetwork itself, rather than what we do which is to investigate how other subnetworks can benefit from that semirobustness. Applying the theory outlined here to such methods could provide an interesting avenue for Continual Learning, where robust subnetworks are sequentially identified and built up over series of tasks by incorporating the theory behind semirobustness.
5 CONCLUSION
Discussion We have introduced here the notion of semirobustness, when a part of a network is adver- sarially robust. The investigation of this characteristic has interesting applications both theoretically and empirically. We prove that if subnetwork is semirobust and its layers have a high dependency with later layers the second subnetwork robust. This has been proven under non-linear dependency (MI) and linear connectivity between layers in two subnetworks. As our method makes no assump- tions on how the subnetwork adversarially trained, is expected to serve as an orthogonal approach to existing adversarial training methods. This is supported by our experimental observations that attack type had little impact on the trends seen for p. We additionally show through our experiments that given semirobust network where fewer than half of the layers adversarially robust (as with VGG16 when f' contains the last 12 trainable layers), training the remaining non-robust portion for a small number of epochs can nearly reproduce the robustness of a network which fully-robust for the same attack. Beyond the potential for subnetwork training to be used alongside other adversarial training methods, the theory outlined here may help provide tools for other methods which rely on training the full network to theoretically challenge this constraint by finding ways to leverage semirobustness within their network.
Looking ahead One open question here is that how we can determine the complexity of the semiro- bust subnetwork performance in terms of convergence rate. The answer to this question involves investigating a bound on performance difference as a function of dependency between layers (p). In addition, although the trend observed between and dataset is consistent and clear, it's less apparent the reason. The narrower range of values in CIFAR- 100 is most likely due either to the larger number of classes (100 vs 10) or the lower resulting predictive accuracy (which is at least in part due the larger number of classes). Imagenette the other hand has the same number of classes as CIFAR- 10, but significantly larger images (224x224 vs 32x32), and fewer samples. Further investigation of this relationship remains an interesting future avenue of investigation.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Naveed Akhtar and Ajmal Mian. Threat of adversarial attacks on deep learning in computer vision: survey. leee Access, 6: 1:440--44430, 2018.
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. In International conference on machine learning, pp. 284-293. PMLR, 2018.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Smdié, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European conference machine learning and knowledge discovery in databases, pp. 387 402. Springer, 2013.
Tom B Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer. Adversarial patch. arXiv preprint 171.0966,, 2017.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39-57. IEEE, 2017.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. International Conference on Machine Learning, pp. 1310-1320 PMLR, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206-2216. PMLR, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust ahysical-worke attacks deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1625-1634, 2018.
Alhussein Fawzi, Seyed-Mohsee Moosavi-Dezfooli, and Pascal Frossard. Robustness of classifiers: from adversarial to random noise. arXiv preprint arXiv:1608 NaKv::600.00967,, 08967, 2016.
Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv: 08011803.00 03635 2018.
Yonggan Fu, Qixuan Yu, Yang Zhang, Shang Wu, Xu Ouyang, David Cox, and Yingyan Lin. Drawing robust scratch tickets: Subnetworks with inborn robustness are found within randomly initialized networks. Advances in Neural Information Processing Systems, 34:1 14:13059-13000 2021.
Justin Gilmer, Nicolas Ford, Nicholas Carlini, and Ekin Cubuk. Adversarial examples are a natural consequence of test error in noise. In International Conference on Machine Learning, pp. 2280- 2289. PMLR, 2019.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and hamessing adversarial examples. In International Conference on Learning Representations, 2015. URL http://
arxiv. hbsgg/abs14445 watii..org/bb///111/ 6572
Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep learning. MIT press, 2016.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Jeremy Howard. Imagenette. URL https::///// htgpee/eeee/ttttte////////tttoo/ http:/ubb.ooooaaaaa..a.aa http://assaai/mmggeeeteeee
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Alek- sander Madry. Adversarial examples are not bugs, they are features. Proceedings the 33rd International Conference on Neural Information Processing Systems, pp. 125-136, 2019.
10

---

Under review as a conference paper at ICLR 2023
Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Physical review E, 69(6):066138, 2004.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, lan J. Goodfellow, and Samy Bengio. Adversarial examples in the physi- cal world. In 5th International Conference Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings. OpenRevicw.net, 2017. URL (tthttps //openreview net. www.ee//oouumioeomemmmme (tt.rummmmm.dHGHHHURGRR
Alexey Kurakin, lan J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Artificial intelligence safety and security, pp. 99-112. Chapman and Hall/CRC, 2018.
Cassidy Laidlaw and Soheil Feizi. Functional adversarial attacks. arXiv preprint arXiv: 1.Ki:19906.00001, 2019.
Jacho Lec, Sejun Park, Sangwoo Mo, Sungsoo Ahn, and Jinwoo Shin. Layer-adaptive sparsity for the magnitude-basee pruning. ICLR, 2021.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. arXiv preprint 111v:1611.02270, 2016.
Xingjun Ma, Bo Li, Yisen Wang, Sarah M Erfani, Sudanthi Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E Houle, and James Bailey. Characterizing adversarial subspaces using local intrinsic dimensionality. In International Conference on Learning Representations, 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare G Bellemare, Alex Graves, Martin Riedmiller, Andreas Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 5 518(7540):529-557 533, 2015.
Kevin R Moon, Kumar Sricharan, and Alfred O Hero. Ensemble estimation of mutual information. In 2017 IEEE International Symposium on Information Theory (ISIT), pp. 3030-3034. IEEE, 2017.
Morteza Noshad, Kevin R Moon, Salimeh Yasaei Sekeh, and Alfred Hero. Direct estimation of information divergence using nearest neighbor ratios. In 2017 IEEE International Symposium Information Theory (ISIT), pp. 903-907.10 IEEE, 2017.
Morteza Noshad, Yu Zeng. and Alfred O Hero. Scalable mutual information estimation using dependence graphs. In ICASSP 2019-2019 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 2962-2966. IEEE, 2019.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Anan- thram Swami. The limitations of deep learning in adversarial settings. In 2016 IEEE European symposium on security and privacy (EuroS&P), pp. 372-387. IEEE, 2016a.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversarial perturbations against deep neural networks. In 2016 IEEE symposium security and privacy (SP), pp. 582-597. IEFE, 2016b.
Qi Peng, Wenlin Liu, Ruoxi Qin, Libin Hou, Bin Yan, and Linyuan Wang. Dynamic stochastic ensemble with adversarial robust lottery ticket subnetworks. arXiv preprint arXiv:2210.02618, 2022.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks, arXiv preprint 1t..iv:15111.06434. 2015.
Aditi Raghunathan, Jacob Steinhardt, and Perey Liang. Certified defenses against adversarial examples. In International Conference on Learning Representations, 2018.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adver- sarially robust generalization requires more data, arXiv preprint arxiv: 804. 1285, 2018.
11

---

Under review as a conference paper at ICLR 2023
Adi Shamir, Itay Safran, Eyal Ronen, and Orr Dunkelman. A simple explanation for the existence of adversarial examples with small hamming distance. arXiv preprint MtXiv:1901.10800 2019.
Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K Reiter. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In Proceedings of the 2016 acm sigsac conference on computer and communications security, pp. 1528-1540, 2016.
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 23(5):828 84188884111 2019.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In 2nd International Conference Learning Representations, ICLR 2014, 2014.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry, Ro- bustness may be odds with accuracy. In International Conference Learning Representations, 2019.
Tsui-Wei Wei Weng, Huan Zhang. Pin- Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. Evaluating the robustness of neural networks: An extreme value theory approach.
arXiv preprint arXiv: 100.11058.. 2018.
Kai Y Xiao, Vincent Tjeng. Nur Muhammad Shafiullah, and Aleksander Madry. Training for faster adversarial robustness verification via inducing relu stability. In International Conference on Learning Representations, 2019.
Cihang Xie, Jianyu Wang, Zhishuai Zhang. Yuyin Zhou, Lingxi Xie, and Alan Yuille. Adversarial examples for semantic segmentation and object detection. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1369-1378, 2017.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. Advances in Neural Information Processing Systems, 32, 2019.
12


Reviews:
[{"review_id": "MYjwSQ4lZ9", "reviewer": "Reviewer_6THo", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper investigates the problem of building robust models by only enhancing a part of the whole model (subnetwork). The paper is mostly theoretical and propose a new concept of semi-robust. The authors also provide empirical evidences to support their claim.", "summary_of_the_review": "As in Strength vs Weakness", "strength_and_weaknesses": "# Strength\nBy only enhancing a part of the whole network, the proposed approach can save the complexity of sota defenses such as adversarial training.\n\n# Weakness\n1. This paper does not illustrate their idea clear enough. Many expressions are informal and confusing. For instance, how do you define \"high correlated layers\" \n2. The theoretical findings seems not enough to support the main claim. Especially the bound of parameter gamma 1 to j is not tight enough to guarantee the meaning of robust.\n3. The authors try to use empirical evidences to illustrate the correctness of their method. Yet, there are many issues in the experiments.  The setup is not very clear. What od *acc and ~acc represent? What is the perturbation radius. What kind of attacks have you chose? Usually the robust accuracy should be around 60%, why your score is so high?\n4. The authors claim that if the subnetwork is robust, then the whole model is robust. Then please explain, if I add a network layer before a trained model such as AlexNet, and the layer do nothing but pass the value through to the network, this layer is apparently robust, how can you prove the model will not be attacked by the adversarial example.  ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "KWO5dra_7J", "reviewer": "Reviewer_fA2z", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a new concept, semirobusness, and a corresponding framework to study if the (semi-)robustness of a subnetwork can indicate the (semi-)robustness of the rest or the all network. The major contribution is a set of theorems to show that under assumptions related to the mutual information between representations output by one layer and the other, the proposed semirobsutness will be carried out to deeper layers (and maybe with a different level). \n", "summary_of_the_review": "For the current moment I incline to reject. I think the theoretical contribution may be over-claimed (but I am not sure about this). I will not exclude the experiment part from the major contribution of this paper as the authors indeed mention the empirical results in their contribution statement. However, the quality of the experiments is not ready for publication.", "strength_and_weaknesses": "### Strength\nThis paper studies an interesting question of whether the robustness of part of the network can imply the rest (or the whole). Towards this end, it proposes this notion of semi-robustness, which is essentially that when the bottom part of the network is fixed, if there exists a top network such that the combination of the bottom and the top produces robust prediction, measured with the correlation of prediction and the label. Although with the dense theorems in the main paper I am not able to examine the proof, I think the theoretical results intuitively make sense. \n\n### Weaknesses\nMy major concerns for this work are (1) if the contribution is over-claimed; and (2) experiment results may overfit to the test set and $\\epsilon$ is missing from the paper. \n\n**Semi-robustness Guarantee**. The paper claims that “For the first time we provide a theoretical framework and prove that under some assumptions if the first part of the network is semirobust then the second part of the network’s robustness is guaranteed”. My understanding of this claim is that for a network $F = F^b(F^a(x))$, if $F^a$ satisfies some assumptions and is semi-robust, then we know something about the robustness of $F$ or $F^b$. However, my observation is that the activation of the first layer of $F^b$ is also part of the assumptions A1 and A2 (please correct me if this is not true). Therefore, the assumption is not only about the bottom network $F^a$ and the current contribution statement sounds like it only requires additional information about $F^a$. Therefore, the current contribution seems to be over-claimed to me. \n\n**Overfitting in the Algorithm 1.** This is my biggest concern in the experiment. The paper directly tries to learn a top network $F^b$ to match the **test robust accuracy** of an adversarially trained network. This is different from matching the robustness of a traditional adversarial training algorithm as you are leaking test information to $F^b$ while the standard algorithm does not. Can you provide the robust accuracy of the partially-trained network on another validation set? I am concerned whether more epochs used in algorithm 1 brings a larger generalization gap. \n\n**Missing Setup.** I found the experiment part also hard to follow as there is a lot of setup missing. What is the size of the $\\epsilon$-ball used by the attacker? How many epochs, the values of $\\lambda$ and other hyper-parameters in Algorithm 1 are used in your experiment. What is the absolute robust accuracy for experiments and datasets mentioned in Figure 2? How many steps do you take for each attack? What did you do for the FGSM attack and how do you adapt that to the $\\ell_2$ case as in the $\\ell_2$ space we do not take the sign of the gradients. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "J9tQbHWf65", "reviewer": "Reviewer_WXTT", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper develops the concept of semirobustness, which indicates the adversarial robustness of a part of the network. The authors prove that if a subnetwork is robust and highly correlated with the rest of the network, then the remaining layers are also guaranteed to be robust. Empirical evaluations are done on CIFAR-10/100 and ImageNet.", "summary_of_the_review": "While the idea of semirobustness is interesting, the empirical evaluation is the main drawback, and there require more accurate details in the theoretical analyses.", "strength_and_weaknesses": "Strength:\n- The idea of semirobustness is somewhat interesting to me and is novel as far as I know;\n\n- The theoretical analyses and conclusions are intuitively reasonable (although I did not carefully check the proofs);\n\nWeaknesses:\n- Lack of details in the claims of Theorem (see Clarity, Quality, Novelty And Reproducibility);\n\n- The attacks (e.g.,  FGSM, I-FGSM, PGD, CW) used for evaluation are weak. Stronger attacks like AutoAttack should be evaluated;\n\n- The robust accuracy in Table 1 is quite high, which is partly because the authors apply $\\ell_{2}$-norm perturbation (there is also no information on the value of perturbation budget $\\epsilon$);\n\n- It is surprising that most of the references cited in this paper are before 2020. This makes me wonder if there is any related work during 2020-2022?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "FjMJpphoq64", "reviewer": "Reviewer_3nVE", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a new notion called the semirobustness of subnetworks. Then, the authors state how subnetwork semirobustness can be extended for the semirobustness of a bigger subnetwork under some condition that relates to the semirobust subnetwork and the rest part. Further theoretical analysis and empirical verification of the statement are provided.", "summary_of_the_review": "I consider the paper novel and valuable work. First of all, a theoretical finding is rare and precious in adversarial machine learning. The semirobustness concept looks to be precious to the field as it opens up new possible training frameworks based on semirobustness. The idea was tested on various models using several attacks, so the experimental support looks to be strong enough.\n\nRegarding some negative factors, the paper did not show enough practical connection between their concept to the field of robust training. Though it is possible to imagine that this concept would yield frameworks for constructing a robust network by stacking semirobust subnetworks, showing such usage is also a part of the authors’ responsibility. Also, details about the experimental setup are missing, and this raises a reproducibility concern. However, I believe that these are amendable issues, and I’m willing to raise the score after they are fixed.\n", "strength_and_weaknesses": "[[Strengths]]\n1. The concept of semirobustness of a subnetwork is novel and has the potential for practical applications.\n2. The proofs look correct, and the experiments well support the theoretical statements.\n3. The experiment used three different models and four attacks, which looks enough for experimental verification.\n\n[[Weaknesses]]\n1. I suggest the authors recheck the formal part carefully. For example, the definitions of $g_\\delta$ and $h_{\\delta,j}$ in Assumptions should be based on subnetworks rather than layers because the input is taken from the input space.\n2. The notion of semirobustness looks valuable, but I’m not fully convinced of its practical usage. The theoretical part contains proof, and the experimental results support the theory. However, what would be the practical usage of the concept? Could the authors add a corollary relating the robustness of the entire network to the semirobustness of its subnetworks? (For example, the authors may propose a bound to the robustness of a network containing subnetwork that is NOT semirobust?) Such a statement would indicate some potential applications in robust training, e.g., robustly stacking more layers to existing robust models, inductively training robust subnetworks, etc. However, how would such applications be more useful than just training a robust network without the knowledge about semirobustness?\n3. In the experiment, the authors reported the accuracy of $f_b^* \\circ f_a^*$ (robust network) and the best accuracy of $\\tilde f_b\\circ f_a^*$ from training the last half of the network with adversarial training. While this is enough to show the theoretical statement (because semirobustness only requires the existence of mapping function $G_j$ that is approximated by $\\tilde f_b$), it would be interesting to compare the accuracy of $f_b\\circ f_a$ and that of $f_b\\circ f_a^*$ (before the additional training of $f_b$) to check the effect of the robust first half in classification. Could the authors add those accuracies to Table1, or justify some reason why such accuracies are not interesting?\n4. The authors did not provide enough detaabout the experimental setup. For example, what are hyperparameters used for the adversarial training (both for the entire network and $\\tilde f_b$)? How many epochs did the authors train $\\tilde f_b$? These details are important for general readers to reproduce the result, and this raises a reproducibility issue.\n5. The discussion in the Conclusion looks like the conclusion from the experiments. Move this discussion to the experiment section and add the conclusion of the entire paper in Conclusion. (If the page limit becomes a problem, I’d suggest the authors move the further discussion from the experiment to the Appendix.)\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject