Learn Robust Features via Orthogonal Multi-Path
Authors: Kun Fang
Xiaolin Huang
Yingwen Wu
Tao Li
Jie Yang
Venue:  (2021)
Keywords: adversarial robustness
orthogonal multi-path

Abstract: 	It is now widely known that by adversarial attacks, clean images with invisible perturbations can fool deep neural networks.
	To defend adversarial attacks, we design a block containing multiple paths to learn robust features and the parameters of these paths are  required to be orthogonal with each other. 
	The so-called Orthogonal Multi-Path (OMP) block could be posed in any layer of a neural network. 
	Via forward learning and backward correction, one OMP block makes the neural networks learn features that are appropriate for all the paths and hence are expected to be robust. With careful design and thorough experiments on e.g., the positions of imposing orthogonality constraint, and the trade-off between the variety and accuracy, 
	the robustness of the neural networks is significantly improved. 
	For example, under white-box PGD attack with $l_\infty$ bound ${8}/{255}$ (this is a fierce attack that can make the accuracy of many vanilla neural networks drop to nearly $10\%$ on CIFAR10), VGG16 with the proposed OMP block could keep over $50\%$ accuracy. For black-box attacks, neural networks equipped with an OMP block have accuracy over $80\%$. The performance under both white-box and black-box attacks is much better than the existing state-of-the-art adversarial defenders. 

Under review as a conference paper at ICLR 2021
LEARN ROBUST FEATURES VIA ORTHOGONAL MULTI- PATH
Anonymous authors Paper under double-blind review
ABSTRACT
It is now widely known that by adversarial attacks, clean images with invisible perturbations can fool deep neural networks. To defend adversarial attacks, we design a block containing multiple paths to learn robust features and the param- eters of these paths are required to be orthogonal with each other. The so-called Orthogonal Multi-Path (OMP) block could be posed in any layer of neural net- work. Via forward learning and backward correction, one OMP block makes the neural networks learn features that appropriate for all the paths and hence are expected to be robust. With careful design and thorough experiments on e.g., the positions of imposing orthogonality constraint, and the rrfde-off between the vari- ety and accuracy, the robustness of the neural networks is significantly improved. For example, under white-box PGD attack with 4. bound 8/ 8/255 (this is a fierce attack that can make the accuracy of many vanilla neural networks drop to nearly 10% CIFAR10), VGG16 with the proposed OMP block could keep over 50% accuracy. For black-box attacks, neural networks equipped with an OMP block have accuracy over 80%. The performance under both white-box and black-box attacks is much better than the existing state-of of-the-art adversarial defenders.
INTRODUCTION
In recent years, Deep Neural Networks (DNNs) have been widely applied many fields (Good- fellow et al. 2016). Despite the great progress, vulnerability of DNNs has also been found. For example, in classification task in computer vision, by adding well-designed, visually-imperceptible perturbations clean images, the resulting perturbed images, a.k.a. adversarial examples, can successfully fool many well-trained DNNs (Szegedy et al. [0144] Goodfellow et al. 2015). Such process of generating adversarial examples called adversarial attack.
Since its proposal, there have been many interesting adversarial attacks, which can be categorized into two types, black-box attack (Papernot et al., 2017 Liu al., 2017 Chen et al. 2017 Su al. 2019) and white-box attack (Goodfellow et al. [015 Kurakin et [1l]2077/Crllnn & Wagner 2017] Madry et al., 2018 Tang et al., (019)). As the name suggests, white-box attacks need complete information of the target model. While black-box attacks rely on the output of target model or transferability across models.
For neural network f(x; 0) with input and parameters 0, we denote the trained parameters Ô and the example to be attacked Xo- Adversarial attack tries to find small Ax such that fxx Ax; 0) * fxxo: 0). To defend the attack, i.c.. to keep the both sides equal, adversarial training (Szegedy et al.) 2014; Goodfellow et al. 2015 Madry et al. 2018) includes a group of
adversarial perturbations in training process keep f(xo + +Ax;6) = ().: Generally speaking, adversarial training is the most efficient defence strategy until now (Athalye et al.) [018 TTamer
jet al. 2020), but the attack needs to known in advance. To adapt to all perturbations, researchers consider the response to perturbations Ax in B. = [Ax]]|Ax < E). If the maximum change is small, then a certified robustness could be guarantced (Raghunathan et al. 2018 Cohen et al. 2019).
Motivated by the coupling of samples and parameters, rameters to enhance the robustness. Consider linear layer, which includes convolution layer and fully-connected layer. Imposing perturbation into samples is equal giving randomness to param-
could also impose randomness pa-
eters: VAx E B., there exists A0 that satisfies (xo Ax, 0) (Xo, 0 + A0). Pioneering and

---

Under review as a conference paper at ICLR 2021
vanilla
OMP
OKE bock
clean
rogular
Padiy
perturbed
network with OMP block regular network
(b) learned features (Ist layer, VGG16)
Figure 1: (a) OMP block is to replace single path by multiple ones and can be posed anywhere in neural network. (b) When clean examples (Ist row) are adversarially perturbed (2nd row, generated by attacking third neural network), the features learned vanilla VGG16 change lot. After imposing an OMP block even in the last layer, the learned features by the first layer become much more robust.
representative works on this direction could be found in (He et al. 2019 Liu et al. 2018b), which aim learn distribution for the parameters. The advantages include network diversity and low dependence on the attacks. However, its learning is not very effective: the learned distribution tends shrink to one optimal solution.
In this paper, we propose to embed multiple paths into a neural network. a regular neural network, block with input z and output 2+1 is denoted as -ー g(x";0,) For the mapping g. will train multiple paths g (z": (0:x;)))) then could give multiple outputs and the rest layers trained to adapt all the paths. A key issue here is that we require the parameters of all paths orthogonal to each other, which guarantees the diversity and coverage. Fig [(a] gives comparison illustration of regular network and a network embedded with L paths.
The proposed Orthogonal Multi-Path (OMP) block can be posed in any layer of a neural network. is not surprising that the follow-up layers are more robust since they are capable to handle features from multiple paths. Let us consider a simple example. VGG16 is trained on CIFAR 10 and we put an OMP block in the first layer. The average feature change A at the layer after OMP block caused by perturbations || S 8/255 on test images is bounded by 1.39 (which is actually a certificated robustness measure) in the vanilla VGG16 and is improved to 0.82 by the OMP block. Interestingly, OMP is also helpful for the front layers. In Fig I(b] we visualize the learned features after the first layer in vanilla network and the same network with OMP block posed the last layer. Although the OMP block is posed on the final layer, it could correct the learned features at the first layer, resulting in much more alike features of clean and perturbed images. This phenomenon explained by the backward correction theory proposed by Allen-Zhu & Li (2020). Actually, this result could be a strong evidence to verify that training higher-level layers improves the features of lower-level ones.
With forward learning and backward correction, OMP block could make the feature extractor adap- tive to multiple paths and then enhance the robustness of the whole networks. For example, under white-box PGD attack with Loc bound 8/255, which could destroy the accuracy of many vanilla networks to nearly 10% on CIFARIO. VGG16 with the proposed OMP block could keep over 50% accuracy. The contributions of this work are summarized as follows:
A novel defence method is proposed, which introduces orthogonal multiple paths into neural network to enhance the robustness.
Extensive empirical results against different white-box and black-bor box attacks indicate the superior robustness of networks with OMP block in vanilla and adversarial training.
A thorough empirical analysis different positions of the OMP block is provided, illustrat- ing the distinct properties. Ablation study also demonstrates the necessity and effectiveness of the OMP block.
2

---

Under review as a conference paper at ICLR 2021
2
RELATED WORK
2.1 ADVERSARIAL ATTACK
White-hox adversarial attacks generally based the gradient information. Given DNN f()) and loss function -(--), Goodfellow et al. (2015) propose a gradient-based white-box attack method, named Fast Gradient Sign Method (FGSM), which generates an adversarial example from clean sample x with the true label y as follows,
x
esign( V (( v))
(1)
where E is the perturbation step size. Kurakin et al. (2017) then propose two iterative variants of FGSM, basic iterative method and iterative least-likely class method. Projected Gradient Descent (PGD) attack proposed by Madry et al.] (2018) the strongest attack until now. Starting from x()) x, the iterative generation process of PGD the (t + 1)-th iteration can be described as,
x (+++) =
esign Vx((f(x()). v)1))
(2)
where P., is the projection to the set [x|[|x 10 S 7) and & is the step size in each update.
Black-box adversarial attacks only have access the input and output of model. To conduct black-box attacks, a typical method based on the transferability of adversarial examples is to first attack substitute model, which is transparent the attackers. Then, the adversarial examples from the substitute model are used to attack the target model (Liu et al., 2017 Papernot et al., (017)). Apart from this, other black-box attacks aim estimating the gradients by keep querying the target network (Chen et al. 2017 Su et al. 2019).
2.2 ADVERSARIAL DEFENCE
To improve the adversarial robustness of DNN models, training based on adversarial examples, called adversarial training. is efficient. Among different adversarial examples generation algorithms, PGD-based adversarial training has been proved the best choice and keeps the highest accuracies un- der different attacks (Madry et al.
2018
Athalye et al. 2018). Other defence methods include net-
work distillation (Papernot et al. 2016),
adversarial detection (Lu et al. 2017: Metzen et al.) 2017)
and feature analysis (Xu et al. 2018 Xie al.. 2019), which are suitable different situations.
Another line of defences is to introduce randomness on parameters motivated by the coupling of samples and parameters. For example, [e et al. (2019) propose add learnable Gaussian noise on parameters, and [iu et al. (2018b) combine adversarial training with Bayes neural network. Apart from learning distribution for the parameters, researchers also try inject randomness into neural networks in other ways: such as fusing the activations with fixed Gaussian noise (Liu et al. 20188) 20) or learnable Gaussian noise (Jeddi et al., 2020), performing dropout with a weighted distribution (Dhillon et al.) (0018), and randomizing the input images (Xie et al. (018)) However, [Athalye et al. 2018) claim that the improvements brought by these methods are due to stochastic gradients.
Last but not least, we notice that very recent concurrent work (Jalwana et al. 2020) also uses orthog- onality to enhance the robustness of neural networks. The proposed OMP block is totally different from (Jalwana et al.) 20200: 1) Jalwana et al. (2020) train new regular network that orthogonal with reference network at every layer. While we pose an OMP block in any layer
network, and
the orthogonality is imposed on the inner paths in OMP block; 2) Jalwana et al. (2000) require thogonality on the gradients, while OMP block imposes orthogonality on the parameters; 3) Jalwana et al. (2020) only investigate the resistance against adversarial examples created from the reference network, while networks with OMP block could defend both white-box and black-box attacks.
3 METHODOLOGY
3.1 ORTHOGONAL MULTI-PATH
We embed OMP block into neural network. This OMP block could be posed at any layer and contains multiple -rthogonl paths. Not only the learned features in the follow-up layers show better robustness, but also the OMP block could correct features learned by front layers.
3

---

Under review as a conference paper at ICLR 2021
Without loss of generality and to make the notation simpler, take the OMP block posed on the last linear layer as an example to derive the objective function, where each path in the OMP block contains linear classifier (see Fig 2()). In regular network, the lincar classifier is denoted g(-) with weight matrix W (assuming no bias term), and the remainder of the network is denoted as ht()). Thus the total network is denoted as g(h(-)) Rd RK where and K the input and output dimension respectively.
Before stepping into the objective function, first give the definition of orthogonality. Generally, two one-dimensional vectors are orthogonal if their inner product equals to zero. For two matrices or tensors, we first vectorize them and then perform ordinary inner product. The inner product of two linear classifiers g" and g is computed as:
(g',g² = (vec W vee (W))))
Two linear classifiers are orthogonal if the inner product of their vectorized weight matrices equals to zero.
To train neural network with an OMP block posed on the last linear layer, we have the following optimization problem:
Σ (h(x))))
min h.g w
(4)
s.t. (G' =0
Vi (1,2,.,
where denotes the number of paths in the OMP block. The objective function in Eq.(4) indicates that we minimize the sum of classification losses of all the networks w.r.t. every path in the OMP block, The equality constraints in Eq.44) require that these classifiers on the paths in the OMP block have to be nutually-orthogonne and the remainder of the network. Incorporating the equality constraints into the objective function, have the following optimization problem,
Parameters to be optimized include all the paths in the OMP block
E((((((x)).)) + Σ Σ 9 j=
min h.g g
(5)
where À is the orthogonality coefficient. In Eq. the inner product term gets squared and summed to achieve the orthogonality of every two paths. For cases where OMP block imposed on other layers (see Fig 2(a) and Fig 2(b). the optimization problem has similar form Eq 4 and Eq.(5) and is omitted.
3.2 TRAINING AND INFERENCE
Alg T illustrates an example to train neural network with one OMP block posed on the last linear layer. The orthogonality constraint can be seamlessly equipped with vanilla training or adversarial training. vanilla training, at each iteration, we compute the loss values of Eq, back propagate the gradients and update parameters. In adversarial training, each iteration, after updating param- eters using clean samples, adversarial examples are generated based on current network, and then
Beck
(a) OMP block on the first convolu- (b) OMP block on the middle con- (c) OMP block on the last lincar tion layer.
volution layer.
layer.
Figure 2: Illustration of three architectures of networks with one OMP block. The parameters in each path the block (in blue) are required to be mutually-orthogonal..
4

---

1: while not reach epoch do Under review as a conference paper at ICLR 2021
Algorithm Example: Training neural network with one OMP block posed on the last linear layer
Require: Training set Xtr = {(x,, yi)) number of paths L, orthogonality coefficient A, learn- ing rate 7,, number of epochs epoch, whether to perform adversarial training adv 0 denotes the to-be-optimized parameters of h, g'.....'''
Ensure: A neural network with OMP block: g (h(-)), (1.
2: for (x.y) in Xr do
Le = = C(g'(h(x)); u): Lo = Ei= Ej= (
3:
4:
loss = A.
5:
0 n. Voloss
6:
adv train then
7:
xadv agdd_attack(x,11
ad = Ei= Lig (h(xadv)) () u): Lo = -E Ej 90'
8:
9:
Iossado = Leadv + t. + N...ooll Volossade
10:
11: end if
end for
13:
end while
adversarial losses are computed according to Eq. to update the parameters again. More training details can be found in the appendix.
In adversarial training, when generating adversarial examples, every time the input passes through the network, path is randomly selected from OMP block. For example, in PGD-based adversar- ial training, the iterative process (q.(2) can be rewritten as
(t++) P. x Esign( (x ( (),)))
where the index is randomly selected from 21, L..
In inference, we evaluate the performance of each single network w.r.t. every path in the OMP block, including the accuracies on clean samples and the robustness against adversarial examples.
4 EXPERIMENTS
4.1 EXPERIMENT SETUP
Data set and network architecture. We evaluate the performance of networks with OMP block on CIFAR 10 data set (Krizhevsky) 2009), which includes 50K training samples and 10K test samples of 32 x 32 x color images. In both training and inference phase, the mean-variance normalization preprocess is removed for convenient and fair comparison of different defences under 1o0 bound attack on image pixel range [0, 1].
The effectiveness of the OMP block is tested two common network architectures: vgg-like net- work (vgg-11/13/1) 1/13/1//))) (Simonyan & Zisserman 2015) and residual network (resnet-20/32) (He) et al. 2016). For both network architectures, we impose OMP block three different positions: the first convolution layer, the middle convolution layer (the middle residual block for residual net- work), and the last linear layer, denoted as OMP-a, OMP-h and OMP-c respectively. Besides, the number of paths in the OMP block is set as 10, i.e., L = 10.
Adversarial attack. For white-box attacks, we consider FGSM (Goodfellow al. 2015) and PGD (Madry et al., 2018), two powerful and popular attacks, and impose them on every network w.r.t. each path in the OMP block. Lxo distance is used to measure the difference between the perturbed image and the clean image. For one-step FGSM, L.0 distance refers to the step size s. For multi-step PGD, lox distance refers to the lac bound y, For black-box attacks, we evaluate the robustness based on the transferability of adversarial examples. Perturbed images created from source model are used to deceive the target model. Due to space limitation, we put the ablation study in appendix A.5
5

---

Under review as a conference paper at ICLR 2021
FCSM sttakk on model-1
FCSM sttack on medel2
FCSM attack on modele
70
12
12
12
stop S6co (2255) FGSM
stop Sico 1/55) of F-220
s10p 5428 1/2551 FTF
Figure 3:
Robustness of each network in model-1 (OMP-a), model-2(OMP-b)) and (OMel-3(OMP-e)) There is small variance among the network performance in model-3 (right), and large variance among the network performance in model-1 and model-2 (left and middle).
4.2 ROBUSTNESS AGAINST WIITTE-BOX ATTACK
Empirically, there is very distinct performance when OMP block is posed on the last linear layer (OMP-c) or the convolution layers (OMP-a and OMP-b). For example, suppose model-1 of OMP-a, model-2 of OMP-b and model-3 of OMP-c, then every model consists of 10 networks, each of which only differs in the paths in the OMP block. We show the results of white-box FGSM attack these 10 networks respectively for model-1, model-2 and model-3 in Fig
In Fig 3 for model-3 of OMP-c (right), the network performance shows a very small variance. Nu- merically, under the same step size of FGSM attack, the maximum difference among the accuracies of these 10 networks is less than two percentage points. Meanwhile, for model-1 of OMP-a and model-2 of OMP-b (left and middle), there is much larger variance among the performance of these 10 networks. Even though the robustness of each single network model-1 and model-2 seems much worse than model-3, but there still exist interesting properties. Next, we discuss the robustness of OMP-c separately with OMP-a a and OMP-b.
OMP block the linear layer. T shows the accuracies of OMP-c clean samples and adversarial examples. For clean samples, we report the average accuracies with standard deviations of the 10 networks of OMP-c. One find that compared with vanilla adversarial training network, OMP-c keeps nearly the same test accuracies on clean samples with almost no reduction. For adversarial examples, we again perform white-box FGSM and PGD attacks on each of the 10 networks in OMP-c and report the average accuracies with standard deviations. By posing one OMP block on the last linear layer, there are evident robustness improvements of OMP-c in the both cases of vanilla and adversarial training, since more robust features are learned according to Fig [bb) Apart from attacking every single path respectively, to further verify the robustness, we also simultaneously attack all the paths in the OMP block and the results can be found in A.3.1
OMP block on the convolution layer. As shown Fig there is large variance among the ac- curacies of networks in OMP-a and OMP-b. Besides, the robustness of single network in OMP-a or OMP-b is also vulnerable. However, suppose one network net-/ w.r.t. one path in the OMP block gets attacked, the corresponding adversarial examples are then reclassified by another network ner-2 w.r.t. another path in the OMP block. Under this setting, accuracies of nel-2 on the adversarial examples get obviously improved compared with ner-1, which indicates that only by changing the specified layer with another orthogonal parameters, network robustness can get enhanced.
For OMP-a, where the OMP block posed on the first layer, we select one path and perform FGSM and PGD attacks on this network. The corresponding accuracy variations are shown as the dashed line in Fig [(a] The adversarial examples created from this network are then reclassified by other networks w.r.t. other paths in the OMP block, which shows significantly improvements as the solid lines Fig (a) There also exists similar phenomenon of OMP-b, shown Fig 4(b)
We give brief summary for the properties of OMP-a, OMP-b and OMP-c:
For OMP-c, which consists of cantually-orhhgeee linear classifiers, there is a small per- formance variance among these networks. The robustness of each network all gets sig-

---

Under review as a conference paper at ICLR 2021
Table Accuracies (%) on clean samples and adversarial examples of test set in CIFARIO. The Lo distance of FGSM and PGD attacks equals to 8/255.
vanilla training
adv. training FGSM
model
clean
FGSM
clean
PGD
vgg OMP-c
91.33
19.59
87.04
41.52
23.81
91.57::0.02
30.27:10.21
86.73-10.03
43.95:10.04
2..53
vgg13 3
93.19
25.93
89.23
48.67
32.48
OMP-c
93.17#0.00
35.12+00117
89.28+00003
50.60:10.03
33.86=t0.06
1ggl6
93.18
19.52
88.77
56.51
44.23
OMP-c
93.02::0.01
49.04=0.477
89.01=0.001
62.42:10.03
50.60:10.10
vgg19
92.86
10.35
89.07
55.25
39.30
OMP-c
92.03
53.00
88.9540.01
63.35:10005
5..52:10.10
resnet20
91.71
13.36
88.03
35.06
22.26
OMP-c
91.75.:0.03
2..66:10.66
87.95-10.03
38.24:10.10
23.64:10.08
resnet32
93.20
28.26
88.65
37.93
23.26
OMP-c
92.85+00.03
24.15:10.59
19.14440.04
03.12:10.16
26.91:10.14
H05M Httcc
150 20:33
FGOM BETOCK
1032
00
60
60
20
40
30
20
10
2005
stop ROSM
(a) Example of OMP-a
(b) Example of OMP-b
Figure 4: Illustration of the robustness of OMP-a and OMP-b. Single network in OMP-a OMP-b shows terrible robustness (dashed lines). But adversarial examples created from this network could be classified correctly by other networks (solid lines). Detailed settings can be found in A.3
nificantly improved, with no performance reduction on clean samples, which indicates the possibility of balancing the trade-off between generalization and adversarial robustness.
For OMP-a and OMP-b, which consist of large performance variance among these networks. The robustness of single network terrible. But, adversarial examples created from one of these networks can be successfully reclassified by other networks. It indicates that by altering the direction of the parameters of one layer to another orthogonal direction, network robustness get improved. To the best of our knowledge, this also the first work to enhance the network robustness by only changing the parameters of one layer.
convolution layers, there is
4.3 ROBUSTNESS AGAINST BLACK- BOX ATTACK
We evaluate the robustness of network with OMP block against black-bo attacks based on the trans- ferability of adversarial examples. In detail, we perform FGSM and PGD attacks on two transparent source models: vanilla vgg 16 and resnet20. The resulting adversarial examples are then reclassified by the target model. Target models include vanilla and adversarial training networks and the corre- sponding OMP-c. Results against black-box attacks from different attacks and network architectures shown in Tab Based on the improved competitive accuracies in Tab OMP-c could suc- cessfully defend Black-box attacks. Results networks with OMP-a -a and OMP-b can be found in the appendix A.4
7

---

Under review as a conference paper at ICLR 2021
Table 2:
Accuracies (%) of black-box attacks on test set CIFARIO. The Lo distance of FGSM
and PGD attacks equals to 8/255.
attacks source models
target model
FGSM on vggl6
PGD on vggl6
FGSM on resnet20
PGD on resnet20
vgg 16 OMP-c
19.52
0.02
49.57
46.81
41.91:10.04
25.46:10.00 0.00
49.44:1:0.03
49.55 0.02
vgg 6-adv
85.13
86.92
86.67
87.46
OMP-c-adv
85.37=0.011
0.02
86.78:+0.01
87.84+10.01
resnet20
37.64
19.36
13.36
0.00
OMP-c
36.79:00.06
21.06=0.044
32.66:10.09
18.08=00055
resnel20-adv
82.98
85.34
84.50
86.22
OMP-c-adv
82.43=0.005
85.18
84.21=00.04
86.19=0.077
4.4 ROBUSTNESS COMPARISON
The robustness of networks with proposed OMP block is compared with two defence methods via imposing randomness on parameters, PNI (He et al. 2019) and advBNN (Liu et al., 2018b). Both methods aim to learn a distribution for the parameters. PNI adds learnable Gaussian noise on pa- rameters. AdvBNN incorporates adversarial training into Bayesian neural network.
Accuracies of different models against white-box FGSM and PGD attacks are shown in Fig 5 These models are of vgg-like architecture. We compare networks with OMP block pose on the last linear layer with other methods, and report the average accuracies and standard deviations of OMP-c. It is clearly seen that in vgg-like model, networks imposed orthogonality show much better robustness compared with networks imposed randomness parameters.
00.
bV
1sh D60-
ssk catinc
snc Onnic
60
chfinn
60-
40
20
112551 cErGoM
suuund (055) of PCC
(220) 1C24
p(225)) et
(a) Comparison based vggl6
Comparison based on vgg19
Figure 5: Robustness comparison of different models against FGSM and PGD attacks.
CONCLUSION
In this work, we propose embed orthogonal multiple paths into a neural network to enhance robustness. The mechanism behind is to learn robust features via forcing the follow-up layers to learn to fit the paths and correcting the features learned by front layers. We investigate distinct properties of different positions of the OMP block against white-box attacks, e.g., balancing the trade-off between generalization and robustness, and enhancing robustness via only changing one- defending black-bor box attacks, robustness comparison and
layer parameters. Besides, experiments ablation study are all adopted to verify the effectivenss of OMP block. We hope that this work could help inspire researchers to better understand the essence of generalization and robustness of DNNs.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Zeyuan Allen-Zhu and Yuanzhi Li. Backward feature correction: How deep learning performs deep learning, arXiv preprint arXiv:2001.0441. 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of curity: Circumventing defenses to adversarial examples. In International Conference on Machine Learning. pp. 274-283, 2018.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE symposium on Security and Privacy (SP), pp. 39-57. IEEE, 2017.
Pin-Yo Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsich. Zoo: Zeroth order opti- mization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 15 15-26, 2017.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. International Conference on Machine Learning, pp. 1310-1320, 2019.
Guneet Dhillon, Kamyar Azizzadenesheli, Zachary C Lipton, Jeremy D Bernstein, Jean Kossaifi, Aran Khanna, and Animashree Anandkumar. Stochastic activation pruning for robust adversarial defense. In International Conference on Learning Representations, 2018
lan Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume 1.
MIT Press, 2016.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp- 770-778, 2016.
Zhezhi He, Adnan Siraj Rakin, and Deliang Fan. Parametric noise injection: Trainable randomness to improve deep neural network robustness against adversarial attack. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 588-597, 2019.
Mohammad AAK Jalwana, Naveed Akhtar, Mohammed Bennamoun, and Ajmal Mian. Orthogonal deep models as defense against black-box attacks. IEEE Access, 8: 1--19944-1197577 2020.
Ahmadreza Jeddi, Mohammad Javad Shafiee, Michelle Karg, Christian Scharfenberger, and Alexan- der Wong- Learn2perturb: an end-to-end feature perturbation learning to improve adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 1241-1250, 2020.
A Krizhevsky. Learning multiple layers of features from tiny images. Master's thesis, University of Tront, 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In International Conference on Learning Representations, 2017.
Xuanqing Liu, Minhao Cheng, Huan Zhang, and Cho-Jui Hsich. Towards robust neural networks via random self-ensemble. In Proceedings of the European Conference Computer Vision (ECCV), pp. 369-385, 20.
Xuanqing Liu, Yao Li, Chongruo Wu, and Cho-Jui Hsich. Adv-bnn: Improved adversarial defense through robust bayesian neural network. In International Conference Learning Representa- tions, 2018b.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial exam- ples and block-box attacks. 2017.
Jiajun Lu, Theerasit Issaranon, and David Forsyth. Safetynet: Detecting and rejecting adversarial examples robustly. In Proceedings of the IEEE International Conference on Computer Vision, pp. 446-454, 2017.

---

Under review as a conference paper at ICLR 2021
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. On detecting adversarial perturbations. 2017.
Nicolas Paperno, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation defense to adversarial perturbations against deep neural networks. In 2016 IEEE Symposium on Security and Privacy (SP), pp. 582-597. IEFE, 2016.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pp. 506-50666 519, 2017.
Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified defenses against adversarial exam- ples. In International Conference on Learning Representations, 2018.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 23(5):828-888 841, 2019.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. 2014.
Sanli Tang, Xiaolin Huang, Mingjian Chen, Chengjin Sun, and Jie Yang. Adversarial attack type Cheat classifiers by significant changes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. arXiv preprint arXiv:2002.083-47 2020..
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. International Conference on Learning Representations, 2018.
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan L Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 501-509, 2019.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. 2018
A
APPENDIX
A. TRAINING
We first give more details in training networks with OMP block:
When computing Eq. 5). we determine equally-weighted loss with respect to each path in the OMP block, i.e., the loss term in Eq actually is 1. (,(2)) v).
For the orthogonality coefficient in Eq 5 we set this hyper-parameter as À = 0.1.
We do not investigate more alternative values since this setting already results in satisfying perfor- mance.
10

---

Under review as a conference paper at ICLR 2021
A.2 NETWORK STRUCTURE
Then, we describe the detailed structures of networks with OMP block:
For network of OMP-a, the OMP block is posed on the first convolution layer. In vgg-like network, each path OMP block contains the convolution kernel parameters of 3 x x 3 x 3 dimension. In residual network, each path in OMP block contains the convolution kernel parameters of 3 x 16 3 3 dimension. The bias term the first convolution layer is omitted.
For network of OMP-b, the OMP block posed on the middle convolution layer. In vag- like network, the OMP block is posed on the convolution layer right after the first max- pooling layer, and each path contains the kernel parameters of x 128 x x dimension. In residual network, the OMP block is posed on the second residual block in the second stage, and each path contains a residual block. Only the convolution parameters in the residual block required to be mutually-orthogonme The bias term in the convolution layer is omitted.
For network of OMP-c, the OMP block is posed on the last linear layer. In vgg-like net- work, each path in OMP block contains the weight matrix of 512 10 dimension. In residual network, each path in OMP block contains the weight matrix of 64 x 10 dimen- sion. The bias term in the last linear layer omitted.
A.3 ROBUSTNESS AGAINST WHITE-BOX ATTACK
We explain how we produce the results in Fig Results in Fig (() and 4(g are based on resnet20 architecture. We first show the accuracies on training and test sets every resnet20 net- work w.r.t each path in OMP-a or OMP-b block respectively in Tab
Table Accuracies (%) of each network w.r.. each path in OMP block on CIFAR clean images.
net-1
net-2
net-3
net-4
net-5
net-6
net-7
net-8
net-9
net-10
training
95.26
96. 108
93.022
13.914
68.96
97.42
96.464
85,98
84,576
94. 104
OMP-a
test
87.11
87.62
85.24
13.82
66.05
88.86
87.32
79.72
78.68
85.59
training 98.868
98.81
98.88
98.948
98.894
98.799 08.798
98.912
98.918
98.814
98.906
OMP-b
test
88.28
88.14
88.01
88.44
88.15
88.35
88.16
88.2
88.28
88.28
In Tab for OMP-a, some networks show distinct accuracies clean samples, c.g. net-4, net- 5, net-8 and net-7. While every network in OMP-b shows similar accuracies on clean samples. However, the robustness of these networks is vulnerable and varies lot (see the left and middle images in Fig In Fig the attacked network of OMP-a (dashed lines) is the net-6 in the table above, which shows the highest accuracies on clean samples. We record the accuracies of net-l, net-2, net-3, net-7 and 10 (solid lines) on the adversarial examples created from net-6. Other networks are not considered since their performance on clean samples is not acceptable.
In Fig 4bb the attacked network of OMP-b (dashed lines) also the net-6 in the table above. We record the accuracies of the other 9 networks (solid lines) on the adversarial examples created from net-6.
A.3.1 A STRONGER WHITE- BOX ATTACK
In this section, we perform stronger white-box attack to further verify the robustness of network with OMP-c. Instead of attack every single network w.r.t every path the OMP block, we si- multaneously attack all the networks w.r.t. all paths in the OMP block. The resulting adversarial examples are then reclassified by all the networks. The average accuracies and standard deviations against FGSM and PGD attacks are shown in Fig It can be that even though all the paths attacked simultaneously, networks with OMP block could still defend this stronger white-box attack and sustain better robustness than adversarial training networks.

---

Under review as a conference paper at ICLR 2021
FGOR stfct
FGD stuck
FCOM stAACE
PCD witscc
ven Sivelt
70
20
2000 stzu
sonddd
stap sice (203) TCON
bound p/225)
(a) Robusiness of vggl6 architecture
(b) Robustness of 1g architecture
Figure 6: Robustness against stronger white-box attack, which simultaneously attacks all the paths in the OMP block. Even under such stronger attack, the networks of OMP-c block still show superior robustness.
A.4 ROBUSTNESS AGAINST BLACK-BOX BOX ATTACK
We add the results of networks with OMP-a and OMP-b block against black-box attacks in Tab
In Tab for networks with OMP-a block, since each network w.r.t. each path shows large variance, only report the largest accuracies among these networks. For networks with OMP-b block, still report the average accuracies of all networks w.r.t. all paths in the block together with the standard deviations. These results indicate that networks with OMP-a or OMP-b both have the ability in defending black-box attacks.
Table 4: Accuracies (%) of black-box attacks test set in CIFARIO.
attacks source models
target model
FGSM on vgg16
PGD on vgg16
FGSM on resnet20
PGD on resnet20
vgg 16
19.52
0.02
49.57
46.81
OMP-a
30.92
27.39
30.26
38.31
OMP-b
35.88++2.10
25.84+1
37.02++2.45
41.78++:2.00
OMP-c
41.91+0.004
25.46-+0.03
49.44++0033
49.55:10002
vgg 16glv
85.13
86.92
86.67
87.46
OMP-a-ady
61.41
64.57
64.88
68.42
OMP-b-adv
80.34-1:1.3
83.41
82.72-11.55
84.88 1 1.44
OMP-c-adv
85.37=0.011
87.28=0.01
86.78=10.01
87.84+0.01
resnet20
37.64
19.36
13.36
0.00
OMP-a
39.35
36.02
38.23
39.67
OMP-b
34.81
18.96
31.88
18. 18.18 18.1810.11
OMP-c
36.7940.00
21.06+0.004
32.66-
18. 08.0
resner20-ady
82.98
85.34
84.50
86.22
OMP-a-adv
77.71
83.40
80.71
85.24
OMP-b-adv
82.70=+0.20
85.46=00.14
84.32-0..19
86.41=10.18
OMP-c-adv
82.43+0.00
85.08
84.21+0.04
86.19
A.5 ABLATION STUDY
In this subsection, explain the necessity of learning the OMP-block. Empirically, once the coef- ficient = 0, then the parameters multiple paths in OMP block will converge to the same one, which is meaningless. Therefore, adopt the ablation study from another perspective. Actually, two random vectors in high- high-dimensional space are apt to orthogonal, in other words, the inner product of two random high-dimensional vectors approximates zero. Hence fix the parameters
12

---

Under review as a conference paper at ICLR 2021
Accurades
Nardew
Leatred Handdomyy-ixol
Randomlyy
80
50-
-0
40
40
20
10
12
stop (223) FESM
(2255)
(a) Performance clean samples
(b) Robustness against adversarial attack
Figure 7: Comparison between networks with learned OMP block (in blue) and networks with randomly-fixed OMP block (in orange). The network architecture resmet20 OMP-c. The aver- age mean accuracies of all the networks and the standard deviations are recorded.
of the OMP block with random values and only train the rest of the neural network. In this way, we could verify the effectiveness of backward correction.
Fig shows the comparison results between networks with learned OMP block and networks with randomly-fixed OMP block. (a) illustrates the performance on clean training and test sam- ples. Fig 7bb illustrates the robustness against FGSM and PGD attacks. If the OMP block is not jointly learned, there is large variance among the network performance. Besides, the performance of network with randomly-fixed OMP block on both clean samples and adversarial examples also becomes than that of network with learned OMP block. Therefore, only by jointly learning the embedded OMP block with the remainder of the network, the network could get robustness improvements and keep high accuracy clean samples simultaneously.
13


Reviews:
[{"review_id": "xXs_eQpCwND", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to learn multiple near-orthogonal paths (OMP) in the CNN which could provide better adversarial training performance by using one random path selected from the OMP block, improving the diversity of the adversarial training examples generated. Results show some improvements over regular adversarial training. Interestingly, the improvements are very significant on the VGG networks, while not quite significant for the ResNet variants tested.\n\nThis paper claimed that it creates orthogonal paths, but it's realistically near-orthogonal since they only added a soft constraint on the OMP regularization term, similar algorithms have been proposed in the past:\n\n[Bansal et al. 2018] Can we gain more from orthogonality regularizations in training deep cnns?\n\nThere have also been quite a few work on learning real orthogonal paths based on Riemannian manifold optimization. Some of these are of similar speed as conventional SGD and Adam. A review paper can be found at:\n\n[Huang et al. 2020] Normalization Techniques in Training DNNs: Methodology, Analysis and Application.\n\nSome of those papers should be cited.\n\nIn terms of performance, I feel this work should be compared against other regularization-based adversarial defense methods. A couple examples of that are:\n\nQin et al. Adversarial Robustness through Local Linearization. NeuRIPS 2019\nMao et al. Metric Learning for Adversarial Robustness. NeuRIPS 2019.\n\nComparisons against those algorithms would further verify the performance of the proposed approach.\n\nBesides, there should be some discussions on potentially why the improvements on VGG networks are very significant and not so much on ResNet.\n\nThere is also some recent evidence on the effect of early stopping on adversarial defenses (e.g. Rice et al. Overfitting in adversarially robust deep learning. ICML 2020). It would be nice if the authors could state when did they stop the training of the respective models.\n\nIn terms of ablation, it would be nice to see different inference schemes. e.g. whether using a subset of the paths in the OMP block would be beneficial against adversarial examples or not.\n\nI look forward to seeing the authors rebuttal and comments from other reviewers.", "extra_scores": {}, "extra_text": {}}, {"review_id": "k5E2KvOXV1T", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper addresses the problem of adversarial defence, by proposing to build multiple parallel orthogonal layers to replace a regular neural network layer. The layers in the OMP block are trained to be diverse and orthogonal to each other. Experiments on both white-box and black-box attacks, with or without adversarial training, have been carried out to show the efficacy of the proposed method, on the CIFAR 10 dataset.\n\n\\- I think for OMP-a and OMP-b, it is no longer considered as white-box attack when we change the direction of the parameters of one layer to another orthogonal direction, as the authors described below Fig 4.\n\n\\- Only Cifar 10 is used to verified the proposed method. Since Cifar 10 is a simpler dataset with low resolutions, it might be helpful to involve other datasets in the evaluation part.\n\n\\- There are some other defenses based on diversity ensemble, which are not discussed in the paper. For example, \"Improving Adversarial Robustness via Promoting Ensemble Diversity\".\n\n\\- From Table 2, does it mean the proposed method is not effective to improve the robustness for black-box attack compared with vanilla adversarial training?\n\nAfter reading the response, my concerns are not fully resolved. For example, based on \"For OMP-a and OMP-b, ... The robustness of single network is terrible. ... adversarial examples created from one of these networks can be successfully reclassified by other networks\", I am feeling the OMP-a and OMP-b are less effective based on a real white-box attack. Also some other concerns are not fully addressed. Thus I am keeping my rating unchanged.", "extra_scores": {}, "extra_text": {}}, {"review_id": "i75nlbBR3UD", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes an adversarial example defense method based on ensembling the final linear layer of a classifier, where the components of the ensemble are jointly trained with each other and with the backbone of the model to minimize cross-entropy and to be approximately orthogonal with each other. Orthogonality here is measured by flattening the layers weight matrices and computing their inner products. Training examples are either original classification examples or adversarial examples computed by the PGD attack, used in a generative adversarial traning approach.\n\nThe authors experiment on standard image classification dataset and compute robustness to both white box and black box attacks, obtaining some improvements over plain GAT. The authors also experiment with ensembling other layers of the model but obtain worse results.\n\nThe proposed method is interesting, however it increases the model size, hence it should be compared to a non-robust or GAT model for the same parameter budget. Also, in the description of the inference procedure it was not very clear how the ensemble predictions are combined. Overall, this appears to be an incremental improvement.", "extra_scores": {}, "extra_text": {}}, {"review_id": "HZ5ppa4jg50", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this work, the authors propose to use orthogonal multi-path(OMP) block to improve the adversarial robustness of deep neural networks. They introduce three types of OMP, OMP-a/b/c, based on where the OMP block is located in the neural network. Experimental results demonstrate the effectiveness of their OMP approach. The idea is interesting and the paper is easy to follow. \n\nHowever, I have some concerns below:\n\n1.\tI feel the contribution of this work is not sufficient as the orthogonal feature learning has been explored in natural training.\n2.\tIn addition, the baselines are also not enough for the convincing. From the perspective of network structure, this work needs to compare with related work of network structure in adversarial robustness studies, such as Feature Denoising [1]. From the perspective of model ensembling or diverse feature learning (since OMP is an ensemble strategy), this work needs to compare with works on model ensembling or diverse feature learning [2]\n3.\tThe comparison is rather limited. The experiments are only conducted on one dataset, CIFAR-10, and the attacks for evaluation are limited.\n\n[1] Feature Denoising for Improving Adversarial Robustness. CVPR 2019.\n[2] Improving Adversarial Robustness via Promoting Ensemble Diversity. ICML 2019.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: I thank the authors and reviewers for the lively discussions. Although reviewers mentioned the work has potentials to improve adversarial robustness, they agreed that the current draft needs a bit more work specially to strengthen its experimental results and comparisons with related works. 


Decision: Reject