ARMOURED: Adversarially Robust MOdels using Unlabeled data by REgularizing Diversity
Authors: Kangkang Lu
Cuong Manh Nguyen
Xun Xu
Kiran Krishnamachari
Yu Jing Goh
Chuan-Sheng Foo
Venue: ICLR 2021 Poster (2021)
Keywords: Adversarial Robustness
Semi-supervised Learning
Multi-view Learning
Diversity Regularization
Entropy Maximization

Abstract: Adversarial attacks pose a major challenge for modern deep neural networks. Recent advancements show that adversarially robust generalization requires a large amount of labeled data for training. If annotation becomes a burden, can unlabeled data help bridge the gap? In this paper, we propose ARMOURED, an adversarially robust training method based on semi-supervised learning that consists of two components. The first component applies multi-view learning to simultaneously optimize multiple independent networks and utilizes unlabeled data to enforce labeling consistency. The second component reduces adversarial transferability among the networks via diversity regularizers inspired by determinantal point processes and entropy maximization. Experimental results show that under small perturbation budgets, ARMOURED is robust against strong adaptive adversaries. Notably, ARMOURED does not rely on generating adversarial samples during training. When used in combination with adversarial training, ARMOURED yields competitive performance with the state-of-the-art adversarially-robust benchmarks on SVHN and outperforms them on CIFAR-10, while offering higher clean accuracy.

Published as a conference paper at ICLR 2021
ARMOURED: ADVERSARIALLY ROBUST MODELS USING UNLABELED DATA BY REGULARIZING DIVER- SITY
Kangkang Lu".!, Cuong Manh Nguyen --1 Xun Xu',
Yu-Jing Goh Kiran Krishnamachari" 12 & Chuan-Sheng Foo" i.
I Institute for Infocomm Research, S*STRR, Singapore 2 National University of Singapore, Singapore
Lu.kangkang, nguyen.manh.cuomn xu_xunn0irr 1eeiirra-stta aww.aar..r.d..........
yyuj inggoh, kirank]ou mus. edu, foo chuan shengei2r. a a-s.aarsta....... edu.
ABSTRACT
Adversarial attacks pose a major challenge for modern deep neural networks. Re- cent advancements show that adversarially robust generalization requires large amount of labeled data for training. If annotation becomes a burden, can unla- beled data help bridge the gap? In this paper, propose ARMOURED, an ad- versarially robust training method based on semi-supervised learning that consists of two components. The first component applies multi-view learning to simul- taneously optimize multiple independent networks and utilizes unlabeled data to enforce labeling consistency. The second component reduces adversarial trans- ferability among the networks via diversity regularizers inspired by determinantal point processes and entropy maximization. Experimental results show that un- der small perturbation budgets, ARMOURED is robust against strong adaptive adversaries. Notably, ARMOURED does not rely on generating adversarial sam- ples during training. When used in combination with adversarial training, AR- MOURED yields competitive performance with the state-of state-of-the-an offtthe-art adversarially- robust benchmarks SVHN and outperforms them on CIFAR-10, while offering higher clean accuracy.
INTRODUCTION
Modern deep neural networks have met or even surpassed human-level performance on a variety of image classification tasks. However, they vulnerable adversarial attacks, where small, calculated perturbations the input sample can fool a network into making unintended behaviors, e.g., misclassification. (Szegedy al. 2014 Biggio al. 2013). Such adversarial attacks have been found to transfer between different network architectures (Papernot et al. (2016) and are serious concern, especially when neural networks are used in real-world applications.
As result, much work has been done to improve the robustness of neural networks against adver- sarial attacks (Miller et al. 2020). Of these techniques, adversarial training (AT) (Goodfellow al.) 2015 Madry et al. 2018) B is widely used and has been found to provide the most robust models recent evaluation studies (Dong et al. 2020 Croce Hein 2020p. Nonetheless, even models trained with AT have markedly reduced performance on adversarial samples in comparison to clean samples. Models trained with AT also have worse accuracy on clean samples when compared to models trained with standard classification losses. (Schmidt et al. (2018) suggest that one reason for such reductions model accuracy is that training adversarially robust models requires substantially more labeled data. Due to the high costs of obtaining such labeled data in real-world applications, recent work has explored sepi-super AT-based approaches that are able to leverage unlabeled
data instead (Uesato al. 2019 Najañ et al. 2019 Zhai et al. 2019 Carmon et al. 2019).
"Equal contribution. Corresponding author.

---

Published as a conference paper at ICLR 2021
Orthogonal to AT-based approaches that focus training robust single models, few works have explored the use diversity regularization for learning adversarially robust classifiers. These works rely on encouraging ensemble diversity through regularization terms, whether model predictions (Pang et al. 2019) or model gradients (Dabouei et al.] 2020), guided by the intuition that diver- sity amongst the model ensemble will make it difficult for adversarial attacks transfer between individual models, thus making the ensemble as whole more resistant to attack.
In this work, we propose ARMOURED: Adversarially Robust MOdels using Unlabeled data by REgularizing Diversity, a novel algorithm for adversarially robust model learning that elegantly uni- fies superviseee learning and diversity regularization through a multi -vew learning framework. ARMOURED applies a pseudo-label filter similar to co-training (Blum & Mitchell, 1998) to en- force consistency of different networks' predictions on the unlabeled data. In addition, we derive regularization term inspired by determinantal point processes (DPP) (Kulesza & Taskar 2012) that encourages the two networks to predict differently for non-target classes. Lastly, ARMOURED maximizes the entropy of the combined multi-view output on the non-target classes. We show in empirical evaluations that ARMOURED achieves state-of-the-art robustness against strong adaptive adversaries long as the perturbations unlike previous semi-supervised methods, ARMOURED does not use adversarial samples during training. When used in combination with AT, ARMOURED is competitive with the state-of-the-art methods on SVHN and outperforms them on CIFAR-10, 10, while offering higher clean accuracy.
within small &oc & norm-bounded balls. Notably,
In summary, the major contributions of this work follows:
1. We propose ARMOURED, a novel semi-supervised method based on multi-view learning and diversity regularization for training adversarially robust models.
2. We perform an extensive comparison, including standard semi-supervised learning ap- proaches in addition to methods for learning adversarially robust models.
3. We show that ARMOURED+AT achieves state-of-the-art adversarial robustness while maintaining high accuracy on clean data.
RELATED WORK
To set the stage for ARMOURED, in this section, we briefly review adversarially robust learning and semi-supervised learning two paradigms in the literature that are related to our work.
2.1 ADVERSARIALLY ROBUST LEARNING
Adversarial attacks: We consider attacks where adversarial samples stay within lp ball with fixed radius around the clean sample. In this setting, the two standard white-box attacks are the Fast Gradient Sign Method (FGSM) (Goodfellow al.) 2015) that computes a one-step perturba- tion that maximizes the cross entropy loss function, and Projected Gradient Descent (PGD) (Madry) cet al., 2018). a stronger attack that performs multiple iterations of gradient updates to maximize the loss; this may be
a multi-step version of FGSM. Auto-PGI) attack (APGD) (Croce & Hein 2020) is a parameter-free, budget-aware variant of PGD which aims at better convergence. However, robustness against these gradient-based attacks may give a false sense of security due to gradient-masking. This phenomenon happens when the defense does not produce useful gradients generate adversarial samples (Athalye al. 2018). Gradient-masking known to affect PGD by preventing its convergence to the actual adversarial samples (Tramèr & Boneh 2019). There exists gradient-basee attacks such as Fast Adaptive Boundary attack (Croce & Hein 2019) (FAB) which is invariant to rescaling, thus is unaffected by maddient-masking FAB minimizes the perturbation norm as long as misclassification is achieved. Black box attacks that rely on random search alone without gradient information, such Square attack (Andriushchenko et al.) (2020, gradient masking. Finally, AutoAttack (Croce & Hein| (2020) is a strong ensemble adversary which applies four attacks sequentially (APGD with cross entropy loss, followed by targeted APGD with liffereence-o-llogis--ne loss, targeted FAB, then Square).
also unaffected by
Adversarial training: Adversarial training (AT) is a popular approach that performs well in practice (Dong et al., 2020). Madry et al. (2018) formulate AT as a min-max problem, where the model is trained with adversarial samples found via PGD. Variants of this method such TRADES (Zhang
2

---

Published as a conference paper at ICLR 2021
et al. 2019b) and ALP (Kannan et al., 2018) further decompose the error into natural error and boundary error for higher robustness. Zhang et al. (2019)); Wang et al. (2019) theoretically prove the convergence of AT. Two drawbacks of AT are its slow training due adversarial example generation
requiring multiple gradient computations, and the significant reduction in model accuracy on clean samples. Several recent works have focused on speeding up AT (Zhang et al.] [009a Qin al.)
2019 Shafahi et al.) 2019); ARMOURED addresses the second limitation, enabling significantly improved performance on clean samples.
Semi-supervised adversarial training: [Schmidt et al. (2018) showed that adversarial robust gener- alization requires much more labeled data. To relieve the annotation burden, several semi-supervised adversarially robust learning (SSAR) methods have been developed to exploit unlabeled data instead. Uesato et al. 2019) introduced unsupervised adversarial training. a simple self-training model which optimizes smoothness loss and a classification loss using pseudo-labels. Carmon et al. (2019) revis- ited the Gaussian model by Scchmidtt et al. (2018) and introduced robust self-training (RST), another self-training model that computes regularization loss from unlabeled data, either via adversarial training or stability training. Zhai et al. (019)) applied a generalized virtual adversarial training to optimize the prediction stability of their model in the presence of perturbations. Najañ et al. (2019) proposed a semi-supervised extension of the distributionally robust optimization framework byiSinha al. (2018). They replace pseudo-labels with soft-labels for unlabeled data and train them together with labeled data. It is worth noting that all of these four state-of-the-ard SSAR methods apply AT in their training procedure.
Diversity regularization: Diversity regularization is an orthogonal direction to AT that has the po- tential to further improve the performance of AT. In earlier work, Pang al. (2018) showed that for a single network, adversarial robustness can be improved when the features learned for differ- ent classes are diverse. Pang et al. (2019) further developed this concept by introducing Adaptive Promoting Diversity regularization (ADP). Given an ensemble of neural network classifiers, ADP promotes diversity among non-target predictions of the networks. ADP inspired by determinan- tal point processes (Hough et al., 2006), an elegant statistical tool to model repulsive interactions among items of a fixed ground set; applications to machine learning are reviewed in (Kulesza & Taskar 2012). Dabouei et al. (2020) enforce diversity the gradients of individual networks in the ensemble instead of their predictions. We note that unlike ARMOURED, the methods described here are developed for the fully- setting, and are not able to utilize unlabeled data.
2.2 SEMI-SUPERVISED LEARNING
Semi-supervised learning (SSL) is an effective strategy to learn from low-cost unlabeled data. There is considerable recent work in this practically relevant and active research area; we will not be able cover all these works here. Existing SSL methods can be broadly categorized into three groups: consistency-ban -based, graph-based, and generative models. Re- cent methods, such Mean Teacher (Tarvainen & Valpola, 2017) and MixMatch (Berthelot et al.) 2019), are consistency-base performance in practice. The key idea behind consistency-bast based methods is that model predictions on different augmentations of the same input should be consistent.
Semi-supervised learning:
as this approach can be adapted to generic problems and have superior
Multi-view learning: Multi-view learning is SSL paradigm that is capable of representing di- versity in addition to consistency. A dataset is considered to have multiple views when its data samples are represented by more than one set of features and each set sufficient for the learning task. In this setting, a multi-view method assigns one modeling function to each view and jointly optimizes the functions to improve generalization performance (Zhao et al. 2017). By analyzing various multi-view algorithms, Xu et al. (2013) summarized consensus and complementary as the two underpinning principles of multi-view learning. The consensus principle states that a multi- view technique must aim at maximizing the prediction agreement on different views, similar to the consistency-based SSL methods discussed above. The complementary principle states that order to make improvement, each view must contain some information that the other views do not carry, that the views should be sufficiently diverse. This principle has been applied to boost generalization capability in regular SSL QQia et al. 2018) and learning with label noise (Han et al.) 2018). In this paper, we argue that multi-view complementarity also plays a critical role in improving adversarial robustness, by reducing the transferability of adversarial attacks across different views.
3

---

Published as a conference paper at ICLR 2021
3 THE ARMOURED METHOD
In this section, we introduce ARMOURED, our proposed semi-supervised adversarially robust learning method. To utilize both labeled and unlabeled data, ARMOURED adopts multi-view framework where multiple networks output different predictions (posterior probabilities, which will refer to as deep views) on the same input image. The networks are then co-optimized by single loss function computed the deep views. We adhere to both the consensus and comple- mentary principles multi-view learning by ensuring that the deep views maximize their consensus on the target class (the ground truth class for labeled examples), but complement each other the non-target classes. To determine a "target" class for unlabeled samples, ARMOURED applies matching filter pick out target class based agreement between views. Since method is designed for adversarial robustness, we place a greater emphasis on the complementary princi- levels of complementarity;
among the deep views via regularizer based on DPP and (ii) among the non-target classes via an entropy regularization ap- plied on the combined multi output. Following this, we will describe ARMOURED in detail. Pseudocode detailing the training procedure is provided in Algorithm iin Appendix A.1
ple. More concretely, we introduce
7
x
x²
N
2
N
N
y
y
avg
f2
[xx,y)
Training Procedure
(b) Inference Procedure
Figure 1: ARMOURED dual-view framework: (a) training and (b) inference procedures. Solid black and dotted red arrows denote forward and backward passes, respectively; double black arrows represent image augmentation; double-dashed green arrows denote pseudo-label filter.
We describe the general M-view model. Consider semi-supervised image classifi- cation task on input image x and target label y from one of K classes, E {1, 2..,...., .e....,,, Kj. In each minibatch, our training data consists of labeled set {(x,, 9:) ) and an unlabeled set U = {2;) For each input image x, we apply random augmentations to generate M differ- ent augmented images {x (m==1 Let {Nm } JM be architecturally similar neural networks with respective parameters {@mh { 1m=1. Each network takes the corresponding augmented input and pro- duces predictions f*"(x) Nx" "aa"",00"" (0") Vm = 1, ..... M. Due to the different augmentations and network parameters, each output fr can treated as one deep view of the original image æ. Finally, we compute loss function on these deep views and backpropagate to optimize the parameters
Overview:
f(x, y) LCE(E, u) + ADPp<opp y) ANEM
(1)
where ADPP and λNEM are model hyperparameters. We describe each component of the overall loss function, LCE, LDPP and LNEM in the following. At inference time, the M outputs are combined networks possess similar learning capability, the final output computed by averaging the deep views: f(x)
to produce single prediction. Since
m= (x). The detailed inference
procedure is given in Algorithm 2 in Appendix A.1 Figure illustrates the ARMOURED multi-
view framework for the dual-view scenario.
Cross-entropy loss (Cog) and pseudo-label filter: For each labeled sample, we minimize the stan- dard cross-entropy loss LCE(Z, y) > log f (xx). While one may train each deep view in- dependently using only the labeled data, the fact that augmented inputs are generated from the same original image enables to add additional constraint that the deep views should agree with each other even on unlabeled samples. Hence, when all M networks assign the highest probability to the same class, be confident about their prediction the sample. We denote such sample
4

---

Published as a conference paper at ICLR 2021
stable sample and define a pseudo-label for it as y arg max k= Vm I..... M. This pseudo-labeling technique has its roots in co-training (Blum & Mitchell 1998). a multi-view technique that conforms to the consensus principle. After stable sample is confirmed, it is treated as a labeled sample and the cross-entropy loss LeE applies. We recompute pseudo-labels for each minibatch to avoid making incorrect pseudo-labels permanent.
hy
S Ky
y
c fy
airpkan
oirplane
airglare
airplan
sirpfane
akylere
thip
ship
<tip
stop
treds
autorebile
auttmobile
Nid
Nird
hird
hird
hird
dest
dear
deer
deer
deer
ot
cat
cs
frog
flgg
frog
frng
Ire
frog
dog
dog
deg
deg
dog
drg
hune
horee
Tors
hores
(a) Multi-view baseline
(b) Identity kernel
(c) Customized kernel
Figure 2: Three models are trained on CIFAR-10-semi-IK setup. We plot the average prediction of test samples with label "airplane". (a) ARMOURED without DPP regularization: each network predicts randomly on non-target classes. (b) ARMOURED-I: with identity matrix kernel, network predictions on non-target classes are orthogonal.
(c) ARMOURED-HI hand-crafted kernel causes a clustering effect, where each network prefers group of classes, either vehicles animals.
DPP regularization (CDPp): Suppose that the number of deep views is smaller than the num- ber of classes, i.c., (M K). Let F be the K x M matrix formed by stacking the deep views horizontally, i.e., F = [ ...., fdd]. Furthermore, let S be a K x K positive semidefinite kernel matrix that measures the pairwise similarity among the classes. For each sample, we ex- tract Fvg and S the submatrices of F and S that correspond to the non-target classes. Let Fiy denote the normalized F3 where each column is scaled to unit length. Inspired by deter- minantal point processes (Kulesza & Taskar 2012), ARMOURED minimizes the following loss:
(DPP(I, y) = log det F (R
This loss is minimized at Fly F., where F, is the horizontal concatenation of the first M dom- inant cigenvectors of Syy; a proof is provided in Appendix A.2 Since eigenvectors are always orthogonal, LDPP encourages the deep views to make diverse predictions non-target classes. If the kernel matrix is predefined, this result allows us to interpret the non-target predictions implied by the DPP regularizer. Specifically, if the kernel S is constructed by similarity measure over the classes, then clustering effect will be observed, where similar classes are "preferred" by the same view. On the other hand, we can also inject prior knowledge or encourage desired behavior by designing a custom kernel. Exploitation of prior knowledge can be beneficial to generalization, especially when labeled training data are limited.
We note that our DPP regularizer generalizes the ensemble diversity regularizer of ADP (Pang et al., 2019) that uses the identity matrix as its kernel (S = /). we decompose the kernel matrix such that $ = + क. then our DPP regularizer is equivalent to the ADP regularizer applied on linear transformation & F. of the non-target predictions. Again, this linear transformation is another way to regulate the deep views, and can either be learned or predefined. Figure illussrates the difference between the predictions from baseline model vs from ARMOURED models with different kernels. Another related work is cost-sensitive robustness (Zhang & Evans 2018), which uses a cost matrix to weigh different adversarial transformations (attacks) among the classes. Our kernel matrix does not serve the same purpose, but the effects are similar. In our model, network preference would prevent adversarial transformations across different groups of classes.
Non-target entropy maximization (CNEM): Besides the multi-view diversity, we further propose an entropy regularizer that encourages larger margins among non-target classes in the final predic- tions f(x). Specifically, let f.y be the (K 1) x 1 vector of non-target predictions, and f,y be
5

---

Published as a conference paper at ICLR 2021
the normalized vector where the elements sum up to 1. We propose maximize the entropy de- fined over the normalized non-target predictions. Our entropy regularizer is therefore defined as the negative entropy LNEM (z, y) = - -H (f.g)) = - Rg log fig
This loss is minimized when all elements of f are equal to (1-(1- Intuitively, this regularizer acts as a balancing force on the non-target predictions. It prevents ARMOURED from assigning high probability any of the incorrect classes. We note that LNEM differs from the entropy maximization technique adopted in Pang et al. (2019) that encourages a uniform distribution over all K classes. Although our regularizer is similar to the complement objective proposed by Chen et al.] (2019), we extend this technique to semi-supervised learning and provide more theoretical insight we show that entropy maximization increases a lower bound on the average (logit) margin under mild assumptions (Theorem A.2 in Appendix A.3) A.3
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Dataset: We evaluate ARMOURED on the CIFAR-I0 and SVHN datasets. We use the official train/test splits (50k/10k labeled samples) for CIFAR-10 (Krizhevsky et al. 2009) and reserve 5k samples from the training samples for validation set. In our semi-supervised setup, the label budget is either 1k or 4k; remaining samples from training set are treated as unlabeled samples. For the SVHN dataset (Netzer et al. 201T) our train/validation/testt split is 65,932 / 7,325 26,032 samples. We use only k samples as the label budget in our semi-supervised setup for SVHN. For simplicity, will refer to our setup as "Dataset-semi semibbugett,,, e.g., CIFAR-10-semi-44 SVHN-semi-lk.
Ad Adversarial attacks: To evaluate robustness, we apply the following adversaries: (i) Fast Gradi- ent Sign Method attack (Goodfellow al.) (015) (FGSM), (ii) Projected Gradient Descent attack (Madry et al. 2018) (PGD) with random initialization and (iv) AutoAttack (Croce & Hein 2020). For Eoo attacks, the default perturbation budget = 8/255; for Ea attacks, 0.5.
To enable fair comparison, the same Wide ResNet (Oliver et al., 2018) backbone is used for all methods. Specifically, we implement "WRN-28-2" with depth 28 and width 2 along with batch normalization, leaky ReLU activation and Adam optimizer. We train each method for 600 epochs on CIFAR- 1ARR--0-seepr 10-semi-4k and WVHN-semi- 1k. Learning rate is decayed by a factor of 0.2 after the first 400k iterations.
Backbone network and training:
AT wrapper for SSL: We notice that many concepts in SSL, such as multi-view diversity or consis- tency, are orthogonal to AT, and that successful defenses against surge-perurrbation attacks always rely on AT (Croce& & Hein 2020). Therefore, we hope to combine the best of both worlds by imple- menting AT as wrapper method for SSL. Algorithm in Appendix(A.]]eesrribll our Method+AT wrapper, which consists three main steps. First, for each batch of semi-supervised data, we apply the inference procedure of the Method (e.g.. Algorithm of ARMOURED) to generate pseudo- labels for unlabeled data. Second, for each input sample in the batch, we compute its adversarial sample using either the true label (if the sample is labeled) the pseudo-label (if the sample is unlabeled). Third, we execute the training procedure of the Method (e.g.. Algorithm Oo AR- MOURED) using the adversarial samples and the original labels. The pseudo-labels computed from the first step are now dropped, so that the training is still semi-supervised. We note that this wrapper algorithm resembles RST (Carmon al. 2019
We design three variants based on the dual-view model shown in Figure that differ only in choice of diversity kernel. ARMOURED-I is our standard model that uses the Identity matrix as its diversity kernel. ARMOURED-H uses Hand-crafted binary matrix intended to group the classes into two predefined clusters. On CIFAR- 10, these are "vehicles" (airplane, ship, truck, automobile) vs. "animals" " (bird, cat, deer, dog, frog, horse). On SVHN, we split the digits into "simple & edgy" (0, 1, 2, 4, 7) vs. "curvy & loopy" (3, 5, 6, 8, 9). The third variant is ARMOURED-F, which uses a learnable Feature-based kernel. From pre-trained SSL model, we first compute the adversarial samples corresponding to the labeled training samples. Then, for each class, we extract feature vectors by averaging over the adversarial samples associated with the class. Finally, we combine the feature vectors into a matrix B and compute a kernel S = B B. More details of the kernels are provided in Appendix A.4 In our experiments, we evaluate the following
ARMOURED variants:

---

Published as a conference paper at ICLR 2021
four variants: ARMOURED-I+AI ARMOURED H+AT, ARMOURED F+AT and ARMOURED- (trained without AT). For the AT wrapper, we apply a 7-step PGD foo attack with total = 8/255 (for CIFAR-10), c= 4/255 (for SVHN) and step size of e/4.
Comparison benchmarks: We test the proposed method against a wide range of statc-of-the- ahtt SSL and SSAR benchmarks: Mean Teacher (MT) (Tarvainen & Valpola 2017). MixMatch (Berth- jelot et al. 2019). RST (Carmon et al.. 2019) (RST has two variants, we implemented RSTadv). and the method of ZZai et al. (2019) that we denote as ARG. In addition, we combine MT with adversarial training (MT+AT) using the wrapper Algorithm in Appendix A To the best of our knowledge, this the first time MT+AT has been evaluated for adversarial robustness. For AT- based methods (RST. ARG), we use a 7-step PGD loo attack in their AT phase, similar to MT+AT and ARMOURED+AT.
4.2 RESULTS
Results on CIFAR-10 (Table Figure On clean data, MixMatch yields the best performance, while ARMOURED-F surpasses all methods trained with AT by large margins (18%-26%) and is even better than MT - a SSL method. ARMOURED variants demonstrate substantially higher clean performance over the SSAR benchmarks. Under standard FGSM and PGD attacks, the most robust defense is still ARMOURED-F, followed by its AT based variants with accuracy drops of 2%-5%. Other methods shows larger gaps: 25%-35% for SSAR and 10%-50% for SSL benchmarks. We note that the improvements by ARMOURED are not due to gradient masking (see Appendix B Under AutoAttack, ARMOURED-F is no longer robust, instead, ARMOURED+AT variants are more resilient. ARMOURED-F+AI is the best defense, outperforming ARG by 5.23% for loo and 9.85% for f- attacks. We also notice that the two best defenses against AutoAttack are trained with the hand-crafted and feature kernels. The former requires only human knowledge while the latter just needs additional computing resources, giving our method flexible ways to boost adversarial robustness with or without prior knowledge.
we plot the robust accuracy against AutoAttack as the perturbation budget gradually In Figure 3
increases. ARMOURED-F obtains highest accuracy clean data as well as for small perturbation budgets, but its accuracy drops rapidly as is increased. Mcanwhile, the ARMOURED+AT variants are able achieve a better trade-off between clean accuracy and robustness.
Table I: Benchmark results on CIFAR-1O-semi
Method
Clean
FGSM Ex.
PGD Ex
PGD &
AutoAttack &x.
AutoAttack &
MT
84.55 + 0.64
14.00 + 1.70
0.03 + 0.02
19.55
0.26
0.01 + 0.01
0.46 + 0.02
MixMatch
89.95 = 0.96
57.42 1: 2.78
0.25 + 0.14
9.10 :: 0.82
0.00 :: 0.00
0.01 0.01
RST
58.81
33.80 0.20
31.28 0.26
43.70 0.33
25.38 0.15
39.79 0.60
ARG
67.07 : 0.22
36.65 + 1.79
32.12 = 1.74
42.95
0.82
30.01 + 1.85
40.86 : 0.69
MT+AT
61.77 : 0.28
30.50 + 3.16
25.76
1 3.16
41.31 1.76
24.28 + 2.90
39.93 + 1.63
ARMOURED-F
84.90 +: 0.22
68.27 +: 1.53
56.42 3.99
67.45
1 2.76
8.70 + 1.27
25.80 + 2.50
ARMOURED-I-AT
77.73 0.11
63.56 1.71
51.22 3.68
61.95
2.11
29.22 - 3.18
44. 41.11 +
ARMOURED-H+AT
76.74 + 0.73
63.41
+ 1.94
54.94 + 4.30
61.89 2.74
31.88 + 7.19
46.29
+ 5.84
ARMOURED F+AT
76.76 +: 1.60
64.05
3.00
55.12
4.90
61.93 3.06
25.24
4.56
49.78 3.80
On clean test samples, ARMOURED-F yields the best performance, with a small improvement over the second best competing method, Against FGSM and PGD at- tacks, even the worst ARMOURED variant more robust than MT+AT (the best benchmark) by 13%-20%. Under AutoAttack, ARMOURED-H++ gap of 15% under lo attacks, while outperforming them by 4%-11% under fz attacks. Overall, ARMOURED shows competitive performance compared to ofte-of-the-art SSL and SSAR bench- marks. Results on CIFAR- 10 and SVHN suggest that MT+AT is a strong defense.
Results on SVHN (Table :):
falls behind MT+AT and ARG by significant
Ablation study (Table 3):
We perform an ablation study to investigate the contribution of each component to the performance ARMOURED-F-a FA agaisnt Attokt First, we remove both DPP and entropy regularization terms from the total loss in equation (T). This model, denoted as w/o
Each result contains mean and standard deviation statistics computed from three independent runs with different random data seeds (for selecting labeled samples).
7

---

Published as a conference paper at ICLR 2021
AutoAttack (fw)
AutoAttack (f2)
70
Acuoracy (%)
Acucracy (%)
50
40
40
20
30
0
1/255
2/255
4/255
8/255
200
0.1
0.3
0.5
Budget
Budget E
MT+AT
RST
ARMOURED-HAT ARMOURED-H-AT
ARMOURED-FoAT
ARG
ARMOURED-F
Figure 3: Robustness against AutoAttack vs. perturbation budget on CIFAR-1O-semi
Table 2: Benchmark results on SVHN-semi-lk
Method
Clean
FGSM
PGD &
PGD
AutoAttack 6x0
AutoAttack fg
MI
93.73 0.96
24.44 2.13
4.14 0.81
34.64 :: 3.12
1,50 +: 0.37
12.43 1.53
MixMatch
91.35 1.00
43.75 +: 23.50
0.05 0.05
3.72 à: 3.31
0.00 + 0.00
0.21 = 0.20
RST
55.67 1.79
30.40 +: 1.30
25.99 1.79
33.71 +: 3:32
17.33 + 1.94
25.08 3.79
ARG
0.51
58.22 0.95
44.42
1.11
1.07
39.50
MT+AT
92.71 0.22
1.02 58.45
44.70 = 0.42
50.42 : 1.02
38.06
0.11
41.71 ::1::00 0.93
ARMOURED
93.93 + 0.79
72.11
1.67
57.91 + 5.08
73.44 + 3.18
7,88 + 0.87
24.62 + 1.38
ARMOURED-I-AT
92.01 0.44
74.22
1.32
62.04 4.93
71.38 - 2.28
22.04 3.04
41.57 2.45
ARMOURED-H-AT
92.53 0.35
73.54 + 1.58
61.11
1.14
70.53
0.71
1.30
45.
ARMOURED-FAAR F+AT
92.44 0.64
74.78 4.37
62.10 = 8.39
71.37 = 6.32
23.35 1: 3.22
44.41 + 3.20
(CDPP LNEM), performs relatively well on clean data, but its performance suffers under attacks, dropping by 26% for lo and by 22% for l attacks. We then keep the term LNEM, but remove the diversity regularizer from the loss functior [ This model w/o LDPP performs worse than the complete model by 1%. We conclude that the entropy regularizer plays a more vital role than the DPP regularizer. Besides, we train ARMOURED-F+AT using only the 4k labeled samples and call this model w/o Unlabeled. Its poor performance reinforce the importance of unlabeled data towards improving adversarial robustness. Finally, we include ARMOURED-F (trained without AT), which performs very well on clean data but fails against AutoAttack. Additional results are provided in Appendix
Visualization of learned representations (Figure 4) On CIFAR-10 test samples, we visualize the feature embeddings (extracted from the last layer WRN-28-2 hefore the linear layer) learned by the four models in ablation study. On clean test samples, ARMOURED-F produces produces the best em- beddings. On adversarial samples, we observe gradual improvements in the representations, starting from (a) diversity regularization to (b) diversity on only labeled samples, with network N show- ing well-defined clusters; then (c) diversity on whole training set with better cluster separation and (d) combining diversity with AT, where clusters less contaminated under attacks.
CONCLUSION
In this work, we presented ARMOURED, novel method for learning adversarially robust models that unifies semi-supervised learning and diversity regularization in multi-view framework. AR- MOURED alone is robust against standard white-box attacks as well as strong adaptive attacks with
The DPP regularization term cannot function properly without entropy regularization, due trivial opti- the one-hot vector 1,, shown by Pang et al. (219). Hence, we must keep LNEM-
8

---

Published as a conference paper at ICLR 2021
Table Ablation study on CIFAR-1O-semi-44
Method
Clean
AutoAttack foo
AutoAttack &2
w/o (LDPP + (NEM)
75.60 0.44
9.27 + 0.28
27.32 0.85
w/o LDPP
77.44 0.85
34,02 + 2.80
48.52 1.79
w/o Unlabeled
72.50 0.21
10.26 + 1.57
25.23 0.59
ARMOURED-F
84.90 + 0.22
8.70
25.80 2.50
ARMOURED-FHAT
76.76 1.60
35.24 4.56
49.78 = 3.80
Clean Samples
Adversarial Samples
Network I
Network N
Network N
Network N
(Cppp ENEM) w/ Unlabeled
(b)
ARMOURED-F ARMOURED-T-AT
(c)
Mirclane
automoble
bind
doer
froq
horeo
truck
samples generated by ablation models: (a) w/o (LDrr + LNEM), (b) w/o Unlabeled, (c) ARMOURED-F and (d) ARMOURED- F+AT. For each of the network/method pairs, the clean and adversarial samples are processed together in single t-SNE run. Adversarial samples are generated with PGD-Gxc (e 8/255). From (a) to (d), the embeddings of adversarial samples are progressively enhanced, while (c) yields best representations on clean data.
Figure 4: t-SNE plots of feature embeddings from CIFAR-
small perturbation budgets. When combined with adversarial training, ARMOURED demonstrates much better robustness against a wider range of perturbation budgets. Additionally, ARMOURED improves clean accuracy when compared with state-of-the-ee semi-supervised adversarial training methods. The empirical performance of ARMOURED+AT suggests that it is possible to learn ad- versarially robust models while upholding a reasonable accuracy on clean samples. Extending this method to exploit more than two views alternative custom kernels for the DPP regularizer could result further performance gains.
ACKNOWLEDGMENTS
This work is supported by the DSO National Laboratories of Singapore. The authors would like to thank the DSO project team. in particular Dr. Loo Nin Teow and Dr. Bingquan Shen for valuable discussions adversarial robustness.

---

Published as a conference paper at ICLR 2021
REFERENCES
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square at- tack: a query-efficient black-box adversarial attack via random search. In European Conference on Computer Vision, pp. 484-501. Springer, 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. arXiv preprint arXiv: 180,:00000000 2018.
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach semi-supervised learning. In Advances in Neural Information Processing Systems, pp. 5050-5060, 2019.
Battista Biggio, Igino Corona, Davide Maiorea, Blaine Nelson, Nedim Sındić, Pavel Laskov, Gior- gio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 387- 402. Springer, 2013.
Avrim Blum and Tom Mitchell. Combining labeled and unlabeled data with co-training, In Pro- ceedings of the eleventh annual conference on Computational learning theory, pp. 92-100. ACM, 1998.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. Unlabeled data improves adversarial robustness. In Advances in Neural Information Processing Systems, pp. 11192-11200 1203, 2019.
Hun Chen, Pei-Hsin Wang, Chun-Hao Liu, Shih-Chich Chang, Jia- Yu Pan, Yu-Ting Chen, Wei Wei, and Da-Cheng Juan. Complement objective training. In International Conference on Learming Representations, 2019.
Thomas M Cover and A Thomas. Determinant inequalities via information theory. SIAM journal on Matrix Analysis and Applications, 93):384-392, 1988.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. arXiv preprint arXiv: 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter- free attacks. arXiv preprint arXiv:2003.0/69, 2020.
Ali Dabouei, Sobhan Soleymani, Fariborz. Taherkhani, Jeremy Dawson, and Nasser M. Nasrabadi. Exploiting joint joint robustness to adversarial perturbations. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2020.
Yinpeng Dong, Qi-An Fu, Xiao Yang, Tianyu Pang, Hang Su, Zihao Xiao, and Jun Zhu. Benchmark- ing adversarial robustness on image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
Bo Han, Quamming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Advances in Neural Information Processing Systems, pp. 8527-8537, 2018.
J Ben Hough, Manjunath Krishnapur, Yuval Peres, Bálint Virág, et al. Determinantal processes and independence. Probability surveys, 3:206-229, 2006.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. arXiv preprint 1oXYv:1803.06333. 2018
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
10

---

Published as a conference paper at ICLR 2021
Alex Kulesza and Ben Taskar. Determinantal Point Processes for Machine Learning. Now Publish- ers Inc., Hanover, MA, USA, 2012. ISBN 1601986289.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
David Miller, Zhen Xiang, and George Kesidis. Adversarial learning targeting deep neural network classification: A comprehensive review of defenses against attacks. In Proceedings of the IEEE, volume 108, pp. 402-433, 2020.
Amir Najafi, Shin-ichi Maeda, Masanori Koyama, and Takeru Miyato. Robustness to adversarial perturbations in learning from incomplete data. In Advances in Neural Information Processing Systems, pp. 5541-5551, 2019.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning. 2011.
Avital Oliver, Augustus Odena, Colin A Raffel, Ekin Dogus Cubuk, and Ian Goodfellow. Realis- tic evaluation deep semi-supervised learning algorithms. In Advances in Neural Information Processing Systems, pp. 3235-3246, 2018.
Tianyu Pang, Chao Du, Yinpeng Dong, and Jun Zhu. Towards robust detection adversarial exam- ples. In Advances in Neural Information Processing Systems, pp. 4579-4589, 2018.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning, pp. 4970-4979, 2019.
Nicolas Papernot, Patrick McDaniel, and lan Goodfellow. Transfcrability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277 2016.
Kaare Brandt Petersen and Michael Syskind Pedersen. The matrix cookbook, nov 2012. URL htp::/www2. imm. dtu. dk/pudd//p. php, 3274:14, 2012.
Siyuan Qiao, Wei Shen, Zhishuai Zhang, Bo Wang, and Alan Yuille. Deep co-training for semi- supervised image recognition. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 135-152, 2018.
Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli. Adversarial robustness through local linearization. In Advances in Neural Information Processing Systems, pp. 13847-13856, 2019.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry, Adver- sarially robust generalization requires more data. In Advances in Neural Information Processing Systems, pp. 5014-5026 2018.
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Advances in Neural Information Processing Systems, pp. 3358-3369, 2019.
Aman Sinha, Hongseok Namkoong, and John Duchi. Certifiable distributional robustness with principled adversarial training. In International Conference on Learning Representations, 2018.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, 2014.
Antti Tarvainen and Harri Valpola. Mean teachers better role models: Weight- averaged consis- tency targets improve cemi-supervised deep learning results. Advances in Neural Information Processing Systems, pp. 1195-1204, 2017.

---

Published as a conference paper at ICLR 2021
Florian Tramèr and Dan Boneh. Adversarial training and robustness for multiple perturbations. In Advances in Neural Information Processing Systems, pp. 5866-5876, 2019.
Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama. Lipschitz-margin training: Scalable certifi-
cation of perturbation invariance for deep neural networks. In Advances in Neural Information Processing Systems, pp. 6541-6550, 2018.
Jonathan Uesato, Jean-Baptiste Alayrac, Po-Sen Huang, Robert Stanforth, Alhussein Fawzi, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In Advances in Neural Information Processing Systems, pp. 12214-12223, 2019.
Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In International Conference on Machine Learning, pp. 6586 6595, 2019.
Chang Xu, Dacheng Tao, and Chao Xu. A survey on multi-view learning. arXiv preprint arXiv:1304.5634, 2013.
Runtian Zhai, Tianle Cai, Di He, Chen Dan, Kun He, John Hopcroft, and Liwei Wang. Adversarially robust generalization just requires more unlabeled data. arXiv preprint /906. 2019.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: accelerating adversarial training via maximal principle. In Advances in Neural Information Processing Systems, pp. 227-238, 2019a.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. International Conference on Machine Learning, pp. 7472-7482, 2019b.
Xiao Zhang and David Evans. Cost-sensitive robustness against adversarial examples. arXiv preprint arXiv:1810.09222. 2018
Jing Zhao, Xijiong Xie, Xin Xu, and Shiliang Sun. Multi-view learning overview: Recent progress and new challenges. Information Fusion, 38:43 54, 2017.
12

---

Published as a conference paper at ICLR 2021
APPENDICES
A ARMOURED METHOD
A.1 DETAILED PSEUDOCODES
Algorithm 1: ARMOURED Minibatch Training Procedure
Input: Labeled samples L = { 11)1; unlabeled samples U {2; )= kernel matrix S; random augmentation vy(x); hyperparameters (APPp, ÂNEM)
Output: Networks { (N m==1 with updated parameters {0m M
m==1
for i 1, ..... ny do
for m = 1,.... M do
m (12;) // random augmentation
f"xx,) = N .m) // forward pass end
if' ýt arg m k f (z.) Vm I...., M then
add (zt, 0:) to L
remove z; from U
end end
for
i
I,.... 14 do
for
=1,- 1.... M do
x = ()(x;) // random augmentation ("((:) N (x 0m) // forward pass
end
C( V:) = (Z y1) + CDPP vi) + ANEMCNEM(EU v.) // sample loss end
-= - L(x,, v1) // batch loss
backpropagate L to optimize {0mh L // backward pass
Algorithm 2: ARMOURED Inference Procedure
Input: Sample x; networks {N' J.M with parameters {0mh JM Jm= random augmentation r7(x) Output: Posterior output f(r); predicted label y
for 1, .,.... M do TT = (ς() // random augmentation ('(()) (r) (n (n) // forward pass
end
M ΣM (x) // posterior output i = arg max 1 (z(x) // predicted label
f(x)
A.2 OPTIMA OF DPP REGULARIZER
For simplicity, we find the maximum of the exponential of negative loss LDPP(E, v). defined as
Q v) = exp [-pprp( y)] = det (FT
(2)
Since Sty is principal submatrix of S, it is also positive semidefinite. We can decompose S.y as follows: Syy V where is square matrix whose k-th column the eigenvector ey of Sgg and D diagonal matrix whose (k, k-)th element X is the k-th largest eigenvalue of i S.y-
13

---

Published as a conference paper at ICLR 2021
Algorithm 3: Method+AT Minibatch Training Procedure
Input: Labeled samples L {(1,, Hi) unlabeled samples {z, )=
SSL technique Method with hyperparameters Methood adversarial attack (2; y)
Output: Method model with updated parameters
create Ladv = 9 and Uadv = - // new empty sets for i I,..., n do
( V4) // adversarial sample add (2s, y4 to Ladv
end
for i I,..., do
apply inference procedure of Method æ; to generate pseudo-label y/ π (1:z (1) // adversarial sample
add (2,) to Uadv
end
execute training procedure of Method with inputs: Ladv; Uadv; Method
The gradient of Q with respect Fly is given by Petersen & Pedersen 2012)
0Q = 2det(F
o R
OF
Let F. be the horizontal concatenation of the first M eigenvectors, i.e., F. [0], ...... var]
Notice that F.T F. Das where DM is the M x M leading principal submatrix of D. We evaluate the gradient at F. as follows
aQ
2 F. F. P.)
(4)
0F 1/ F.
-ddet(DM) 8 y. F.D
=2det(D D D
(6)
Interestingly, since DM a diagonal matrix, det(Dxx equals the product of the first M eigenval- ues of Sty This product is also nonnegative because Sty is positive semidefinite. Therefore, the gradient at F. is a nonnegative scaling of F. itself. Since F, is normalized unit length, adding this gradient does not update any further, i.e., the angular gradient at F. is zero. As shown by Cover& & Thomas (1988), given fixed positive semidefinite kernel, the determinant in equation is a concave function of F.g. Thus, F. is maximum of Q-
Note that F. is the only maximum. Let R be M x M orthogonal matrix, that F.R is rotation of F.. Then, F,R R. is also a maximum of Q, because
F. (RR)
RT SF. F.). R DM R R DM F.T S.y.F.
(8)
This means that a family maxima exists for Q, which includes F, and its orthogonal transforma- tions in the M -ddimensional subspace spanned by F.. For example, when
2, objective Q is maximized at F. = =[1, v2]
([u) ST of ST (
det (i)
X 2) = \1 (9)
det 11 Syy [e]
02] det
U2
S
fa S
Any rotation of (u1, U2) in the 2-dimensional plane spanned by them is also maximum.
14

---

Published as a conference paper at ICLR 2021
A.3 ANALYSIS OF TON-TARGET ENTROPY MAXIMIZATION
For ease of exposition, we denote g()) as the unnormalized logits of f(r), the Lipschitz constant Ln as the scalar satisfying,
Il g0x + +)ll2 x Lx|lell2
(10)
The guarded adversarial area (suzuuu al. 2018) is defined the hypersphere satisfying the following condition, where e is the largest perturbation radius measured in lo distance
YE ||elpp S + (r + E) 2 max +c)
(11)
The max/average logit gap is the gap bewteen target class logit and maxmimal/average non target class logit,
margap(x) = (x)) max (a()),
aeggap(x) = By (y(r) avg (x) kyl
(12)
We start by introducing the following lemma which is related Proposition 1 of Tsuzuku et al. 20188.
Lemma A.1 For any adversarial perturbation e smaller than the logit gap divided by the Liptschitz constant, it is guaranteed the class prediction does not change.
Proof. Lemma A.1 can be written as the following.
margap(x) 9y(x) max (x) 2 VLLx|l|| = By +c) -max)mmma---------a- (gx + +c) c)}20 (13) key
proof that Lemma A. is the same with the proof for Proposition Tsuzuu al. (0018).
This lemma suggests that it is possible to increase the robustness, the guarded adversarial area by either decreasing the Lipschitz constant and/or increasing the logit gap. as with the analysis in Tssuzuu et al.) (2018), that the Lipschitz constant for large neural network very hard to quantify. Instead we find is easier to enlarge the logit gap by non-target entropy maximization and reveal relation between them as follows,
often acknowledged,
Theorem A.2 The non-target entropy H (f...) is a lower bound of average logit gap plus a constant.
The entropy maximization term will encourage a uniform distribution over non-tare non-target classes, i.e. margap(ı) = avggap(z)) By referring to Lemma A.1 this theorem suggests maximizing non- target entropy (ff,,) leads to higher guarded adversarial attack area e. As result, the overall robust- ness to adversarial attack is improved by introducing the additional non-target entropy maximization loss.
Proof. We first write the theorem to prove as the following:
H S (x) avg((gn (x)) +C
(14)
kzy
Before we provide the proof, we introduce the following two lemmas and make mild assumption: Lemma A.3 LogSumExp smooth approximation to and upper bounded by the maximum func- tion plus a constant.
log
exp gk
max gk +log(K -1)
(15)
kty
kzy
Proof. We relax the summation with maximization and arrive at the following inequalities.
log
exp < log(((K - -1) eexp(maxx gk)) key
k
(16)
max 9k + log(( 1) kfg
Lemma A.4 The following inequality holds for real number vector g of length K.
9k exp gk
avg gk <
(17)
Σ
exp gke
15

---

Published as a conference paper at ICLR 2021
Proof. W.l.o.g. we assume 9: is in descending order, i.. Vi j, 9i 2 9/. The proof is rewritten as,
91 (exp 91 + ...... w...eppppp............. gk) + gK (exp 91 + ................................ gx) 2 Kg exp 91 + Kgk expeexpae gK (18)
The difference between RHS and LHS is written as,
RHS LHS (exp 91 cxp exp 92) (exp - expps)(0 ga)+
(19)
.... + (exp 91 exp gk)+
(20)
(exp - exp (12 93) (exp exp 94)+
(21)
(exp exp (92 gN)+
(22)
(exp ge gK-1 exp 9K (gK-1 gk)
(23)
Obviously, RHS - LHS is non-negative, thus the inequality holds.
Assumption A.5 Assume the clean samples are mostly correctly classified.
max gn (x) S (2)
(24)
kØy
Given the fact that we can achieve relatively high classification accuracy on clean samples, the assumption is realistic in most cases.
Now we prove the inequality for equation 14 holds.
H H(E.
exp Ok
log
exp Ik
(25)
Σ
exp gk
Σ
exp 9k
exp 9k
(log
exp 9k gk)
(26)
exp gk
kgy
exp (max Ib + log(K - 1) - 9k)
(27)
exp
key
1 k2
exp Jk
(gv + logg(K - 1) - gk)
(28)
Σ exp gb
K kyy
exp gk
(gy log(K - 1)) avg 9h
(29)
exp gk
kg
< (gg avg 9k + log( K 1])
(30)
kØy
A.4 IMPLEMENTATION DETAILS
Hyperparameters: We fine tune Appp and ANEM with ARMOURED.I+AT model trained on CIFARIO and SVHN individually. The tuning ranges are as follows: AdpP E [0.25, 0.5, 1] and ANEM [1, 2, 4). Each model is trained with one seed and is evaluated on the standard validation set (5k labeled samples for CIFAR- 10 and 7325 labeled samples for SVHN), Please see Table A.4
and Table A.5 for the numerical results. After tuning, we decide to apply (ADPP, ANEM) (1, for SVHN and (ADrr, ANEM) (1, 0.5) for CIFAR-10.
Random augmentations: Regarding the random augmentations 17(x), we apply translations and horizontal flips on CIFAR-10 10 images and apply only random translations for SVHN images.
Feature-based kernel: We learn the feature-bases kernel following the steps below. The learned kernels are plotted in Figure A.5
1. Train a MT+AT model using both labeled and unlabeled training data.
2. Using the teacher model, feed forward the adversarial samples generated from labeled train- ing data. Extract feature vectors from the last layer of WRN-28-2 before the linear layer.
16

---

Published as a conference paper at ICLR 2021
3. For each class, compute the average feature vectors: b1, b2,... bg.
4. Normalize each feature vector by its L-2 norm.
5.
Generate the feature matrix bb1 ... bx] and compute the kernel as B B.
6.
Normalize $ by its largest eigenvalue (cquivalent L-2 normalization).
Table A.4: Fine-tuning of ADPP and ANEM on CIFARIO-semi (reporting validation accuracy)
Autoattack Exc
Autouttack E
ANEM
ANEM
2
4
APPF
Appp
0.25
28.50
25.67
6.48
0.25
44.08
41.56
20.02
0.5
32.60
26.25
6.45
0.5
48.08
41.21
21.04
32.45
27.87
7.83
46.07
41.29
22.80
Table A.5: Fine-tuning of ADPP and ANEM on SVHN-semi-1k (reporting validation accuracy)
Autoattack too
Autoattack &
ANEM
ANEM
4
ADPP
ApPp
0.25
24.42
22.46
7.70
0.25
44.38
41.56
24.38
0.5
22.82
16.53
8.41
0.5
42.62
41.21
24.15
25.15
24.83
7.60
1
44.31
21.74
(a) CIFAR-IO-semi-ae
(b) SVHN-semi-IK
(c) CIFAR-IO-semit
Figure A.5: Visualization of the learned feature-based kernels.
B SUPPLEMENTARY RESULTS
Additional results on CIFAR-10-Semi--e (Table B.6): In this table, we provide the numerical results that are plotted in Figure 3
Additional results on SVHN-semi-lk (Table B.7 Figure B.6): Here, we evaluate the robustness of ARMOURED variants and SSAR benchmarks against AutoAttack with varying perturbation bud- gets. The results show that MT+AT achieves the best robustness. Among ARMOURED variants, ARMOURED-H+AN and ARMOURED-F4AT are the most robust and are comparable to each other.
Additional results from ablation study (Table B.8): this table, we report the full evaluation results from ablation study, adding results from standard attacks. In addition, create model w/ Hf) by replacing H (f. in LNEM by the entropy of the averaged prediction f over all classes, similar to the term used by Pang et al. (2019). This model less robust than ARMOURED- F+AT against AutoAttack, suggesting that entropy regularization is better.
Check on gradient masking (Table B.9 We evaluate ARMOURED-F, ARMOURED FFAT and other benchmarks against individual components of AutoAttack. The results show that both
17

---

Published as a conference paper at ICLR 2021
Table B.6: Benchmark against AtutoAttack with varying budgets on CIFAR-10-semi-44
AutoAttack foo
AutoAttack &y
Method
1/255
2/255
4/255
RST
54.38 0.30
50..80
0.32
51.34 0.37
47.44
0.39
ARG
62.57 + 0.23
57.83
0.3G 53.94
48.09 = 1.05
02.27 + 0.14
57.11 = 0.20
51.83 +: 0.53
MT+AT
59.26 - 0.55
1.13
43.12 + 1.99
59.80 +: 0.46
54.91
+ 0.73
49.86 :: 1.21
ARMOURED-F
70.72 0.88
67.36
1.59
33.52
1.93
71.93
0.65
59.40 = 0.93
16.4511888
ARMOURED-I+AT
65.26 = 1.07
60.43
1.42
48.
65.95 + 1.11
61.43 -: 1.60
55.53
ARMOURED.HANN H+AT
65.20 1.39
60.36
2.64
49.79 + 5.31
65.72
61.61 + 2.70
56.14 8.53
ARMOURED-FAAE F+AT
66.75 2.01
62.14 + 2.33
52.99
+ 3.53
65.98 + 1.90
62.94 = 2.34
58.77
2.90
Table B.7: Benchmark against AtutoAttack with varying budgets on SVHN-semi-Ik
AutoAttack Ex
AutoAttack E
Method
1/255
2/255
4/255
0.1
0.2
0.3
RST
50.23 + 1.88
44.76
2.07
34.25 + 2.08
49.74 +: 2.26
43.65 + 2.77
37.33 +: 3.16
ARG
88,25 - 0.32
83.52
0.21
70.55
0.72
85,99 + 0.31
76.37 0.76
62.31
1.22
MT+AT
89.37 0.33
84.53
0.55
71.42 0.70
87.99
0.45
80.51
0.65
69.26
1.04
ARMOURED&F
81.54 1.46
64.45
2.08
33.68
2.07
82.37 + 1.14
66.33 + 1.31
49.34 1.18
ARMOURED-I+AT
83.55 0.21
73.85 # 1.83
51.96
2.96
83,80 +: 0.07
74.17 +: 1.33
63.09 + 1.98
ARMOURED -HAAT
84.58 + 0.34
75.37
0.988
55.49 + 1.87
85.09 + 0.17
76.25
+ 0.66
66.27 1.40
ARMOURED-F4AT
-4.78
75.59
2.03
54.64 + 3.42
85.11 + 1.09
76.24 + 1.85
65.71 +
ARMOURED-F and ARMOURED-F+AT are very robust against black-box attacks (FAB and Square), which suggests that
is less likely to exist in our models.
Utilization of unlabeled data (Table B.10): We define the utilization rate as the ratio between the number of stable samples and the total number of unlabeled samples in each minibatch. For each setup, we report the average utilization rate in the last 1000 training iterations. While the utilization rates CIFAR-1O-Semi-- semi-1k (65%-80%) and SVHN-semi- Ik (about 80% except for ARMOURED-F) We suspect that the low utilization rates negatively affect the performance of ARMOURED but was not able to conduct further investigation on this issue.
are high (about 90%), they are much lower CIFAR-10-
Regularization effect of DPP kernel (Figure B.7 We illustrate the average prediction of test sam- ples generated by ARMOURED variants. From all the subplots, we can clearly see that each network has developed preference on high or low posterior for each class. For example, in Figure B.7b network 2 (right side) tends to have high predictions for airplane, automobile, ship and truck, while network N (left side) has higher predictions on the remaining is promoted by the hand-crafted kernel. The feature-based kernel (Figure B.7c and Figure (B77d) encourages a similar grouping of classes, even though the distinctions are less severe. With the identity matrix as kernel, the predictions in Figure B.7a also form two groups, but the correlation among classes of the same group are less intuitive.
classes. This behaviour
Results CIFAR-10-semis (Table B.11 Table B.12 Figure B.8 We conduct experi- ment on CIFAR-IO-semi- 10-semi-lk setup. The results in Table B.IT show that ARMOURED variants achieve higher clean accuracy and better robustness against standard attacks, when compared to SSAR benchmarks. However, under AutoAttack, the best benchmark (ARG for fos and MT+AT for (2) outperforms ARMOURED-F+AT by about 3%. We suspect that the drops in the performance of ARMOURED are due to low utilization rate (see Table I0). but were not able to investigate this issue further. In addition, we plot the robustness against Attokt for varying perturbation budgets in Figure B.8 Similar to the results on CIFAR- 10--smm--44,, ARMOURED-F shows better performance than SSAR benchmarks under small attacks.
18

---

Published as a conference paper at ICLR 2021
AutoAttack (2.))
AutoAttack (!2)
100
100
90
80
80
Acuoracy (%)
Acucracy (%)
70
40
20
30
%
20
2/255
B/255
Budget E
Budget
MT+AT
RST
ARMOURED-HAT ARMOURED-HHAT
ARMOURED-F+AT
ARG
ARMOURED-F
Figure B.6: Robustness against AutoAttack vs. perturbation budget on SVHN-semi-Ik.
Table B.8: Ablation study on CIFAR-10-semie (full results)
Method
Clean
FGSM foc
PGD 8
PGD
AutoAttack &
AutoAttack f
w/o (Copp + (XRM)
75.60 +: 0.44
42.19 + 0.40
18.96 + 0.22
10.05
0.91
9.27 = 0.28
27.32 :: 0.85
w/o Cpep
77.44 +: 0.85
66.42 + 1.27
59.65
2.07
65.18
1.14
34.02 = 2.80
48.52 + 1.79
H[()
76.91 + 0.71
63.81
35.83
3.70
1
31.20 6.22
45.07 6.48
w/o Unlabeled
72.50 +: 0.21
41.66
1.29
22.26
+ 2.54
36.29
1.42
10:26 + 1.57
25.23 + 0.59
ARMOURED-F
84.90
0.22 76.76
68.27
1.53
56.42
+: 3.93
67.45
2.76
8.70 1.27
25.80
2.50
ARMOURED-F+AT
61.05
3.00
35.12
4.90
61.93
35.24 4.56
49.78
3.80
Table B.9: Benchmark against components of AutoAttack on CIFAR-10-semi--
Attack
MT+AT
ARG
RST
ARMOURED-F
ARMOURED-FAAT
Clean
64.77
0.28
67.07 0.22
58.81
0.28
84.90 0.22
76.76 1.60
APGD (8/255)
24.68 1 3.03
30.98 1 1.78
30.66
0.38
20.24 2.57
44.51 1 3.97
FAB-t (8/255)
24.63 2.95
30.34
1.81
25.62
0.18
44.17 + 0.82
62.60 + 2.25
Square (8/255)
29.84 2.994
35.37
0.22
77.90
70.99 1.37
AutoAttack (8/255)
24.28 2.90
30.01
1.85
25.38 0.15
8.70 1.27
35.24 4.56
APGD (0.5)
40.48 1.72
41.71 0.66
43.05
0.41
41.25 2.76
57.62 +
FAB-t (0.5)
40.13 1.63
41.15 0.70
39.94
0.56
50.14 1.29
65.69 = 1.75
62
Square (0.5)
51.87 1. 1.12
54.26 1 0.58
48.76
0.26
82 05 0.46
72.49 1.32
AutoAnack (0.5)
39.93 1.63
40.86 + 0.69
39.79 0.60
25.80 + 2.50
49.78 + 3.80
Table B.10: Utilization rate of unlabeled data
CIFAR- 10-semi-4k
SVHN-semi-Ik
CIFAR-IO-semi-
ARMOURED-IsAR ARMOURED- F-AT ARMOURED-F
87.97 : 0.86
83.79 1 6.94
81.93 1: 7.04
1.20
2.33
87.95 + 1.01
82.28 + 0.58
65.84 + 6.13
96.55 = 1.22
94.35 0.17
78.08 2.08
Table B. 11: Benchmark results on CIFAR-I0-semi-
Method
Clean
FGSM f
PGD &x
PGD
AutoAttack &
AutoAttack Eg
MT
65.12 +: 2.69
4.40 + 0.16
0.35 +: 0.31
9.21 + 2.43
0.06 0.06
3.98 4:8:: 2.15
MixMatch
75.70 1.35
16.99 1.96
0.00 +: 0.00
0.85 :: 0.32
0.00 0.00
0.02 0.02
2.07
21.31 0.74
19.33 1.17
32.05 2.38
14.28 +: 1.78
ARG
51.20 +: 0.67
23.57
+: 0.35
20.49
0.79
32.28
1.11
18.84 0.99
30.55 1.08
MT+AT
50.59 1: 0.37
20.39
+: 2.03
17.44 =: 1.96
32.11 1.30
16.63 + 1.83
31.21 +: 1.32
ARMOURED
62.64 + 01.88
42.16
4.64
31.94
6.27
40.20 .4
9.40 1.51
22.93 0.75
ARMOURED-I-AT
56.35 1.09
27.31
4.17
15.73
3.48
2.83
9.59
2.61
22.40-2 2.67
ARMOURED-H+AT
51.26 1.18
26.26
+: 1.33
13.55
6.38
26.14 1 6.32
8.27 1.91
19.62 + 6.13
ARMOURED-FFAT
56.74 0.97
(35.37 6.34
23.90
5.99
4.41 34.91
15.10 2.72
28.31 + 3.75
19

---

Published as a conference paper at ICLR 2021
airplane
40
bird
30
20
dog
horse
airpiane
sulbmobile
bWrd
698T
60g
kgg
hore
ship
Buck
airpiane
aubmobile
Bird
deer
8og
bog
Rick
ship
(a) ARMOURED-HAA
airplane
automoblee
bird
deer
frog
horse
airpann
Bulbmobile
berd
deer
hore
ship
Bick
airplane
aubomobila
bird
horee
8hip
Bick
truck
(b) ARMOURED H+AT
air airlann
automoblee
cat
20
dog
frog
oirians
automobile
bird
66g
100
hOTe
ship
Bick
airiane
auboobble
band
deer
dog
Bog
hone
ship
ship
truck
(c) ARMOURED-FAAE
60
airplane
automobile
bird
40
cat
dog
horse
Brrlane
aulombble
bidd
60g
k0g
ship
Ruck
aimiane
aulomabile
B0g
hore
8h0
Ruck
ship
ARMOURED-F
Figure B.7: Posterior outputs on CIFAR-10 test data generated by networks N (left side) and
N (right side). Horizontal labels show the ground truth classes, vertical axis shows the predicted probabilities multiplied by 100. All models are trained on
20

---

Published as a conference paper at ICLR 2021
Table B.12: Benchmark against AtutoAttack with varying budgets on CIFAR-10-semi 1k
AutoAttack foo
AutoAttack f
1/255
2/255
4/255
0.1
0.2
0.3
RST
40.89 +:9:222 2.12
36. 30.30-25
1.89
27.46
2.01
42.06 +: 2.37
38.34 +: 2.27
35,06 + 3.54
ARG
46.50 0.7.5
41.80 0.85
33.12 + 0.75
46.89 + 0.39
42.37 + 0.85
38.58 U.84
45.65
40.59 1.22
31.10
42.88
1.29
26.57
35.83 U.86
ARMOURED-I+AR
39.19
33.97
2.92
23.17
3.31
39.93 + 2.41
35.54 + 2.62
30.90 +: 2.89
ARMOURED-HAAD
35.98 3.41
30. 60 1.87
20.35 1 6.15
36.75 + 3.12
32.41 + 4.36
27.52 +: 5.18
ARMOURED-F4AN
38.11 3.40
39.39
2.76
35.42
AutoAttack (f=)
AutoAtlack (t>)
Acucracy (%)
Acucracy
30
30
20
10
20
0
1/255
2/255
4/255
B/255
10
0.1
0.3
0.5
Budget
Budget
MT+AT
RST
ARMOURED-HAT ARMOURED-HAAT
ARMOURED-F+AT
ARG
ARMOURED-F
Figure B.8: Robustness against AutoAttack vs. perturbation budget e on CIFAR- -E-AR-1--semi--
21


Reviews:
[{"review_id": "p7lt7mBs5cc", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Update** : Since most of my issues have been addressed, I have changed my rating from 6 to 7\n\n\nSummary:\n\nThis paper proposed ARMOURED, a semi-supervised training algorithm that combines multi-view learning to utilize unlabeled data, along with diversity regularization to decrease inter-model transferability for adversarial inputs. As a consequence, the final models achieve near state-of-the-art adversarial robustness, without explicitly generating them during its training process. Based on experiments on the CIFAR10 and SVHN classification tasks, the proposed technique produces models that maintain accuracy on clean as well as adversarially perturbed data.\n\n##########################################################################\n\nReasons for score:\n\nThe proposed regularization terms of using unlabeled data and an ensemble of models with different views to indirectly promote adversarial robustness are quite interesting. However, the absence of evaluation against an adaptive adversary, along with signs of gradient masking makes this work in its current form not ready for publication. If the authors can run tests with an appropriately designed adaptive adversary, along with stronger evaluations (including gradient-free and black-box adversaries), and if the model performs well under those settings, the proposed work would be qualified to be part of the conference proceedings. \n\n##########################################################################\n\nPros:\n\n- The idea of increasing inter-model diversity to decrease transferability between models is a very interesting concept. The fact that it leads to better robustness without explicitly training for it shows that the authors' intuition in using this regularization term is, in fact, sound.\n\n- Using the idea that augmented images are distorted views of the same image and thus their deep views should agree is great! However, I must point out that the idea of using augmentations to enforce consistency on unlabeled data is now new. One example of such work is [Unsupervised Data Augmentation for Consistency Training](https://arxiv.org/pdf/1904.12848.pdf). Please cite this (and/or relevant work) and do include a comparison.\n\n- Using the concept of a stable sample (based on consensus) is great, since it might help filter out noisy labels.  \n\n##########################################################################\n\nCons:\n\n- Since this is primarily a paper that focuses on adversarial robustness (ultimately), I feel some very basic yet important changes are required. In general, the authors could follow the advice in the paper [On Evaluating Adversarial Robustness](https://arxiv.org/pdf/1902.06705.pdf). To be a bit more concrete, I would recommend having at least the following:\n  - Using FGSM to evaluate is not recommended, since it is not nearly as strong an attack as something like PGD. Instead, I would recommend swapping FGSM evaluation with a black-box attack like Boundary++, or some perturbation-minimizing attack like C&W \n  - Since the final model is trained using a combination of loss functions and non-trivial processes, it might be beneficial to run evaluations against an adaptive adversary. Space could be made for it by shifting to Figure 3 to supplementary material.\n\n- Is there any particular reason for skipping augmentation at inference time?\n\n- The part that involves using proper class-level information to incorporate it in the DPP regularizer seems similar to work that uses cost-functions to differentiate between different classes when working with adversarial inputs. One such work is [Cost-Sensitive Robustness against Adversarial Examples](https://arxiv.org/abs/1810.09225). It might be worthwhile to talk about and compare the matrices used in the experiments with the matrices used in this work.\n\n- Although the models do theoretically possess similar learning capabilities, it might be interesting to see if different models end up being more \"contributing\" than others in their predictions. Maybe some analysis might reveal that some augmentations/models are much more useful than the rest (which could be used for pruning), or perhaps trends like 'some models help with clean accuracy, while others help with robustness'. If space/time permits, it would be nice to have such analyses (perhaps in additional materials).\n\n- It has been observed in the literature that the performance of PGD attacks is significantly degraded when random restarts are not used. In your experiments, please run them with random-restarts and report numbers.\n\n- Tables 1&2 have '+-' numbers with them. Why are they there? Are these numbers accuracy, or error rates? Were the attacks run multiple times? If yes, how many? Also, are those numbers s.d? Please clarify.\n\n- The use of unlabeled data along in this case, along with multiple models to have an ensemble effect, seems to be somewhat related to this work [A2-LINK](https://ieeexplore.ieee.org/document/9104705). The authors might benefit from looking at similarities in the algorithm, and if some techniques can be borrowed/adapted to make the current algorithm stronger in its performance.\n\n- The fact that even under strong PGD attacks classes remain recognizable, along with almost unscathed performance for ARMOURED-B for increasing perturbation budgets, strong hints at some form of gradient masking in the final model. Please use black-box evaluation and other methods to make sure this isn't the case. One simple check would be to use [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://arxiv.org/abs/2003.01690). Their code is publicly available [here](https://github.com/fra31/auto-attack)\n\n########################################################################## \n\nTypos/Minor edits:\n\n- Abstract, last line \"substantial gains in accuracy, while maintaining high accuracy...\". Did the authors mean \"substantial gains in robustness...\"?\n\n- Introduction, third line \"..in the input image..\". Since the definition being given is for adversarial attacks in general, it should be for any data points with human-imperceptible changes, not just images. Also, in the more broad sense, the examples lead to unintended/unexpected behavior, not necessarily \"incorrect classifications\". \n\n- Figure 1: Please do not rely on color to differentiate between different arrows (the reader may be color blind). Either switch to some other attribute (like the width of the arrow) or explicitly label them to differentiate between red and blue.\n\n- Formatting: Table 2 could be shifted elsewhere (with text wraparound) to get more space for the authors to focus on things like adaptive adversaries. The same comment for Figure 1 (a lot of extra space on both sides of the image); could remove the padding?\n\nPlease address and clarify the cons above. Most importantly, it is crucial to have evaluation against stronger attacks as well as an adaptive adversary, before claims of robustness can be substantiated.", "extra_scores": {}, "extra_text": {}}, {"review_id": "ehP8gRQpDtC", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "## Summary\nThis paper introduces a semi-supervised learning procedure that does not require labeled adversarial data to learn an ensemble model that is robust to adversarial attacks on classification tasks.\n\n## Quality\nThis paper is very well written; the design decisions of the training procedures are all supported by ablation tests and comparisons to other modern adversarial baselines. However, I am slightly worried by the ablation study results -- from Table 3, it seems like the DPP component of the loss does not provide a statistically significant lift. \n\n## Clarity\nThis paper is very clearly written! The choice of parameterization, experiment settings, and overall proposed methodology were clearly explained and motivated.\n\n## Originality\nAs the authors have mentioned, improving diversity in ensembles is not a novel approach to improving robustness to adversarial attacks; the originality of the method lies in the parameterization of diversity through a DPP component and multi-view complementarity. \n\n## Significance\nImproving robustness to adversarial attacks is in of itself an important contribution to this field. This paper proposes a method that achieves a significant improvement over other competitors (Tables 1, 2); the proposed method is additionally intuitive, and can incorporate prior beliefs over the the data distribution, making it adaptable to several different settings.\n\n## Questions\n### Methodology\nMy main question lies in the uses of the DPP loss: did the authors use the determinant directly? Depending on the choice of kernel, I would expect the determinant portion of the loss to eclipse the other terms, potentially hampering learning. \n\nOn a related note, I am curious to know how sensitive the ARMOURED performance is to the $\\lambda_{DPP}$ and $\\lambda_{NEM}$ hyper-parameters.\n\n### Experiments\nDo the authors have insight in the significant gap between ARMOURED-B and its variants for a large $\\epsilon$ budget with $L_\\infty$ PGD? I am surprised by the significant gap in Table 3.\n\nAgain in Table 3, comparing the -F, no DPP, and -B variants, it seems like the highest lift in performance comes from the diversity in the NEM component. Do you know if a similar lift is observed by removing the NEM component and keeping the DPP? \n\nAt inference time, you state that the augmentation step is skipped. Naively, I would not have been surprised to see these augmentations improve resilience to adversarial attacks. Am I incorrect?", "extra_scores": {}, "extra_text": {}}, {"review_id": "LfyBS78it_t", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper considers training an adversarially robust model in a semi-supervised setting. The authors propose an ensemble-based algorithm for this goal. The algorithm uses a regularization term to induce diversity in the ensemble. The algorithm also leverages the idea of multi-view training in semi-supervised learning to make use of unlabeled data. The experimental results show that the proposed algorithm outperforms several baselines.\n\nOverall I enjoyed reading this paper. It is relatively clearly written, and easy to follow. I think the basic idea of this paper is a combination of the ADP algorithm by Pang et al and the classic multi-view idea in semi-supervised learning. In fact, some of the figures in this paper are quite similar to those in Pang et al 2019. It is not surprising that this algorithm outperforms ADP since it makes use of many unlabeled data. This makes me feel that the proposed algorithm does not have enough novelty. However, I do think this paper is beneficial to the adversarial robustness (AR) community since it brings an important idea of multi-view semi-supervised learning to AR research. Therefore, I decided to give a score of weakly accept.\n\n======\nAfter author response: I have read other reviews and the revised version. I think the paper's overall quality has improved. I decided to change my score to 7.", "extra_scores": {}, "extra_text": {}}, {"review_id": "IRfGKK2yy03", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### **Summary**:\n \nThis work introduces ARMOURED, a new method for learning models that are robust against adversarial attacks. The method uses **multi-view**-learning (as in using multiple models with different parameters to cast their votes/views on a given sample), semi-supervised learning to pseudo-label new data based on a consensus, and a diversity regularizer. The approach is evaluated in CIFAR-10 and SVHN against recent baselines in the presence of white-box attacks. The results yield by ARMOURED in these scenarios are superior.\n \n---\n \n### **Reasons for score**: \n \nMy recommendation is to accept the paper. I like the proposal, it is simple yet is able to produce better results than existing methods. It also performs well with clean samples, which is something that many other methods struggle with. The only thing I miss is to see experiments in ImageNet.  \n \n---\n### **Strengths**:\n \n* The problem is relevant for the community and the proposed approach seems to do a better job than the existing state-of-the-art leveraging weak-supervision/semi-supervision (definitely a plus). \n* The method is simple and seems easy to adapt to other architectures and tasks.\n\n---\n\n### **Weaknesses**:\n\n* The ablation experiments point to the unlabelled data as one of the core reasons for the method to work. I was expecting to see additional experiments with extra unlabelled data to get a better feeling of this component.\n\n---\n\n### **Questions to be discussed during the rebuttal period**:\n\nPlease, address the questions/comments expressed in the **Weaknesses** section. \n\n---\n\n### **Other considerations**:\n\nNone\n\n\n---\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper focuses on adversarial robustness with unlabeled data. The philosophy behind sounds quite interesting to me, namely, utilizing unlabeled data to enforce labeling consistency while reducing adversarial transferability among the networks via diversity 
regularizers. This philosophy leads to a novel algorithm design I have never seen, i.e., ARMOURED, an adversarially robust training method based on semi-supervised learning.

The clarity and novelty are clearly above the bar of ICLR. While the reviewers had some concerns on the significance, the authors did a particularly good job in their rebuttal. Thus, all of us have agreed to accept this paper for publication! Please carefully address all 
comments in the final version.

Decision: Accept (Poster)