EMPIR: Ensembles of Mixed Precision Deep Networks for Increased Robustness Against Adversarial Attacks
Authors: Sanchari Sen
Balaraman Ravindran
Anand Raghunathan
Venue:  (2020)
Keywords: ensembles
mixed precision
robustness
adversarial attacks

Abstract: Ensuring robustness of Deep Neural Networks (DNNs) is crucial to their adoption in safety-critical applications such as self-driving cars, drones, and healthcare. Notably, DNNs are vulnerable to adversarial attacks in which small input perturbations can produce catastrophic misclassifications. In this work, we propose EMPIR, ensembles of quantized DNN models with different numerical precisions, as a new approach to increase robustness against adversarial attacks. EMPIR is based on the observation that quantized neural networks often demonstrate much higher robustness to adversarial attacks than full precision networks, but at the cost of a substantial loss in accuracy on the original (unperturbed) inputs. EMPIR overcomes this limitation to achieve the ``best of both worlds", i.e., the higher unperturbed accuracies of the full precision models combined with the higher robustness of the low precision models, by composing them in an ensemble. Further, as low precision DNN models have significantly lower computational and storage requirements than full precision models, EMPIR models only incur modest compute and memory overheads compared to a single full-precision model (<25% in our evaluations). We evaluate EMPIR across a suite of DNNs for 3 different image recognition tasks (MNIST, CIFAR-10 and ImageNet) and under 4 different adversarial attacks. Our results indicate that EMPIR boosts the average adversarial accuracies by 42.6%, 15.2% and 10.5% for the DNN models trained on the MNIST, CIFAR-10 and ImageNet datasets respectively, when compared to single full-precision models, without sacrificing accuracy on the unperturbed inputs.

Published as a conference paper at ICLR 2020
EMPIR: ENSEMBLES OF MIXED PRECISION DEEP
NETWORKS FOR INCREASED ROBUSTNESS AGAINST
ADVERSARIAL ATTACKS
Sanchari Sen
Balaraman Ravindran
Center for Brain-Inspired Computing School Electrical and Computer Engineering Purdue University West Lafayette, IN, USA 9 Suin9@@uudueee. edu
Department of Computer Science and Engineering Robert Bosch Centre for Data Science and Al Indian Institute of Technology (IIT) Madras Chennai, TN, India ravi ccse iitm. ac. in
Anand Raghunathan
Center for Brain- Insiielespired Computing
School of Electrical and Computer Engineering
Purdue University
West Lafayette, IN, USA
raghhmatthan@purdue.edd
ABSTRACT
Ensuring robustness of Deep Neural Networks (DNNs) is crucial to their adoption in safety-critical applications such as self-driving cars, drones, and healthcare. Notably, DNNs are vulnerable to adversarial attacks in which small input per- turbations can produce catastrophic misclassifications. In this work, we propose EMPIR, ensembles of quantized DNN models with different numerical precisions, new approach to increase robustness against adversarial attacks. EMPIR is based on the observation that quantized neural networks often demonstrate much higher robustness to adversarial attacks than full precision networks, but the cost of a substantial loss in accuracy on the original (unperturbed) inputs. EM- PIR overcomes this limitation achieve the "best of both worlds". i.e., the higher unperturbed accuracies of the full precision models combined with the higher ro- bustness of the low precision models, by composing them in ensemble. Further, as low precision DNN models have significantly lower computational and storage requirements than full precision models, EMPIR models only incur modest com- pute and memory overheads compared to a single full-precision model (<25% our evaluations). We evaluate EMPIR across suite of DNNs for 3 different image recognition tasks (MNIST, CIFAR-10 10 and ImageNet) and under 4 different adversarial attacks. Our results indicate that EMPIR boosts the average adversar- ial accuracies by 42.6%. 15.2% and 10.5% for the DNN models trained on the MNIST, CIFAR- and ImageNet datasets respectively, when compared to single full- full-precision models, without sacrificing accuracy the unperturbed inputs.
INTRODUCTION
The success of Deep Neural Networks (DNNs) in different machine learning tasks has fueled their in safety-critical applications like autonomous cars, unmanned acrial vehicles and healthcare, wherein errors (misclassifications) made by DNNs can lead to severe - in the extreme case, fatal consequences. Therefore, robustness, i.e., the ability cope with erroneous malicious inputs fed to an application, emerging as an important requirement for DNNs.
Several efforts have in fact shown that DNNs behave in unexpected and incorrect ways for small, specifically designed input perturbations (Goodfellow et al. (2014)). An attacker can take advan- tage of this behavior intentionally modify the inputs in manner that forces the DNN model to

---

Published as a conference paper at ICLR 2020
mis-classify, and the overall system that uses the DNN to fail. A variety of methods for launching adversarial attacks on DNNs have been proposed over the years. These adversarial attacks system- atically modify given original input to cause misclassification while keeping the imput distortion minimal. A few examples of adversarial attacks that have been successfully applied to various DNN models are the Fast Gradient Sign Method (FGSM) (Goodfellow et al. (2014)), Jacobian-based Saliency Map Attack (JSMA) (Papernot et al. (2015)), Carlini-Wage Wagner (CW) (Carlini & Wagner (2016)) and the Basic Iterative Method (BIM) (Kurakin et al. (2016)).
Prior works have tried to overcome these vulnerabilities by proposing various defense mechanisms against adversarial attacks. Adversarial training (Goodfellow et al. (2014)), defensive distillation (Papernot et al. (2015)) and input gradient regularization (Ross & Doshi- Velez (2017)) are a few representative defense techniques. Each of these approaches, albeit promising, has limitations with respect to the kind of attacks they can defend against, the increase in training complexity, as well as their effect on the model's accuracy on the original unperturbed inputs. To address these short- comings, we propose EMPIR, ensemble of mixed precision DNN models, as new form of defense against adversarial attacks and demonstrate that it can significantly improve the robustness of variety of DNN models across wide range of adversarial attacks.
Ensembles have been widely explored as an approach to improve the performance of machine learn- ing models and classifiers (Hansen & Salamon (1990)). Examples of various successful ensembling methods include averaging, bagging (Breiman (1996)). boosting (Dietterich (2000)), etc. Recently, it has also been suggested that ensembles may help boost the robustness of DNNs (Strauss et al. (2017); Pang et al. (2019); He et al. (2017); Tramèr et al. (2017)). The individual models in these ensembles are restricted to full precision DNN models, i.e., models utilizing 32 bits of numeri- cal precision to represent different data-structures. Such ensembles are very expensive in terms of the computational and memory overhead (e.g.. 10x the baseline for an ensemble with 10 models (Strauss et al. (2017))). In contrast, the use of quantized models in EMPIR, which entail the use of significantly lower number of bits in storage and compute, ensures that the overhead modest (less than 25% in our evaluations).
Quantized DNNs are characterized by the use of lower numbers of bits to represent DNN data- structures like weights and activations (Venkataramani et al. (2014); Hubara et al. (2017); Zhou et al. (2016); Courbariaux et al. (2015)). They have been widely explored as an approach to reduce the high computational and memory demands of DNNs. Recent studies have also observed that these quantized models demonstrate higher robustness to adversarial attacks (Galloway et al. (2017); Siraj Rakin et al. (2018); Panda et al. (2019)). However, the loss in information associated with the quantization process often makes these quantized models perform significantly worse than their full- precision counterparts while classifying the original unperturbed inputs. This motivates the design of EMPIR, which successfully combines the higher robustness of low-precision models with the higher unperturbed accuracy of the full-precision models. In the general case, EMPIR comprises of M frel-precision models and N low- oow-precision models with the final prediction determined by an ensembling technique such as averaging the probabilities or counting the number of predictions for each class. In practice, we find that M = 1 and N = 2 or 3 provides a significant improvement in adversarial accuracy with small overheads.
In summary, the key contributions of this work are
We propose the use of ensembles of mixed precision models as a defense against ad adversarial attacks on DNNs.
We analyze the effect of ensemble size and ensembling techniques on the overall robustness as well as the computational and storage overheads of the ensemble.
Across suite of 3 different DNN models under 4 different adversarial attacks, we demon- strate that EMPIR exhibits significantly higher robustness when compared to individual models as well as ensembles of full-precision models.
Here, the term precision refers to the numerical precision of the models, the number of bits used to represent their weights and activations.
2

---

Published as a conference paper at ICLR 2020
2 ADVERSARIAL ATTACKS: BACKGROUND
Adversarial attacks modify inputs in a manner that force a DNN model to misclassify, while ensur- ing that the input changes small and imperceptible to human eyes. In the context of DNNs that operate on images, which are the focus of most prior work, various attack methods have been pro- posed to systematically modify pixel values the input image so as result in mis-classification A few such methods are described below.
Fast Gradient Sign Method (FGSM) (Goodfellow et al. (2014)). FGSM single-step at- tack that operates by calculating the gradient of the loss function with respect to the input pixels (7, (V,L(0, X, )). Based on the sign of the loss, the input pixels are increased or decreased by a small constant, c, to help move the image towards the direction of increased loss. The adversarial input, Xadv can be computed as:
Xade = X +
(I)
Here, X the original input image associated with an output Y and refers to the weights of the network.
Basic Iterative Method (BIM) (Kurakin et al. (2016)). BIM is an iterative version of the FGSM at- tack which performs finer optimization by modifying pixels by small values in each iteration. Fur- ther, the image generated in each iteration has its pixel values clipped to ensure minimal distortion. Mathematically, this attack can be described as:
ado X = Clipx, adv + aSign (0, x Xados8)) B)) X = X,
(2)
Here, the terms X, Y. 0 and have the same meaning as in Equation and X edv refers to the adversarial input generated at the /tt iteration and is the step size in each iteration.
Gagner (CW) (Carlini & Wagner (2016)). CW is another iterative attack that employs optimizers to create strong adversarial inputs by simultaneously minimizing the input distortion and maximizing the misclassification error. be described mathematically as:
min 1811 f(X + 8) such that (X + 6) E [0. 1]"
[(X) max (sft
(x():: (X),;))
(3)
Xudo = X +
where 8 the input distortion, is the Lagrangian multiplier, ((X) is the logit output for the input X, is the target class and f(X) is objective function that satisfies the condition f(X + 6) for all misclassifications.
Projected Gradient Descent (PGD) (Madry et al. (2017)). PGD is third type of iterative attack very similar in nature to the BIM attack. Unlike BIM, which starts with the original image itself, PGD starts with random perturbation of the original input image. PGD can be described by the following equations:
X-
ado X FandomonnffommmsssppppC (-c,et)
(4)
N+1 adv = -Cipxx adv +
X y))}
Here, the terms X, N.
adv- 0, c and a have the same meaning as in Equation 2.
To summarize, different adversarial attacks have been proposed that expose the lack of robustness in current DNN models by constructing adversarial inputs that force a misclassification. Developing defenses these adversarial attacks is critical enable the deployment of DNNs in safety-critical systems.
3 EMPIR: ENSEMBLES OF MIXED PRECISION DEEP NETWORKS FOR INCREASED ROBUSTNESS AGAINST ADVERSARIAL ATTACKS
To improve the robustness of DNN models, propose EMPIR, ensembles of mixed precision models. In this section, we will detail the design of EMPIR models and discuss the overheads associated with them.

---

Published as a conference paper at ICLR 2020
3.1 ADVERSARIAL ROBUSTNESS OF LOw-PRECISION NETWORKS
DNNs have conventionally been designed full precision models utilizing 32-bit floating point numbers to represent different data-structures like weights, activations and errors. However, the high compute and memory demands of these full-precision models have driven efforts to move towards quantized or low-precision DNNs /Venkataramani et al. (2014); Hubara et al. (2017); Zhou al. (2016); Courbariaux al. (2015); Wang al. (2018)). multitude of quantization schemes have been proposed to minimize the loss of information associated with the quantization process. While our proposal is agnostic to the quantization method used, for the purpose demonstration we adopt the quantization scheme proposed in DoReFaNet (Zhou et al. (2016)), which has been shown to produce fow-precision models with competitive accuracy values. The quantization scheme can be described by Equation 5.
tanh(w) guantize:(x) 2 round((2* -1)
(5)
+
1 - as quantizex(a)
Wg quantizeal maze[[anh((]]]]
where i refers the number of quantization bits in the low precision network, w and w/k refer to weight values before and after quantization, and a and ag refer to activation values before and after quantization.
ooneeae le accuracy
1.00
Umperturbed accurac)
0.4
0.99
2
M
0.98
0.97
Presicted probobility
0.2
Input
32
0.95
Image
technique
predicted class
Activation bits
16 8
24 16
Weight bits
Activation bits
24
16
8
24
Weight bits
16 8
LP models
32
0.15
0.20
0.25
0.30
0.980
0.984
0.98
EMPIR Model
Figure 1: Unperturbed accuracies and adversarial accuracies of lowppreciioon models trained for the MNIST dataset
Figure Overview of EMPIR
In addition to the widely known advantages of reduced model size and reduced complexity of arith- metic computations, recent research efforts have also brought to light another lesser known advan- tage of low-precision models in the form of increased robustness adversarial attacks. It has been observed that low low-precision models general exhibit higher values of adversarial accuracy than full- full-precision models with identical network structures (Galloway al. (2017); Panda et al. (2019)). One possible explanation for this property is that higher quantization introduces higher amounts of non-linearity, which prevents small changes in the input from drastically altering the output and forcing misclassification (Galloway et al. (2017)). Figure shows the adversarial accuracies of different low precision models trained on the MNIST dataset under the FGSM attack. Unlike the activations and weights, the gradients utilized in the attack generation process were not quantized, allowing the adversary to launch stronger attack. From the figure, it is apparent that models with lower numbers of bits used for representing weights and activations exhibit significantly higher lev- els of adversarial accuracy.
However, increasing the robustness of a system by simply replacing the full-precision model with its low-precision variant can negatively impact its accuracy on the original unperturbed inputs (un- perturbed accuracy). In other words. the model may start to mis-classify inputs that were not adversarially perturbed. Figure I also shows the unperturbed accuracies of low- now-precision models. As expected, models with weights and activations represented using lower numbers of bits exhibit lower unperturbed accuracies.
Based on the above observations, propose the of ensembles of mixed-precd precision models to achieve the best of both worlds, i.e., increase robustness against adversarial attacks without sacrific- ing the accuracy unperturbed inputs.
4

---

Published as a conference paper at ICLR 2020
3.2 EMPIR: OVERVIEW
Figure 2 presents overview of EMPIR. In the general case, EMPIR model comprises of M full- precision (FP) models and N low pww-riiiiionn (LP) models. The frel-precision models help in boosting the unperturbed accuracy of the overall model, while the low-precision models contribute towards higher robustness. All the individual models are fed the same input and their predicted classes or probabilities combined with the help of ensembling technique at the end to determine the final prediction of the EMPIR model. In practice, we found that a single full-precision model (M=1) and a small number of low-precision models (N=2 3) sufficient achieve high adversarial accuracies without any noticeable compromises in the unperturbed accuracies.
The ensembling function plays a vital role in the overall performance of the model determines the final classification boundary. In this work, we consider two of the most commonly used en- sembling functions, namely, averaging and max voting. The averaging function averages the output probabilities of each model and identifies the class with the maximum average probability the fi- nal predicted class. On the other hand, max voting considers the predictions each model as votes and determines the class with the maximum number of votes to be the final class. In our experi- ments, we found that averaging achieves better adversarial accuracies on ensembles of size while max voting achieves better adversarial accuracies ensembles of size greater than 2.
In order to allow an ensemble model to work better than a single model, the individual models should also be designed to be diverse (Hansen & Salamon (1990)). This ensures that the models dont produce similar errors and hence, that the probability of two models misclassifying the same input is lower We introduce diversity in the individual models of EMPIR by training them with different random initializations of weights.
3.3 COMPUTATIONAL AND MEMORY COMPLEXITY OF EMPIR
The ensembling of multiple full-precision and low-precision models in EMPIR increases its com- putational and storage requirements as these models need to be stored and evaluated. In this work, we keep these memory and computational complexities within reasonable limits by restricting the precision of weights and activations the low-precision models of EMPIR to a maximum of 4 bits.
The increasing popularity of low-precision DNN models has prompted recent hardware platforms including GPUs and neural network accelerators to add native hardware support for operating on low precision data (Fleischer et al. (2018); Kilgariff et al. (2019)). These hardware platforms reconfigure a common datapath to perform computations on full-precision data (32 or bits) as well as low- precision data (4, 8 or 16 bits). Low-precision operations can achieve higher throughputs than full-precision operations on these platforms as the same number of compute elements and the same amount of memory bandwidth can support larger number of concurrent operations. Consequently, the additional execution time required to evaluate the low-pree pweeiiiinoon models in EMPIR much less than that of full- precision model. Overall, we quantify the execution time and storage overhead of an EMPIR model using the formula described by Equation 6.
Ops-per per secr_ssccFF FP) Ops-per _ec(kk)
TinCOeerhaa((EMPIMMMM. M +
Σ
(6)
N
h
StorageOverheadl (MMM:/aMMMMMMMMMMMM = M +
FP
where k; is the precision of the 5th low-precision model, FP is the precision of the full-precision models, and Ops-per ppss_per_secle per_sec(b) ) is the throughput of & bit operations on the underlying hardware plat- form.
EXPERIMENTS
In this section, we describe the experiments performed to evaluate the advantages of EMPIR models over baseline full-precision models.
5

---

Published as a conference paper at ICLR 2020
4.1 BENCHMARKS
We studied the robustness of EMPIR models across three different image recognition DNNs, namely, MNISTconv, CIFARconv and AlexNet. The individual full-precision and-low precision networks within the EMPIR models were designed to have identical network topologies. The details of the individual networks in these benchmarks are listed in Table within Appendix A. The benchmarks differ in the number of convolutional layers, fully connected layers as well as the datasets. We consider three different datasets, namely, MNIST (Lecun et al. (1998)), CIFAR-10 (Krizhevsky (2009)) and ImageNet (Deng et al. (2009)) which vary significantly in their complexity. The low precision networks were obtained using the quantization scheme proposed DoReFa-Net (Zhou et al. (2016)). The full precision models were trained using 32 bit floating point representations for all datu-sttuc,
4.2 EVALUATION OF ROBUSTNESS
within TensorFlow (Abadi et al. (2015)) and have released the source code for implementation 2. The robustness of the EMPIR models measured in terms of their adversarial accuracies under a variety of white-box attacks within the Cleverhans library (Papernot al. (2018)). We specifically consider the four adversarial attacks described in Section 2. The adversarial parameters for the attacks on the different benchmarks are presented Table 1. The attacks were generated the entire white-box attacks involves computation of the gradient V, L0, X, Y) (Section 2), which is not directly defined for ensembles. For the EMPIR models, we compute this gradient as an average over all individual models for an averaging ensemble and as an average over the individual models that voted for the final identified class for max voting ensemble.
We implemented EMPIR
dataset for each of the benchmarks. Generating these
Network
CW
FGSM
BIM
PGD
Attack
6= 0.3, a=0.01
c= 0.3,
a=0.01
MNISTconv
<= 0.3
iterations 50
No. of iterations 40
No. of iterations 40
Attack
== 0.1, a=0.01
== 0.1, a=0.01
CcoA
iterations 50 == 0.1
No. of iterations 40
No. of iterations 40
Attack
6= 0.1, 0=0.01
6= 0.1, a=0.01
AlexNet
<= 0.1
iterations
No. of iterations 40
No. of iterations
Table 1: Attack parameters
RESULTS
In this section, we present the results of our experiments highlighting the advantages EMPIR mod- els.
5.1 ROBUSTNESS OF EMPIR MODELS ACROSS ALL ATTACKS
Table 2 presents the results of our experiments across different benchmarks. The EMPIR mod- els presented in the table are the ones exhibiting highest average adversarial accuracies under the constraints of <25% compute and memory overhead and <2% loss in unperturbed accuracy. We observed that across all the benchmarks, ensembles comprised of two low-precision and one full-precision model combined with the max-voting ensembling technique satisfy these constraints. However, the individual configurations of the low-precision models, i.e., the precisions of weights and activations in the ensembles, differ across the benchmarks. For example, both low-precision models in the EMPIR model for MNISTconv have weight precisions of bits and activation pre- cisions of 4 bits. On the other hand, the two low-precision models in the AlexNet EMPIR model have (weight,activationn) bit-precisions of (2,2) and (4,4), respectively. In general, we observe that
httpps:/ thub. com/ sanchari sen /PMP 1R

---

Published as a conference paper at ICLR 2020
Unperturbed
Adversarial Accuracy (%)
Network
Approach
Accuracy (%)
CW
FGSM
BIM
PGD
Average
Baseline FP
98.87
3.69
14.32
0.9
0.77
4.92
EMPIR
98.89
86.73
67.06
18.61
17.51
47.48
Defensive Distill.
98.12
2.34
40.22
7.61
3.28
13.36
MNISTconv
Inp. Grad. Reg.
99.01
6.83
30.15
1.14
1.20
9.83
FGSM Adv. Train
99.06
3.09
76.56
0.87
0.39
20.23
EMPIR (FGSM
Adv. Train)
99.09
90.54
75.98
33.16
5.17
51.21
Baseline FP
74.54
13.38
10.28
11.97
10.69
11.58
EMPIR
72.56
48.51
20.45
24.59
13.55
26.78
FGSM Adv. Train
72.36
14.36
41.58
12.92
11.24
20.03
CIFARconv
EMPIR (FGSM
Adv. Train)
73.62
45.73
31.67
29.55
14.74
30.42
PGD Adv. Train
73.55
12.62
12.45
10.97
8.52
11.14
Baseline FP
53.23 55.09
9.94
10.29
10.81
10.30
10.34
AlexNet
EMPIR
29.36
21.65
20.67
11.76
20.86
Table Unperturbed and adversarial accuracies the baseline and EMPIR models across different attacks
the EMPIR models exhibit substantially higher adversarial accuracies across all attacks for the three benchmarks.
We also compare the benefits of EMPIR with four other popular approaches for increasing robust- ness, namely, defensive distillation (Papernot et al. (2015)), input gradient regularization (Ross & Doshi Velez (2017)). FGSM based adversarial training (Goodfellow et al. (2014)) and PGD based adversarial training (Madry et al. (2017)). The distillation process implemented with softmax 100, the gradient regularization was realized with regularization penalty of 100, while the adversarial training mechanisms utilized adversarial examples generated with a maximum possible perturbation of c = 0.3. Table 2 presents the results for the approaches that were able achieve <5% loss in unperturbed accuracy for a particular benchmark. We observe that FGSM based adversarial training significantly boosts the adversarial accuracies of the MNIST- conv and CIFARconv models under the FGSM attack but is unable to increase the accuracies un- der the other three attacks, often hurting them in the process. A similar result is observed for the MNISTconv model trained with defensive distillation and gradient regularization. PIR successfully increases the robustness of the models under all four attacks. In fact, it can even be combined with the other approaches to further boost the robustness, as evident from the adversar- ial accuracies of an EMPIR model comprising of adversarially trained models for the MNISTconv and CIFARconv benchmarks. EMPIR also achieves a higher adversarial accuracy than PGD based adversarial training for the CIFARconv benchmark. Overall, EMPIR increases robustness with training overhead, as opposed to considerable training overheads associated with the other defense strategies like adversarial training, defensive distillation and input gradient regularization.
temperature of
contrast, EM-
5.2 COMPARISON WITH INDIVIDUAL MODELS
Figure 3(a) illustrates the tradeoff between the adversarial and unperturhed accuracies of the individ- ual DNN models and EMPIR models for two of the benchmarks under the CW attack. The circular blue points correspond to individual models with varying weight and activation precisions while the red diamond points correspond to the EMPIR models presented in Section 5.1. The figure clearly indicates that the EMPIR models both the benchmarks are notably closer to the desirable top right corner with high unperturbed well high adversarial accuracies. Among the individual mod- els, the ones demonstrating higher adversarial accuracies but lower unperturbed accuracies (towards the top left comer) correspond to lower activation and weight precisions while those demonstrating

---

Published as a conference paper at ICLR 2020
Individual
Baseline FP model
EMPIR model
MNISToomv
CIFARconv
0.6
Adversria Accumayy
0.8
Adversana Accuracy
0.5
g
4
0.6
6
02
Up
edreso
0.2
0.1
0
Predicted
Predicted
Unperturbed Acturacy
Unperturbed Acturacy
200 400 600 800 1000 1200 No. images
0 200 400 600 800 10000001200
(a)
(b)
No. images
Figure 3: (a) Tradcoff between unperturbed and adversarial accuracies the individual and EM- PIR models across 2 benchmarks. (b) Confusion matrices of the bascline FP and EMPIR model for the MNISTconv benchmark.
Soaanne (2a ad00 acr
Norman memory
1.00
3.0
3.0
0.7
m 0 am
2.5
0.98
0.6
2.0
2.0
0.97
0.4
1.0
1.0
1.5
No. LP Un models
0.95
0.5
No. LP models
No. of FP models
No. of FP models
of models
No.
models
No. of models
No. of models
0.2 04
05
0.6 0.7
10 15 25
[a]
[b)
(c)
[d]
Figure 4: Effects of varying the number of LP and FP models in EMPIR (a) Unperturbed accuracies, (b) Adversarial accuracies, (c) Execution time overheads and (d) Storage overheads
lower adversarial accuracies and higher unperturbed accuracies (towards the bottom right corner) correspond to higher activation and weight precisions.
5.3 ANALYSIS OF CONFUSION MATRICES
Figure 3(b) presents the confusion matrices of the baseline FP model and the EMPIR model for the MNISTconv benchmark under the FGSM attack. The actual ground truth class labels are listed vertically while the predicted labels are listed horizontally. The colors represent the number of images in the test dataset that correspond to the combination of actual and predicted class labels. The diagonal nature of EMPIR's confusion matrix clearly illustrates its superiority over the model, which frequently misclassifies the generated adversarial images.
5.4 IMPACT OF VARYING THE NUMBER OF LOW-PRECISION AND FULL-PRECISION MODELS
In this subsection, we vary the number of low-precision and full-precision models in EMPIR be- tween 0 and observe its effect the unperturbed and adversarial accuracies of the MNISTconv benchmark under the FGSM attack. We also measure the execution time and memory footprint of the EMPIR models to quantify their overheads with respect to baseline single full-precision model. We restrict the low-precisin models to have weight and activation precisions between 2 and 4 bits and choose the configurations that maximize the adversarial accuracies of the EMPIR models while introducing <1% drop in unperturbed accuracies.
Figure presents the results this experiment. Figure 4(a) and (b) clearly indicates that higher number of low-precision models in EMPIR helps in boosting the adversarial accuracies while higher number of full- precision models help in boosting the unperturbed accuracies. For instance, an EMPIR model comprising of only three low-precision models demonstrates unperturbed and ad- versarial accuracies of 98.8% and 56.9% respectively while an EMPIR model comprising of only three full-precision models demonstrates unperturbed and adversarial accuracies of 19.224 99.2%% and 31 %,
8

---

Published as a conference paper at ICLR 2020
respectively. The execution time and memory footprint associated with the former only 0.38 and 0.25x over the baseline, as opposed to 3x in case of the latter. Overall, we observe that an EM- PIR model comprising of single full-precision model and two low-precision models (configuration presented in Table 2) achieves a good balance between adversarial and unperturbed accuracies with modest execution time and storage overheads.
RELATED WORK
Popular defense strategies against adversarial attacks include adversarial training, defensive distilla- tion and input gradient regularization. Adversarial training (Goodfellow et al. (2014); Madry et al. (2017)) involves modifying the loss function to include the adversarial loss term, which tries to re- duce the effect of input perturbations. Defensive distillation (Papernot et al. (2015)), on the other hand, is based on the technique of distillation that was originally proposed to efficiently transfer knowledge across different DNN models. It involves training networks on the output probabilities of classes instead of the conventional approach of training on hard output class labels. As shown in Table 2, the benefits of these techniques are limited to only one couple of adversarial attacks. In contrast, EMPIR is able to boost the adversarial accuracies of DNNs across all four white-box attacks considered here.
Recent efforts have also proposed the use of ensembles of full precision models for defending DNNs against adversarial attacks (Strauss et (2017); Pang et al. (2019); He al. (2017); Tramèr et al. (2017)) However, the presence of multiple full full-precision models in these ensembles increases the compute and memory requirements significantly (10x for an ensemble with 10 models in Strauss et al. (2017)), which prevents the application of this approach to larger state-of-the art models. In contrast, with the use of low precision models in the ensemble, we are able to reduce the overhead significantly and restrict it to <25%. Also, shown in Figure 4, mixed-precision ensembles demon- strate higher adversarial accuracies than the full full-precision ensembles for identical number of models in the ensemble.
In addition to the above efforts, there have been a parallel set of efforts studying the robustness of low- -w-pprccision quantized DNNs. For example, binary neural networks with single bit precisions for weights and activations have been shown to exhibit higher adversarial robustness than their full- precision counterparts different white-box attacks (Galloway et al. (2017); Panda et al. (2019)). Stochastic quantization of activations has also been proposed as an approach to make DNNs more robust (Siraj Rakin et al. (2018)). However, shown in Figure 1, the individual quantized models in these efforts often demonstrate lower accuracies on unperturbed or clean examples due to the loss in information associated with the quantization process. On the other hand, the combination of full- precision models along with low- ow-precision models in EMPIR helps to overcome this limitation and achieve the best of both worlds - higher robustness combined with high unperturbed accuracy.
CONCLUSION
As deep neural networks get deployed in applications with stricter safety requirements, there a dire need to identify new approaches that make them robust to adversarial attacks. In this work, we boost the robustness of DNNs by designing ensembles of mixed-precision DNNs. In its most generic form, EMPIR comprises of M full-precision DNNs and N low-precision DNNs combined through ensembling techniques like max voting or averaging. EMPIR combines the higher robust- ness of low- -reppieciionn DNNs with the higher unperturbed accuracies of the full- full-precision models. Our experiments on 3 different image recognition benchmarks under 4 different adversarial attacks reveal that EMPIR is able to significantly increase the robustness of DNNs without sacrificing the accuracies of the models on unperturbed inputs.
ACKNOWLEDGMENTE
This work was supported by C-BRIC, one of six centers in JUMP, a Semiconductor Research Cor- poration (SRC) program, sponsored by DARPA.

---

Published as a conference paper at ICLR 2020
REFERENCES
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving. Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens. Benoit Steiner, llya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wat- tenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learn- ing on heterogeneous systems, 2015. URL https:////// /ww. tensorflww. org/ Software available from tensorflow.org.
Leo Breiman. gust 1996. ISSN 0885-6125.
Bagging predictors.
Mach. Learn., 24(2):123-440, Au- doi:
http:://e //tp://xxx/o doi. 2ttp//0..02//111111153333 10.1023/A1001805314333.
URL
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. CoRR, abs/1608. 2016. / hbs//bs/// 1608. 04644.
Matthieu Courbariaux, Yoshua Bengio, and (can-Pierre David. Binaryconnect: Training deep neural networks with binary weights during propagations. In Advances in Neural Information Processing Systems 28, pp. 3123-3131.
Curran Associates, Inc., 2015.
J. Deng. W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09, 2009.
Thomas G. Dietterich. Ensemble methods in machine learning. In Multiple Classifier Systems, pp- 1-15, Berlin, Heidelberg, 2000. Springer Berlin Heidelberg. ISBN 978-3-540-45014-6.
Bruce M. Fleischer, Sunil Shukla, Matthew M. Ziegler, Joel Silberman, Jinwook Oh, Vijayalakshmi Srinivasan, Jungwook Choi, Silvia Mueller, Ankur Agrawal, Tina Babinsky, Nianzheng Cao, Chia-Yu Chen, Pierce Chuang, Thomas W. Fox, George Gristede, Michael Guillorn, Howard Haynie, Michael Klaiber, Dongsoo Lee, Shih-Hsien Lo, Gary W. Maier, Michael Scheuermann, Swagath Venkataramani, Christos Vezyrtzis, Naigang Wang, Fanchieh Yee, Ching Zhou, Pong- Fei Lu, Brian W. Curran, Leland Chang, and Kailash Gopalakrishnan. Scalable Multi- TeraOPS Deep Learning Processor Core for AI Trainina and Inference. 2018 IEEE Symposium on VLSI Circuits, pp. 35-36, 2018
Angus Galloway, Graham W. Taylor, and Medhat Moussa. Attacking Binarized Neural Networks. 17.0449,,, arXiv e-prints, art. arXiv: 100449, Nov 2017.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adversarial Examples. arXiv e-prints, art. arXiv: 12.6572, Dec 2014.
I. K. Hansen and Salamon. Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(10):993-1001 Oct 1990. ISSN 0162-8828.
Warren He, James Wei, Xinyun Chen, Nicholas Carlini, and Dawn Song. Adversarial example defenses: Ensembles of weak defenses are not strong. In Proceedings of the 11th USENIX Con- ference Offensive Technologies, WOOT' 17, pp. 15-15, Berkeley, CA, USA, 2017. USENIX Association. URL http: //dl//dd// acm. org http:iitiiiiooooo cfm? -ine2dd=3111770 3154783.
Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El- Yaniv, and Yoshua Bengio. Quantized neural networks: Training neural networks with low precision weights and activations. J. Mach. Learn. Res.,
January 2017. ISSN 1532-4435.
E. Kilgariff, H. Moreton, N. Stam, and B. Bell. NVIDIA Turing Architecture In- Depth NVIDIA Developer Blog. [online] NVIDIA Developer Blog, 2019. URL https: htt::/ddvvllggg nvidia.
www.-raccchttture in in-ddeth/...
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial examples in the physical world. CoRR, abs/1607.022 02533, 2016. URL htRp:t/::// rxivv v. www.wiiwwv...........oo.o.ooeeee abs/ 02533.
10

---

Published as a conference paper at ICLR 2020
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-basede learning applied to document recog nition. Proceedings of the IEEE, 86(11):2278 2324, Nov 1998. ISSN 0018-9219.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards Deep Learning Models Resistant Adversarial Attacks. arXiv e-prints, art. arXiv:1706.0608. Jun 2017.
Priyadarshini Panda, Indranil Chakraborty, and Kaushik Roy. Discretization based solutions for secure machine learning against adversarial attacks. CoRR, abs/1902.03151,, 2019. URL
http://arxiv.. www.voorgaasss.... wwwsss9att 03151.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In ICML, 2019.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as Defense to Adversarial Perturbations against Deep Neural Networks. arXiv e-prints, art. arXiv:1511. 1.04508 Nov 2015.
Nicolas Papernot, Patrick D. McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Anan- thram Swami. The limitations of deep learning in adversarial settings. CoRR, 2015. URL http: aawai.iivvv..... org albs///////// 1511. 01128.7728
Nicolas Papernot, Fartash Faghri, Nicholas Carlini, lan Goodfellow, Reuben Feinman, Alexey Ku- rakin, Cihang Xie, Yash Sharma, Tom Brown, Aurko Roy, Alexander Matyasko, Vahid Behzadan, Karen Hambardzumyan, Zhishuai Zhang, Yi-Lin Juang, Zhi Li, Ryan Sheatsley, Abhibhav Garg, Jonathan Uesato, Willi Gierke, Yinpeng Dong, David Berthelot, Paul Hendricks, Jonas Rauber, and Rujun Long. Technical report on the cleverhans v2.1.0 adversarial examples library. arXiv preprint vrxiv:/610..00000 2018.
Andrew Slavin Ross and Finale Doshi-Velez. Improving the adversarial robustness and interpretabil- ity of deep neural networks by regularizing their input gradients. In AAAI, 2017.
Adnan Siraj Rakin, Jinfeng Yi, Boqing Gong, and Deliang Fan. Defend Deep Neural Networks Against Adversarial Examples via Fixed and Dynamic Quantized Activation Functions. arXiv e-prints, art, arXiv: 18Xiv:1807.000 0877006714 Jul 2018.
Thilo Strauss, Markus Hanselmann, Andrej Junginger, and Holger Ulmer. Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks. arXiv e-prints, art. arXiv:100.00342. Sep 2017.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Dan Boneh, and Patrick D. McDaniel. Ensemble adversarial training: Attacks and defenses. ArXiv, abs/1705.07204, 2017.
Swagath Venkataramani, Ashish Ranjan, Kaushik Roy, and Anand Raghunathan. AxNN: Energy- efficient neuromorphic systems using approximate computing. In Proceedings of the 2014 In- ternational Symposium on Low Power Electronics and Design, ISLPED 14, pp. 2732, New York, NY, USA. 2014. Association for Computing Machinery. ISBN 9781450329750. doi: 2t115//2266766..22711111 URL httht:t///// doi. http://ooooooo//////////o 11w....111............. 2627613.
Naigang Wang, Jungwook Choi, Daniel Brand, Chia-Yo Yu Chen, and Kailash Gopalakrishnan. Train- ing deep neural networks with 8-bit floating point numbers. NeurIPS, 2018
Shuchang Zhou, Zekun Ni, Xinyu Zhou, He Wen, Yuxin Wu, and Yuheng Zou. DoReFa- Net: Training low bitwidth convolutional neural networks with low bitwidth gradients. CORR, abs/1606.06166, 2016. URL http: / arxiv. org/ abs 1606. 6, 06160.
11

---

Published as a conference paper at ICLR 2020
A
BENCHMARK DETAILS
Network
Dataset
Configuration
C ReLU, Conv(6x6x 128), ReLU, C 128), ReLUU Fully Connected(10), SoftMax
MNISTcomy
MNIST
Conv(5x5x32), ReLU, X33)) Conv(8x8x64), ReLU 3vg Conv(8x8xx) 64), ReLU, Xv) Fully-Comeecte((6)) fully-Coomeected(10)) SoftMax
CIFARconv
CIFAR-10 10
Conv(12x12xx6), ReI.U, Conv(5x5x2560 BatchNorm, ReLU, MaxPool(3x3), X BatchNorm, ReLU, MaxPool(3 x3), x BatchNorm, ReLU, X x 256)), BatchNorm, ReLU,, MaxPool( x3),, fully_Conn(4096), BatchNorm, ReLU, Fully_ Connn(096), BatchNorm, ReLU, MallyyConn(1O0) (000), SoftMax
AlexNet
ImageNet
Table 3: Benchmarks
B ROBUSTNESS TO ATTACKS VARYING STRENGTHS
To further illustrate the benefits of EMPIR, observed its robustness under attacks of varying strength. We specifically varied the c value in the FGSM attack between 0.1 and 0.8 and the number of attack iterations in the CW attack between 10 and 90, and measured the adversarial accuracies of the EMPIR model as well as the baseline FP model for the MNISTconY benchmark. Figure 5 clearly illustrates that EMPIR exhibits higher adversarial accuracies across attacks of different strengths for both FGSM and CW.
FGSM attack
CW attack
Baseline FP
Adversorial Accurage 000
Adversarial Accuracy Accuraay
0.8
EMPIR
Baseline FP
0.6
EMPIR
0.2
0.2
0.4
0.6
0
20
40
60
80
100
Attack Iterations
(a)
(b)
Figure 5: (a) Adversarial accuracies under FGSM attack of varying strength (b) Adversarial accura- cies under CW attack of varying strength
12


Reviews:
[{"review_id": "HJgE7loI5H", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper suggests using ensemble of both full-precision and low-bits precision models to defense adversarial examples.\n\nFrom methodological point of view, this idea is quite straightforward and not novel, since there are already several works that applied ensemble methods to improve the robustness of NNs, including the Strauss et.al 2017 and (the following references are not included in the manuscript)\n\"Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong\nWarren He, James Wei, Xinyun Chen, Nicholas Carlini, Dawn Song\" \n\"Ensemble Adversarial Training: Attacks and Defenses\nFlorian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, Patrick McDaniel\" .\n\"Improving Adversarial Robustness via Promoting Ensemble Diversity\nTianyu Pang, Kun Xu, Chao Du,  Ning Chen,  Jun Zhu \" ICML 2019\n\nThough these methods only considered combining full-precision models, the idea is the same in essence and let the low-bits networks involve into the ensemble is quite natural and straightforward. So I don't think the methodology contribution of this paper is enough for publication.\n\nWhen checking the empirical results, the compared baselines miss a very common-used and strong baseline PGD adversarial training. And also the performance of this ensemble is not significant. \n\nConsidering the weakness of the paper both in methodology development and empirical justification, this work does not merit publication from my point of view. ", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "H1xg6DlTFr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose an ensemble of low-precision networks as a solution to providing a neural network with solid adversarial robustness whilst also providing good accuracy.\n\nI found the paper easy to read with a high quality introduction and background, the results are very convincing and the idea is simple but intriguing. I think this will shift the community towards seriously considering low precision networks a partial solution to adversarial attacks (alongside adversarial training).\n\nI could not work out from the paper whether the adversarial attacks on the low-precision networks were performed at full precision. I.e. someone could clone the low-precision networks, cast them to full precision, perform an adversarial attack like FGSM and then evaluate on the quantized network. It would be good to clarify this (or make it clearer in the text how you handle this).", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "BkgEnMcyqS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I think the paper reads well. It proposes to use ensembles of full precision and low-precision models in order to boost up robustness to adversarial attacks. It relies on the fact that low precision models are known to be more robust to adversarial attacks though performing poorly, while ensembling generally boosting up performance. \n\nI think the premise of the paper is quite clear, and the results seem to be intuitive.  At a high level one worry that I have is if ICLR is the right conference for this work. \n\nI would have expected maybe a more thorough empirical exploration. E.g. using resnets for ImageNet rather than AlexNet. Providing more baselines for the larger (and more reliable datasets) rather than MNIST which might be a bit misleading. I think the work does a decent job at looking at different number of components in the ensemble and analyzing the proposed method, but maybe not enough comparing and exploring other mechanism proposed as a defense for adversarial attacks. \n\nHowever I think the message is clear, the results seem decent and I'm not aware of this being investigated in previous works. ", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposed to apply emsembles of high precision deep networks and low precision ones to improve the robustness against adversarial attacks while not increase the cost in time and memory heavily.  Experiments on different tasks under various types of adversarial attacks show the proposed method improves the robustness of the models without sacrificing the accuracy on normal input.  The idea is simple and effective.  Some reviewers have had concerns on the novelty of the idea and the comparisons with related work but I think the authors give convincing answers to these questions.
Decision: Accept (Poster)