Ensemble-in-One: Learning Ensemble within Random Gated Networks for Enhanced Adversarial Robustness
Authors: Yi Cai
Xuefei Ning
Huazhong Yang
Yu Wang
Venue: ICLR 2022 Submitted (2022)
Keywords: Adversarial robustness
ensemble learning
random gated network
parameter sharing

Abstract: Adversarial attacks have threatened modern deep learning systems by crafting adversarial examples with small perturbations to fool the convolutional neural networks (CNNs). Ensemble training methods are promising to facilitate better adversarial robustness by diversifying the vulnerabilities among the sub-models, simultaneously maintaining comparable accuracy as standard training. Previous practices also demonstrate that enlarging the ensemble can improve the robustness. However, existing ensemble methods are with poor scalability, owing to the rapid complexity increase when including more sub-models in the ensemble. Moreover, it is usually infeasible to train or deploy an ensemble with substantial sub-models, owing to the tight hardware resource budget and latency requirement. In this work, we propose Ensemble-in-One (EIO), a simple but effective method to enlarge the ensemble within a random gated network (RGN). EIO augments the original model by replacing the parameterized layers with multi-path random gated blocks (RGBs) to construct an RGN. By diversifying the vulnerability of the numerous paths through the super-net, it provides high scalability because the paths within an RGN exponentially increase with the network depth. Our experiments demonstrate that EIO consistently outperforms previous ensemble training methods with even less computational overhead, simultaneously achieving better accuracy-robustness trade-offs than adversarial training. 

Under review as a conference paper at ICLR 2022
ENSEMBLE-IN-ONE: LEARNING ENSEMBLE WITHIN RANDOM GATED NETWORKS FOR ENHANCED AD- VERSARIAL ROBUSTNESS
Anonymous authors Paper under double-bling review
ABSTRACT
Adversarial attacks have threatened modern deep learning systems by crafting adversarial examples with small perturbations to fool the convolutional neural networks (CNNs). Ensemble training methods are promising to facilitate better adversarial robustness by diversifying the vulnerabilities among the sub-models, simultancously maintaining comparable accuracy as standard training. Previous practices also demonstrate that enlarging the ensemble ness. However, existing ensemble methods are with poor scalability, owing to the rapid complexity increase when including more sub-models in the ensemble. Moreover, it is usually infeasible to train or deploy an ensemble with substantial sub-models, owing the tight hardware resource budget and latency requirement. In this work, we propose Ensemble- in- One (EIO), a simple but effective method enlarge the ensemble within random gated network (RGN). EIO augments the original model by replacing the parameterized layers with multi-path random gated blocks (RGBs) to construct an RGN. By diversifying the vulnerability of the numerous paths through the super-net, it provides high scalability because the paths within an RGN exponentially increase with the network depth. Our experi- ments demonstrate that EIO consistently outperforms previous ensemble training methods with even less computational overhead, simultancously achieving better accuracy-robustnes trade-offs than adversarial training.
improve the robust-
INTRODUCTION
With the convolutional neural networks (CNNs) becoming ubiquitous, the security and robustness of neural networks is attracting increasing interests. Recent studies find that CNN models are inherently vulnerable to adversarial attacks (Goodfellow et al., 2014). which craft imperceptible perturbations on the images, referred without accessing the target model, an adversary can still generate adversarial examples from other surrogate models attack the target model by exploiting the adversarial transferability among them.
adversarial examples, mislead the neural network models. Even
Such vulnerability of CNN models has spurred extensive researches on improving the robustness against adversarial attacks. One stream of approaches targets on learning robust features for an indi- vidual model (Madry et al. 2017: Brendel et al. 2020). Informally, robust features are defined as the features that are less sensitive the adversarial perturbations added on the inputs. representative approach, referred to as adversarial training (Madry et al. 2017), on-line generates adversarial exam- result, adversarial training encourages the
ples on which the model minimizes the training loss. model to learn the features that are less sensitive to the adversarial input perturbations, thereby alle- viating the model's vulnerability. However, such adversarial training methods often have to sacrifice the clean accuracy for enhanced robustness (Zhang et al.) 2019), since they exclude the non-robust features and become less distinguishable for the examples with high similarity in the feature space.
Besides empowering improved robustness for individual model, another stream of researches fo- cuses on forming strong ensembles to improve the robustness (Yang et al.) 2020 Bagnall et al. 2017] Pang et al., 2019; Kariyappa & Qureshi 2019). Generally speaking, an ensemble constructed by aggregating multiple sub- models. Intuitively. an ensemble is promising to facilitate better robust- ness than individual model because successful attack needs mislead the majority of the

---

Under review as a conference paper at ICLR 2022
#sub-model
Baseline
ADP
GAL
DVERGE
3
1.0%/1.5%
0.0%:/9.6%
39.7% 11.4%
53.2%
0.0%
0.0%
32.4%
57.2%
8
0.0%
0.0%
22.4%
63.6%
Table I: Adversarial accuracy of the ensembles trained by different methods, with 3, 5, and 8 sub- models respectively (Yang et al., 2020). The number before and after the slash represent black-box and white-box adversarial accuracy under perturbation strength 0.03 and 0.01 respectively,
sub-models rather than just one. While the robustness of an ensemble highly relies on the diversity of the sub-models, recent study finds that CNN models trained independently on the same dataset are with highly-overlapped adversarial subspaces (Tramèr et al.) 2017). Therefore, many studies pro- pose ensemble training methods diversify the sub-models. For example, DVERGE (Yang al.] 2020) proposes to distill non-robust features corresponding to each sub-model's vulnerability, then isolates the vulnerabilities of the sub-models by mutual learning such that impeding the adversarial transferability among them.
There is another learned insight that the ensembles composed by more sub-models tend to capture greater robustness improvement. Table shows the robustness trend of the ensembles trained with various ensemble training methods. Robustness improvement can obtained by including more sub-models within the ensemble. This drives to further explore whether the trend will continue when keeping enlarging the ensemble. However, existing ensemble construction methods are with poor scalability because of the rapidly increasing overhead, especially with mutual learning which trains the sub-models in a round-robin manner, the complexity will rise at a speed of O(n)).
We propose Ensemble- in One, novel approach that improve the scalability of ensemble train- ing and introduce randomness mechanism for enhanced generalization, simultaneously obtaining better robustness and higher efficiency. For dedicated CNN model, we conduct Random Gated Network (RGN) by substituting each parameterized layer with a Random Gated Block (RGB) on top of the neural architecture. Through this, the network can instantiate numerous sub-models by controlling the gates in each block. Enneemble-in-On substantially reduces the complexity when scaling up the ensemble. summary, the contributions of this work are listed as below:
Ensemble-in -nnenb is simple but effective method that learns adversarially robust ensem- bles within one over-parametrized random gated network. The EIO construction enables employ ensemble learning techniques learn more robust individual models with minimal computational overheads and no extra inference overhead.
Extensive experiments demonstrate the effectiveness of Ensemble-in-One. It outperforms the previous ensemble training methods with even less computational overhead. Moreover, EIO also achieves better sccuracy-robustnee trade-offs than adversarial training method.
RELATED WORK
2.1 ADVERSARIAL ATTACKS AND COUNTERMEASURES.
The inherent vulnerability of CNN models poses challenges on the security of deep learning systems. An adversary can apply an additive perturbation on an original input generate adversarial example that induces wrong prediction in CNN models (Goodfellow et al. 2014). Denoting an input as x,, the goal of adversarial attacks is to find a perturbation ò s.t. Tady the model, where ||اا| satisfies the intensity constraint | S To formulate that, the adversarial attack aims at maximizing the loss L for the model with parameters on the input-label pair (as, y). i.c. ó argmax, (z: + 8, y). under the constraint that the E, norm of the perturbation should not exceed the bound e. Usually, we use foo norm (Goodfellow et al. 2014 Madry et al. 2017) of the perturbations to measure the attack's 's effectiveness smaller perturbation to successfully deceive the model is regarded to stronger. Correspondingly. defense that enforces the attacks enlarge perturbation intensity is regarded to be more robust.
can mislead
model's robustness. An attack that requires
Various adversarial attack methods have been investigated to strengthen the attack effectiveness. The fast gradient sign method (FGSM) (Goodfellow et al. 2014) exploits the gradient descent method
2

---

Under review as a conference paper at ICLR 2022
to generate adversarial examples. As improvement, many studies further show the attack can be strengthened through multi-step projected gradient descent (PGD) (Mddry et al. 2017) generation, random- random-starting strategy. and momentum mechanism (Dong al.) 2017). Then SGM (Wu et al. 2020) further finds that adding weight to the gradients going through the skip connections can make the attacks more effective. Other prevalent attack approaches include C&W losses (Carlini & Wag- ner, 2017b). M-DI--FGSM (Xie et al., 2019), etc. These attacks provide strong and effective ways to generate adversarial examples, rendering a huge threat to real-world deep learning systems.
To improve the robustness of CNN systems, there are also extensive countermeasures for adversarial attacks. One active research direction targets improving the robustness of individual models. Ad- versarial training (Madry et al. (017) optimizes the model on the udversarial examples generated in every step of the training stage. Therefore, the optimized model will tend to drop non-robust fea- tures to converge better on the adversarial data. However, adversarial training encourages the model to fit the adversarial examples, thereby reducing the generalization on the clean data and causing significant degradation of the clean accuracy.
2.2 TEST-TIME RANDOMNESS FOR ADVERSARIAL DEFENSE
Besides the aforementioned training techniques, there exist studies that introduce test-time random- ness to improve the robustness. Feinman et. al. (Feinman et al. (017) utilize the uncertainty measure in dropout networks to detect adversarial examples. Dhillon et. al. (Dhillon et al.) 2018) and Xie et. al. (Xie et al. al.]| (217)) incorporate layer-wise weighted dropout and random input transformations dur- ing test time improve the robustness. Test-time randomness is found to be effective in increasing the required distortion on the model, since test-time randomness makes generating white-box ad- versarial examples almost as difficult generating transferable black-box ones (Carlini & Wagner 2017a). Nevertheless, test-time randomness increases the inference cost and can be circumvented to some extent with the ponerattioooovee--raaffoommoomeo technique (Athalye et al. 2018).
2.3 ENSEMBLE TRAINING FOR ADVERSARIAL DEFENSE.
Besides improving the robustness of individual models, another recent research direction is in- vestigate the robustness of model ensembles in which multiple sub-models work together. The basic idea is that multiple sub-models can provide diverse decisions. Ensemble methods can combine mul- tiple weak models to jointly make decisions, thereby assembling as a stronger entirety. However, it demonstrated that independent training of multiple models tends capture similar features, which would not provide diversities among them (Kariyappa & Qureshi 2019).
Therefore, several studies propose ensemble training methods fully diversify the sub-models to improve the ensemble robustness. For example, Pan et. al. treat the distribution of output predic- tions as a diversity measurement and they propose an adaptive diversity promoting (ADP) regular- izer (Pang et al.) 2019) to diversify the non-max predictions of sub-models. Sanjay et. al. regard the gradients w.r.t. the inputs as discrimination of different models, thus they propose a gradient align-
ment loss (GAL) (Kariyappa & Qureshi, (219) which takes the cosine similarity of the gradients as a criterion to train the sub-models. The very recent work DVERGE (Yang et al. 20200) claims that the similar non-robust features captured by the sub-models cause high adversarial transferabilitty among them. Therefore, the authors exploit nob-ro feature distillation and adopt mutual learn- ing diversify and isolate the vulnerabilities among the sub-models, such that the within-ensemble transferability is highly impeded. However, as mentioned before, such ensemble methods are over- whelmed by the fast-increasing overhead when scaling up the ensemble. For example, DVERGE takes 11 hours to train an ensemble with three sub-models while needs approximately 50 hours when the sub-model count increases to eight. Therefore, a more efficient ensemble construction method is highly demanded to tackle the scaling problem.
3 ENSEMBLE-IN-ONE
3.1 BASIC MOTIVATION
The conventional way to construct ensembles is to simply aggregate multiple sub-models by aver- aging their predictions, which is inefficient and hard to scale up. An intuitive way enhance the
3

---

Under review as a conference paper at ICLR 2022
Aggrogate
Figure I: Normal ensemble training of multiple sub-models (Left) and the proposed ensemble-in- one training within random gated network (Right). By selecting the paths along augmented layers, the ensemble-in- one network can instantiate n sub-models, where n represents the augmentation factor of the multi-gated block for each augmented layer and L represents the number of augmented layers in the network.
scalability of the ensemble construction is introduce an ensemble for each layer in the network. As shown in [ we can augment a dynamic network by augmenting each parameterized layer with an re-path gated block. Then by selecting the paths along the augmented layer, the dynamic network can instantiate nz varied sub-models ideally. Taking ResNet-20 ing each convolution layer (ignoring the skip connection branch) with a two-path gated module, the overall path count will approach 2 524288. Such augmentation way provides approximation to training a very large ensemble of sub-models. Then through vulnerability diversification mutual learning, each path tends to capture better robustness. Following this idea, propose Ensemble- in-One to further improve the robustness of both individual models and ensembles.
example, by replac-
3.2 CONSTRUCTION OF THE RANDOM GATED NETWORK
Denote candidate neural network N(on, 02, Om.), where o; represents an operator in the network. To transform the original network into a random gated network (RGN), we first extract the neural architecture obtain the connection topology and layer types. On top of that, we replace each parameterized layer (mainly convolutional layer, optionally followed by batch normalization layer) with a random gated block (RGB). As shown in Fig. 2 each RGB simply repeats the original layer by times, and leverages binary gates with uniform probabilities to control the open or mutation of corresponding sub-layers. These repeated sub-layers are with different weight parameters. We denote the RGN as (Vd,, d2,..., dm where d, (o.1...., Oin). Let gi be the gate information in the deh RGB, then a specific path derived from the RGN can be expressed as P (g) d1,92- d2, ..... 9m d.).
For each RGB, when performing the computation, only one of the n gates is opened at a time, and the others will be temporarily muted. Thus by. only path of activation is active in memory during training, which reduces the memory occupation of training an RGN to the same level of training an individual model. Moreover, to ensure that all paths can be equally sampled and trained, each gate in a RGB is chosen with identical probability, i.e. 1/n if each RGB consists of n sub-operators. Therefore, the binary gate function can be expressed
1. 0, ..... 0] with probability 1/n,
00 1, ..... 0)
with probability 1/n,
9:
[0, 0, 0,.,....1 1] with probability 1/n.
An RGN is analogous to the super network in parameter-sharing neural architecture search, and the forward process of an RGN is similar to evaluating a ubbaarcitecture (Pham et al.) 2018 Cai et al. 2018). Compared to conventional ensemble training methods, method is easter to scale up the ensemble. It only incurs nx memory occupation for the weight storage, while still keeping the same memory requirement for activation individual model.
4

---

Under review as a conference paper at ICLR 2022
INPUT
Original Layer
Random Gated Block
CONV
CONV
CONVix
Original Layer (CONV3x3)
Random Gated Block (CONV3x3)
1/I
1/n
I/n
Probahility
Original Layer ++3
Random Gated Block
Binary
1
Forward
OUTPUT
Backward
(a) Construction of random gated network
(b) The schematic diagram of randomm gated block
The construction of random gated network based on random gated blocks. The forward propagation will select path to allow the input pass. Correspondingly, the gradients will also propagate backward along the same path.
Figure 2:
3.3 LEARNING ENSEMBLE IN ONE
The goal learning ensemble-in-one is to encourage the vulnerabilitics diversity of all the paths within the RGN by mutually learning from each other. Let Pi and P, be two different paths, where we define two paths as different when at least one of their gates different. To diversify the vulner- abilities, we need first distill the non-robust features of the paths so that the optimization process can isolate them. We adopt the same non-robust feature distillation strategy as previous work (Ilyas et al., 2019 Yang et al. 2020). Consider two randomly randomly-samples independent mabet-label pairs (Ze, Mr and (T Vo) from the training dataset, the distilled feature of z, corresponding to by the 41 layer of path P; can be achieved by:
d'pp (xt, = argmin, Bigmin_l1//.66 (1, s.t. || - xslloc S ed-
Such feature distillation aims to construct sample ''pp by adding perturbations 2, that the response in 4t layer of P. of zip: is similar as that of while the two inputs ia and are completely different and independent. This exposes the vulnerability path P, on classifying z.. Therefore, for another different path P,. it can learn on the distilled data to correctly classify them circumvent the vulnerability. The optimization objective for path Pj minimize:
E (r...) J (1'py (Ie, 22). V.).
As it is desired that each path can learn from the vulnerabilities of all the other paths, the objective of training the ensemble-in-one RGN is minimize:
Σ
Σ
Lin, (x/op (a),
(4)
VPJEN
VPIEN
where A is the of all paths the RGN. While is obviously impossible involve all the paths in training iteration, we randomly sample a certain number of paths by stochastically set the binary gates according to [ We denote the number of paths sampled in each iteration p. Then the selected paths can temporarily combine as subset of the RGN, referred to as S. The paths in the set S keep changing throughout the whole training process, such that all paths will have equal opportunities to be trained.
The training process of the RGN is summarized by the pseudo-code Algorithm Before starting vuinerability diversification training, we pre-train the RGN based on standard training settings to help the RGN obtain basic capabilities. The process simple, where a random path will be sampled in each iteration and trained on clean data. Then for each batched data, the process of vulnerability diversification contains three basic steps. First, randomly sample p paths to be involved in the iteration. Note that the sampled paths should be varied, i.e. if the distilling layer is set to I, for any P,, P, in S, there must be at least one different gate among the top gates, i.c. 3k e [1, s.t. P. [k] * P, [k). Second, distilling the vulnerable features of the sampled paths according to
5

---

Under review as a conference paper at ICLR 2022
Algorithm Training process for learning Ensemble-in-Om
Require: Path samples per ietration
Require: Random Gated Network N with L parameterized layers
Require: Pre-traine Pre-training epoch E... training epoch and data batch Ba
Require: Optimization loss C, learning rate Ir
Ensure: Trained Ensemble-ie One model
I: # pre-training of .V
2: for eo 1,2 F.w do
bo 1, 2, ..., Bg do
4:
Random Sample Path P, from N 5: Train P: in batched data
6: end for
end for
8: # learning vulnerability diversity for
9: for e 2,
E) do
10:
for 1, .... Ba) de
11:
Random sample [1, L]
12:
# randomly sample p paths
13:
S=[P1. P.. PpJ. s.t. Vi 3k s.. P. + PJ[k]
14:
Get data (X,, Y.). (X.,, Y.) +-)
15:
# Get distilled data
16:
for 2, p do
17:
X
=
x pl X.)
18:
end for V t
19:
20:
for = 1, .... do
21:
Vp. V( spti Lrv, (X,(
22:
VN VN + Vp.
23:
end for
24:
= Ir VN
25:
end for
26:
end for
Eq. 2 The distillation process is the same as proposed in DVERGE, by applying PGD scheme for approximating the optimal perturbations. Third, mutually train each path with the distilled data from the other paths in round-robin manner. Because the paths unavoidably share proportion of weights owing to the weight sharing mechanism super-net, the gradients of the weights will not be updated until all sampled paths are included.
3.4 MODEL DERIVATION AND DEPLOYMENT
Once the training of RGN finished, we can then derive and deploy the model in two ways. One way is to deploy the entire RGN, then in inference stage, the gates throughout the network will be randomly selected to process an input. The advantage is that the computation is randomized, which may beneficial for improving the robustness under white-box attacks, because the transferability among different paths was impeded during diversity training. However, the disadvantage is that the accuracy is unstable owing to the dynamic choice of inference path, where the fluctuation reaches 1-2 percentage.
Another way to derive individual models from the RGN. By sampling a random path and elim- inating the other redundant modules, an individual model can be rolled out. We can also sample multiple paths and derive multiple models to combine as an ensemble. Deploying models in this way ensures the stability of the prediction the randomness is eliminated. In addition, the derived models can be slightly finetuned with small learning rate for a few epochs to compensate for the under-convergence
the training process of RGN cannot fully train all paths the probability of each specific path being sampled is relatively low. our implementation, we exploit the latter method to derive individual model for deployment.

---

Under review as a conference paper at ICLR 2022
100
100
bbox
80
2m2 Abox
wfooc
000000 boox
thox
20000, dous
30
0.01
0.02
0.08
0.04
0.05
0.08
0.07
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.02
0.08
0.00
Perturbation Strength
Perturbation Strongth
Perturation Strength
Figure 3: Investigation on the training. All these experiments are implemented ResNet-20 over CIFAR-10 dataset. Left: The black-box adversarial accuracy under different sample count per iteration; Middle: The black-box adversarial accuracy under different distillation perturbation cz; and Right: the adversarial accuracy under different augmentation factor n.
involved in the Ensemble-in-On construction and
EXPERIMENTAL RESULTS
4.1 EXPERIMENT SETTINGS
Benchmark. The experiments are constructed on the ResNet-20 (He et al. 2016) and VGG-16 networks with the CIFAR dataset (Krizhevsky et al. 2009). Specifically, we construct the ResNet- 20 and VGG-16 based RGNs by substituting each convolution layer to a n-path RGB (in default 2). Overall, there are 19 RGBs (containing 19 convolution layers in the straight-through branch) for ResNet-20 and 14 RGBs for VGG-16 (containing the 14 convolution layers). To evaluate the effectiveness of our method, we compare Ensemble-in-O0 with multiple counterparts, including the Baseline which trains the models in a standard way and three previous ensemble training methods:
ADP (Pang al. 2019), GAL (Kariyappa & Qureshi 2019), and DVERGE (Yang al. 2020) Meanwhile, we also add the adversarial training (Adv?) method into the comparison.
Training Details. The trained ensemble models of Baseline, ADP. GAL and DVERGE are referred to the implementation which is publicly released in (Yang et al. 2020 We train the Ensemble-in- One networks for 200 epochs using SGD with momentum 0.9 and weight decay 0.0001. The initial learning rate is 0. and decayed by 10x at the 100-th and the 150-th epochs respectively. When deriving the individual models, we fine-tune the derived models for 0-20 epochs using SGD with learning rate 0.001, momentum 0.9 and weight decay 0.0001. Note that the fine-tuning process is optional and can adjust the epochs for a dedicated model. default, for an RGN training, we sample 3 paths construct temporary eabbenl per iteration. The augmentation factor for each RGB is set to 2, and the PGD-based perturbation strength eg for feature distillation is set to 0.07 with 10 iterative steps and each step size of ca/ 10.
Attack Models. We categorize the adversarial attacks as black-box transfer attacks and white-box attacks. The white-box attack assumes the adversary has full knowledge of the model parameters and architectures, and the black-box attack assumes the adversary cannot access the target model and can only generate adversarial examples from surrogate models launch the attacks. For fair comparison, we adopt exactly the same attack methodologies and the same surrogate models as DVERGE evaluate the robustness. The detailed adversarial settings can be found in Appendix We believe the attacks are powerful and can identify the robustness of the various models.
A.1
4.2 ROBUSTNESS EVALUATION
Hyper-parameter Exploration. Recall that three important hyper- syper-parameters are involved in the training procedure. One is the count of sampled paths to participate in each training iteration, one is the strength of feature distillation perturbation eœ as illustrated in Bq and the other the augmentation factor n for constructing the RGN, i.e. how many times will an operator be repeated

---

Under review as a conference paper at ICLR 2022
Black-box Attuck
White-box Attack
Baseined
100
ResNot-20
Beseinws
ReaNet-20
80
BaselinolB
80
ADP3
ADPIS
Accuracy
40
GAUS
40
QAL-8
DVERGEO
DVERGES
DVERGE
0.01
0.02
0.0G
0.04
0.05
0.08
0.07
0.01
0.02
0.03
0.04
0.05
0.06
0.07
Perturbation Strength c
Perturbation Strength
100
VGG-16
VGG-16
80
80
Accurccy (16)
Baselinn33
60
ADP/S
GAL-S
DVERGE
20
0.01 0.02
0.03
0.04
0.05
0.06
0.07
Perturbution Strength
Perturbution Strength
Figure 4: Contrasting the robustness of Ensemble-in-One with previous ensemble training meth- ods. Left: adversarial accuracy under black-box transfer attack; and right: adversarial accuracy under white-box attack. The number after the slash stands for the number of sub-models within the ensemble. The evaluations include ResNet-20 and VGG-16 over the CIFAR-10 dataset. The distillation perturbation strength of "GG-16-based EIO is ed 0.03.
to build a RGB. We make experiments to investigate the impact of these hyper-parameters on the clean accuracy and the adversarial robustness.
3 (Lefft) shows the curves of block-box adversarial accuracy under different sampled path count per training iteration. As is observed, when the sampled paths increase, the robustness of the derived individual model also improves. The underlying reason is that more samples of paths participating each iteration allows more paths to be mutually trained, thereby each path is expected learn from more diverse vulnerabilities. However, the clean accuracy drops with the increasing of path sample count, because a single operator has to adapt to diverse paths simultaneously. Moreover, the training time will also increase as the training complexity satisfies Opp²).
Fig (Middle) shows the curves of black-box adversarial accuracy under different feature distilla- tion eg. We find similar conclusions presented in DVERGE. A larger ea push the distilled data "/pp (Ie, x.)) share more similar internal representation as 2,, While the objective to reduce the loss of P, on classifying z the larger loss will boost the effectiveness of learning the vulnera- bility, thereby achieving better robustness. However, we also find the clean accuracy drops with the increase of ed. And there exists a switching point where will stop obtaining robustness improve- ment from continually increasing eg. The experimental results suggest e 0.07 to achieve higher robustness and clean accuracy simultaneously.
Fig 3 (Right) shows the comparison of adversarial accuracy when applying different augmentation factor n for constructing the RGN. Observe that increasing the factor n brings no benefit on either the clean accuracy or adversarial accuracy. It stands to reason that augmenting 2x operators for each RGB has already provided sufficient candidate paths. Moreover, increasing the n leads to more severe under-convergenee of training because each path has decreased probability of being sampled. Therefore, we suggest the augmentation factor as n=2 for each convolution layer.
Comparison with Ensemble Methods. Fig 4 shows the overall adversarial accuracy of the mod- els trained by different methods with a wide range of attack perturbation strength. ResNet-20 and VGG-16 16 selected as the basic network to construct the ensembles and the EIO super-networks. The results show that through our Ensemble- in- One method, an individual model derived from the RGN can significantly outperform the heavy ensembles trained by previous ensemble training meth-
8

---

Under review as a conference paper at ICLR 2022
bbx Attack
Woxte-hox Attack
100
100
Accuracy (%)
80
80
AdvT, -000000
AdvT, c10.00
60
EID, h-epoch-5
60
AdvT, c=0.031
40
AdVT, em0.00
40
Advi, -00.00
AdvT, c40.005
20
EIO. /tepoch=S
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.01
0.02
0.03
0.04
0.05
0.08
0.07
Perturbation Strength
Perturbation Strength
Figure 5: Contrasting the robustness of Ensemble-in-O and AdvT with different adversarial per- turbation settings. The experiments are implemented on ResNet-20 over CIFAR-10. The "ft-epoch" means the fine-tuning epoch of the derived model. When aligning the clean accuracy, EIO achieves better robustness than AdvT.
ods with higher adversarial accuracy under both black-box and white-box attacks, simultaneously achieving comparable clean accuracy. More experiments and analysis can refer to the Appendix[A2
AA33] and A.4 These results demonstrate that we successfully train an ensemble within one RGN network and improves the robustmess of an individual model to outperform the ensembles such that the deployment overhead can be substantially reduced.
Comparison with Adversarial Training. AdvT has been demonstrated as a promising approach on enhancing the robustness. Prior work attributes the enhancement to the exclusion of non-robust features during AdvT. However, these non-robust features might be usefiul to the classification ac- curacy, resulting in trade-offs between the clean accuracy and the robustness. One can adjust the perturbation strength in the AdvT to acquire different combinations of clean accuracy and adver- sarial robustness, as shown Fig 5 can be figured out that EIO significantly outperforms AdvT when aligning their clean accuracy (AdvT w/ 0.005), which suggests that EIO learns more useful, robust features while excluding more useless, non-robust features than AdvT.
5 DISCUSSION & FUTURE WORK
There are also several points that are worthy further exploration while we leave to future work. First, current implementation of augmenting the RGN is simple, by repeating the convolution layers for multiple times. Nevertheless, as observed in Fig (Right), enlarging the augmentation factor brings no benefit on improving the robustness. Hence, there might be better ways of constructing RGNs that can compose stronger randomized network, e.g. subtracting some of the unnecessary RGBs augmenting by diverse operators instead of simply repeating. Second, although black- box attacks are more prevalent in real world, defending against white-box attacks is still in demand because recent research warns the high risks of exposing the private models the adversary (Hua et al. 2018: Hu et al. 2020). Randomized multi-path network can provide promising solutions to alleviating the white- box threats. If the adversarial transferability among the different paths can be impeded, the adversarial example generated from one path will be ineffective for another path. Hence, it will make the white-box attacks as difficult as black-box transfer attacks. We believe valuable direction to explore defensive method based randomized multi-path network.
6 CONCLUSIONS
In this work, we propose Ensemble-in-One, a novel approach that constructs random gated network (RGN) and learns adversarially robust ensembles within the network. The method is inherently scalable, which can ideally instantiate numerous sub-models by sampling different paths within the RGN, By diversifying the vulnerabilities of different paths, the Ensemble-in -eeemble--i--neeeee method can efficiently obtain models with higher robustness, simultaneously reducing the overhead of model training and deployment. The individual model derived from the RGN shows much better robustness than previous ensemble training methods and achieves better trade-offs than adversarial training.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of se- curity: Circumventing defenses to adversarial examples. In International Conference on Machine Learning, pp. 274-283. PMLR, 2018.
Alexander Bagnall, Razvan Bunescu, and Gordon Stewart. Training ensembles to detect adversarial examples. arXiv preprint arXiv: 171.0040... 2017.
Wieland Brendel, Jonas Rauber, Alexey Kurakin, Nicolas Papernot, Behar Veliqi, Sharada P Mo- hanty, Florian Laurent, Marcel Salathé, Matthias Bethge, Yaodong Yu, et al. Adversarial vision challenge. In The NeurlPS 18 Competition, pp. 129-153. Springer, 2020.
Han Cai, Ligeng Zhu, and Song Han. Proxylessnas: Direct neural architecture search on target task and hardware. arXiv preprint arXiv:/812.00332. 2018.
Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 3-14, 2017a.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). pp. 39-57. IEEE, 2017b.
Guneet S Dhillon, Kamyar Azizzadenesheli, Zachary C Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, and Anima Anandkumar. Stochastic activation pruning for robust adversarial de- fense. arXiv preprint 1rXiv:1803.014 2018.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Xiaolin Hu, and Jun Zhu, Discovering adversarial 2017.
examples with momentum. arXiv preprint
Reuben Feinman, Ryan R Curtin, Saurabh Shintre, and Andrew B Gardner. Detecting adversarial samples from artifacts. arXiv preprint arXiv:1703.004100 2017.
Ian J Goodfellow, Jonathon Shlens. and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6574 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Xing Hu, Ling Liang, Shuangchen Li, Lei Deng, Pengfei Zuo, Yu Ji, Xinfeng Xic, Yufci Ding, Chang Liu, Timothy Sherwood, et al. Deepsniffer: A dnn model extraction framework based on learning architectural hints. Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 385-3993999 2020.
Weizhe Hua, Zhiru Zhang, and G Edward Suh. Reverse engineering convolutional neural networks through side-channel information leaks. In 2018 55th ACM/ESDA/IEEE Design Automation Con- ference (DAC), pp. 1-6. IEEE, 2018
Andrew lyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. arXiv preprint 2019.
Sanjay Kariyappa and Moinuddin K Qureshi. Improving adversarial robustness of ensembles with diversity training. arXiv preprint (tt:::901.09900 2019.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.
Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 17X.:110660608. 2017.
10

---

Under review as a conference paper at ICLR 2022
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning, pp. pp. 4970-497777 PMLR. 2019.
Hieu Pham, Melody Guan, Barret Zoph, Quoe Le, and Jeff Dean. Efficient neural architecture search via parameters sharing. In International Conference on Machine Learning, pp. 4095- 4104. PMLR. 2018.
Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. The space of transferable adversarial examples. arXiv preprint UrXiv:1704.03453 2017.
Dongxian Wu, Yisen Wang. Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip connections matter: On the transferability of adversarial examples generated with resnets. arXiv preprint arXiv:2002.059990 2020.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. arXiv preprint 1rXiv:111.0199)) 2017.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L. Yuille. Improving transferability of adversarial examples with input diversity. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 2730-2739, 2019.
Huanrui Yang, Jingyang Zhang, Hongliang Dong, Nathan Inkawhich, Andrew Gardner, Andrew Touchet, Wesley Wilkes, Heath Berry, and Hai Li. Dverge: diversifying vulnerabilities for en- hanced robust generation of ensembles. arXiv preprint arXiv:2009. htti::2099./72000 /4720, 2020.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, pp. 7472-7482. PMLR, 2019.

---

Under review as a conference paper at ICLR 2022
Black- box Attack
White-box Attack
80
80
ResNct-20
Baselin/311
ResNct-20
Baseline311
GAL/2/1
GAL/311
Accuracy (%)
60
AdvT (v=0.081)
DVERGE/8/1
Accuracy (%)
60
AdVT (cw0.031)
DVERGE/811
EIO
EIO
40
20
20
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.01
0.02
0.03
0.04
0.06
0.06
0.07
Perturbation Strength
Perturbation Strength
Figure 6: Contrasting the robustness of Ensemble-in-One with other methods CIFAR-100 dataset. Because ensemble training brings significant clean accuracy enhancement on CIFAR-100 dataset (e.g. individual baseline 67 67.2% v.s. 3-sub-model ensemble baseline: 72.6%), we randomly sample an individual model from the ensemble for fair comparison. The number after the first slash repre- sents the number of sub-models contained in the ensemble, and the number after the second slash represents that an individual model is sampled from each ensemble for evaluation. The ADP method excluded because its training on CIFAR-100 100 fails to converge in our reproduction.
A APPENDIX
In this appendix, we provide some additional results further discuss the advantages and disadvan- tages of our Ensemble-in-One method and other previous methods.
A.1 ADVERSARIAL SETTINGS
We adopt strong attack methodologies and settings for evaluating the robustness of CNN models. For black-box transfer attacks, the involved attack methods include: (1) PGD with momentum and with three random starts (Madry et al. 2017); (2) M-DI--FGSM (Xie et al., 2019); and (3) SGM (u et al. (020). The attacks are with different perturbation strength and the iterative steps set to 100 with the step size of c/5. Besides the cross-entropy loss, we also apply the C&W loss to incorporate with the attacks. Therefore, there will be 3 (surrogate models) 5 (attack methods, PGD with three random starts, M-DI--FGSM. and SGM) x 2 (losses)= = 30 adversarial versions. For white-box attacks, we apply 50-step PGD with the step size of e/5 with five random starts. Both the black-box and white-box adversarial accuracy is reported in an all-or-nothing fashion: a sample judged be correctly classified only if its 30 (for black-box attack) or 5 (for white-box attack) adversarial versions are all corrected classified by the model. In default, we randomly sample 1000 instances from the test dataset for evaluation.
A.2 CIFAR- 100 EVALUATION
We further evaluate the methods 100 dataset which utilizes the ResNet-20 basic network. We apply the default hyper-parameter configurations for training the EIO network. The black-box setting utilizes 10 adversarial versions (2 surrogate models x 5 random starts applied in PGD) for transfer attacking the target model, and the white-box attack keeps the same setting as described before. As shown in Fig 6 training methods whether under black-box or white-box attack scenarios. AdyT shows much better robustness while sacrificing significant clean accuracy. Overall, the results again suggest that EIO is a more effective approach for improving the adversarial robustness.
EIO still significantly outperforms the other ensemble
A.3 TRAINING CURVE AND COST OF ENSEMBLE-IN-ONE
We investigate the training convergence of EIO. The training curves of baseline and EIO training shown in Fig 7 (Left). Two training stages are involved in EIO training. One is the diversity training of the RGN, and the other the fine-tuning of the extracted single path. EIO slightly deteriorates the convergence effectiveness than baseline training, while still obtaining competitive
12

---

Under review as a conference paper at ICLR 2022
100
Training Curve
Training Time
90
Training Time (hours)
Clean Accurscy (3)
80
Fine-tune
70-
-20
Baseline Training Curve (wio AdvT)
60
EIO protrained weight
EIO w/ protrained weight
10
80
40
50
100
150
200
Baseline
AdvT DVERGE: DVERGESS OVERGE
EIO
Epoch
Figure 7: The training curves and time cost of Ensemble-in-One The experiments are made on ResNet-20-based EIO training, with default eyper-parameters Left: The clean validation accuracy throughout the training process. The fine-tuning epoch is extended to 40 epoch. As is observed, 20 epochs adequate to help the derived model fully converge. Right: The training time cost of Baseline, AdvT, DVERGE, and EIO, which are evaluated on an NVIDIA GeForce RTX 2080 Ti.
90
80
ResNel-20
VGG-16
Accurncy (56
70
Random Path
Random Path
Random Path
Accuracy
Random
Random Path
Random Path
Random Path 4
50
Random Path
Random Path
Path
Random
Rancom Path
Random Path 8
Random Path
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.02
0.05
0.07
Perturbation Strength
Perturbation Strength
Figure 8: Investigation on the stability of derived models from the RGN. The experiments are made on CIFAR-10 dataset based ResNet-20 and VGG-16 16 respectively. Black-box adversarial accuracy evaluated.
clean accuracy after a slight fine-tuning. We also summarize the training time cost of different methods in Fig (Right). Because we use a PGD-based data distillation for training and sample paths per training iteration, the training time cost of an EIO network approximately 2.5x than AdvT. While the training time is substantially reduced compared to the DVERGE when scaling up the ensemble. As for the GPU memory and FLOPS cost, training EIO closely equivalent to training individual model because by controlling the gating, only sub-model is involved at each training step. As is desired, our approach significantly reducing the training cost (both time and hardware cost), simultaneously obtaining better robustness than conventional ensemble training.
A.4 MODEL DERIVATION STABILITY
In the deployment phase, an individual model (or several models) will be derived from the random gated network (RGN) and fine-tuned for few epochs. Because the model derived by randomly sampling a path in the RGN, it is important to ensure the stability of derived models. Hence, we randomly derive eight sub-models from a same RGN and test their performance and robustness. As can be observed from Fig 8 the sampled eight sub-models demonstrate almost the same robustness with very slight fluctuations on the adversarial accuracy against the black-box transfer attacks. Thus, the results be seen evidence that the derivation is a stable process and the randomness would not bring high variance on the derived models.
13


Reviews:
[{"review_id": "luRxFYCaDoh", "reviewer": "Reviewer_a3La", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n1.The paper is well written and easy to read.\n2.The idea is interesting and clearly explained.\n3.Experimental setup is comprehensive and appropriate ablation studies have been performed.\n\nCons:\n1.Adversarial training (AT) seems a better option. As shown in Figure 5&6, the AT can easily beat EIO as perturbation strength increase from 0.01 under the White-box setting. Considering the AT only takes less than half the training time of EIO (Figure 7), it makes EIO less attractive. It seems that EIO can be easily combined with AT together. I would suggest that the author conducts the experiment to check the performance of the combination of EIO and AT.\n\n\n2.Lack of interpretation of results. Since the DVERGE and EIO share the same vulnerability diversification training, why EIO can outperform DVERGE significantly? Is it because EIO can impede more adversarial transferability among sub-models within the ensemble? In Figure 6, why do you only take one sub-model in the ensemble to evaluate the performance of GAL and DEVRGE? Is that the reason why GAL and DEVRGE completely fail to defend against White-box attacks?\n\n3.The major contribution of the paper is to construct ensemble by repeating the original layer by n times. However, the increase of this key parameter n (from 2 to 3) will lead to performance degradation (Figure 3), which does not match the intuition of the paper.  Given enough training time, will the model trained with n=3 beat the model trained with n=2? \n\nMinor:\n1.\tTable 1 is not clear, what does the number mean before and after the slash, is that accuracy against White-box/Black-box attacks?\n2.\tThere are typos in legend of Figure 4 (Baseline.5 and DVERGE.8).\n", "summary_of_the_paper": "This paper proposed a new way to generate an ensemble of networks against adversarial attacks. Different from other methods, which train different sub-models, the proposed method repeats convolution layers multiple times and controls them with random gates.  The experiment demonstrates that it outperforms other ensemble training with a smaller computational overhead. ", "summary_of_the_review": "Overall, I think this paper is marginally below the acceptance threshold.  I like the idea of the ensemble with random gates in deep neural networks. Although this method can beat other ensemble methods, it is less attractive than basic adversarial training since the latter takes less training time and is flexible to be tuned with different perturbation strengths. If my concerns are addressed, I would like to raise the score.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "jT9RBnT4Upz", "reviewer": "Reviewer_vVnY", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. It seems to me that this training is complicated and computationally expensive. The defender needs to consider (1) different data pairs (N^2) (2) all possible paths (although the authors choose p paths in each data batch, the training epoch should be much larger to cover possible paths) (3) generating different clean-label adversarial perturbations. \n\n2. Some parts of the training process are quite confusing:  (1) My understanding of this training process is that it forces some paths to remember certain perturbation patterns. But how can the method guarantee that (a) the memorizations will not be rewritten by other perturbations?  (b) adversarial inputs will go into the corresponding paths in the inference stage? And what is the benefit of using such a method compared with forcing the neural network to remember all perturbations? (2) The objective here is to generate a perturbation that relies on the path and certain training data. In fact, Equation 2 and its variants can be used to generate clean-label attacks. However, its application here is not for defending such an attack. I don't understand why the authors choose to use equation 2 instead of using the conventional non-targeted adversarial attack. Also, if one considers $l$ to be the output layer, then basically, the adversarial generation in RGN reduces to a targeted attack. Of course, using the targeted attack in robust training is not plausible.\n\n3. The inference phase of RGN searches a subnetwork to defend against attacks. This raises a question. Does the robustness come from its complicated training process, or does it come from the distillation? In fact, many works have explored the lottery ticket hypothesis in the standard training and robust training regimes, e.g., [1].\n\n[1] Towards Practical Lottery Ticket Hypothesis for Adversarial Training", "summary_of_the_paper": "This paper proposes a robust training and defending method RGN by applying control gates with binary status. During the training, the proposed method generates adversarial examples in a clean-label attack manner and mitigates the adversarial perturbation through training on another path. During the inference, RGN finds a subnetwork to defend against adversarial attacks.", "summary_of_the_review": "Although applying control gates is an interesting idea, I have some concerns about the RGN method. I have listed my major concerns in the main review.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "HgA1l2CK70", "reviewer": "Reviewer_7WHx", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The idea of using random gated network (RGN) to make the ensemble scalable is very interesting.\n2. Strong empirical results when compared with SOTAs.\n\nWeakness:\n1. Instead of just presenting the numerical results, the authors could try to provide some theoretical analysis on the proposed methods. A good example could be the ICLR 2018 paper: Ensemble Adversarial Training: Attacks and Defenses \n2. The concept of the random gated network is similar to the work of stochastic path network (``Deep Networks with Stochastic Depth\"). \n3. The authors discussed two ways to deploy the final model in the testing phase. Why not consider the case where we just aggregrating the results, possibly by average  pooling, from different paths at each layer ?", "summary_of_the_paper": "This paper uses ensemble learning to improve the adversarial robustness. The authors introduce the concept of ``Ensemble-in-One\",  in which a simple but effective method called random gated network (RGN) is ultilized to enlarge the ensemble size in a very effective manner.", "summary_of_the_review": "The idea is quite interesting and the results are good. However, the current version lacks the theoretical analysis and other key empirical comparisons as mentioned above.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "46xJx3WhiJK", "reviewer": "Reviewer_wUHR", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. The random gated block enable the efficient/scalable model ensemble but the introduced randomness may greatly decrease the model capacity. It would be great if ImageNet dataset can be tested to prove algorithm can fit highly complex data distribution.\n2. The random gated method looks very similar to applying dropout not only during the model training phase but also the model inference phase. It would be good to clearly address the key difference between random gating and dropout. It would be expected that some similar theoretical discussions like the following paper can be included in the draft.\n(1) Gal, Yarin, and Zoubin Ghahramani. \"Dropout as a bayesian approximation: Representing model uncertainty in deep learning.\" In international conference on machine learning, pp. 1050-1059. PMLR, 2016.\n", "summary_of_the_paper": "This paper proposes a scalable ensemble training with random gated block to enhance the model adversarial robustness.", "summary_of_the_review": "The model capacity can be a big concern and the technical depth is somehow limited. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a stochastic network, named Ensebmle-in-One (EIO), to increase adversarial robustness. EIO replaces the layers in a given architecture by so called random gated blocks (RGBs) in which a random gate switches between multiple copies of the original layers. By sampling from the random gates different subnetworks can be sampled which can be arranged to form an ensemble. During training non-robust feature distillation (as proposed in previous work) between models is applied. For inference in the experiments a single subnetwork is sampled, and the robustness of that subnetwork is compared against several ensemble methods and adversarial training.  

One reviewer was worried about model capacity and recommended to perform experiments on image net to demonstrate scalability to large datasets. In turn, authors added experiments on CIFAR-100 during rebuttal period. Another critique was that the model does not show a significant advantage over vanilla adversarial training (AT), which can easily tuned with different perturbation strengths and only takes half of the training time. While other ensemble techniques, like DVERGE, can be combined with AT to improve their robustness, combing EIO with AT does not lead to improvements as shown by experiments performed during the rebuttal period. Two reviewers stated that adding a theoretical analysis will improve the paper. Another suggestion for improving the paper was to add a comparison to stochastic path networks, which is related work, and to investigate model performance when results from several sub-networks are aggregated. 

Overall, the paper can not be accepted in its current state, but I would recommend the authors to continue the direction of work and to incorporate reviewers suggestions in a future version of the manuscript.
Decision: Reject