A Panda? No, It's a Sloth: Slowdown Attacks on Adaptive Multi-Exit Neural Network Inference
Authors: Sanghyun Hong
Yigitcan Kaya
Ionuț-Vlad Modoranu
Tudor Dumitras
Venue: ICLR 2021 Spotlight (2021)
Keywords: Slowdown attacks
efficient inference
input-adaptive multi-exit neural networks
adversarial examples

Abstract: Recent increases in the computational demands of deep neural networks (DNNs), combined with the observation that most input samples require only simple models, have sparked interest in input-adaptive multi-exit architectures, such as MSDNets or Shallow-Deep Networks. These architectures enable faster inferences and could bring DNNs to low-power devices, e.g., in the Internet of Things (IoT). However, it is unknown if the computational savings provided by this approach are robust against adversarial pressure. In particular, an adversary may aim to slowdown adaptive DNNs by increasing their average inference time—a threat analogous to the denial-of-service attacks from the Internet. In this paper, we conduct a systematic evaluation of this threat by experimenting with three generic multi-exit DNNs (based on VGG16, MobileNet, and ResNet56) and a custom multi-exit architecture, on two popular image classification benchmarks (CIFAR-10 and Tiny ImageNet). To this end, we show that adversarial example-crafting techniques can be modified to cause slowdown, and we propose a metric for comparing their impact on different architectures. We show that a slowdown attack reduces the efficacy of multi-exit DNNs by 90–100%, and it amplifies the latency by 1.5–5× in a typical IoT deployment. We also show that it is possible to craft universal, reusable perturbations and that the attack can be effective in realistic black-box scenarios, where the attacker has limited knowledge about the victim. Finally, we show that adversarial training provides limited protection against slowdowns. These results suggest that further research is needed for defending multi-exit architectures against this emerging threat. Our code is available at https://github.com/sanghyun-hong/deepsloth. 

Published as a conference paper at ICLR 2021
A PANDA? No, IT's S A SLOTH: SLOWDOWN ATTACKS ON ADAPTIVE MULTI-EXIT NEURAL NETWORK INFERENCE
Sanghyun Hong', Yigitcan Kaya', Ionut-Vlad Modoranu', Tudor Dumitras
University of Maryland, College Park, USA i Alexandru Ioan Cuza University, Iaşi, Romania
shheng@cs udd, yigi sigtccacccs.... umd. edu, modoranu, onut vl vlad@hotmai com, tudor@umd.edu
ABSTRACT
Recent increases in the computational demands of deep neural networks (DNNs), combined with the observation that most input samples require only simple models, have sparked interest in input-adaptive multi-exit architectures, such as MSDNets or Shallow-Deet Networks. These architectures enable faster inferences and could bring DNNs to low-power devices, e.g., in the Internet of Things (IoT), However, it is unknown if the computational savings provided by this approach are robust against adversarial pressure. particular, an adversary may aim to slowdown adaptive DNNs by increasing their average inference time threat analogous the denial-of-servee systematic evaluation of this threat by experimenting with three generic multi-exit DNNs (based on VGG16, MobileNet, and ResNet56) and custom multi-exit architecture, on two popular image classification benchmarks (CIFAR-IF 10 and Tiny ImageNet). To this end, we show that adversarial example-crafting techniques be modified their impact on different architectures. We show that slowdown attack reduces the efficacy of multi-exit DNNs by 90-100%, and it amplifies the latency by 1.5-5x in a typical IoT deployment. We also show that it is possible to craft universal, reusable perturbations and that the attack can be effective in realistic black-box scenarios, where the attacker has limited knowledge about the victim. Finally, we show that adversarial training provides limited protection against slowdowns. These results suggest that further research is needed for defending multi-exit architectures against this emerging threat. Our code is available bttps://githu..comm//e
attacks from the Internet. In this paper, we conduct a
cause slowdown, and we propose metric for comparing
INTRODUCTION
The inference-time computational demands of deep neural networks (DNNs) are increasing, owing the "going deeper" (Szegedy et al., 2015) strategy for improving accuracy: as a DNN gets deeper, it progressively gains the ability to learn higher-level, complex representations. This strategy has enabled breakthroughs in many tasks, such as image classification (Krizhevsky et al. 2012 or speech recognition (Hinton et al. 2012). at the price of costly inferences. For instance, with 4x more inference cost, 56-layer ResNet (He et al. 2016) improved the Top-1 accuracy on ImageNet by 19% over the 8-layer AlexNet. This trend continued with the 57-layer of-te-of-the-arr EfficientNet (Tan Le 2019: ; it improved the accuracy by 10% over ResNet, with 9x costlier inferences.
The accuracy improvements stem from the fact that the deeper networks fix the mistakes of the shallow ones (Huang et al. 2018). This implies that some samples, which are already correctly classified by shallow networks, do not necessitate the extra complexity. This observation has motivated research on input-adaptive mechanisms. in particular, multi-exit architectures (Teraapitaaanon al. 2016 Huang et al.) 2018, Kaya al., 2019: Hu et al. 2020). Multi-exit architectures save computation by making input-specife decisions about bypassing the remaining layers, once the model becomes confident, and are orthogonal to techniques that achieve savings by permanently modifying the
Authors contrihuted equally.

---

Published as a conference paper at ICLR 2021
model (Li et al. 2016 Banner et al. 2018 Han al. 2015 Taylor al. 2018 Figure iillustrates
how a multi- exit model (Kaya et al. 2 based on a standard VGG- 16 architecture, correctly classifies selection of test images from "Tiny ImageNet' before the final layer. We see that more typical samples, which have more supporting examples in the training set, require less depth and, therefore, less computation.
is unknown if the computational savings provided by multi-exit architectures are robust against adversarial pres-
1st
5th
14th
Lion Lemon
sure. Prior research showed that DNNs are vulnerable to
wide range of attacks. which involve imperceptible imput
perturbations (Szegedy et al.] [2014] Goodfellow et al., [2015] Papernot et al., 2016 Hu et al. 2020). Considering that a multi-exit model, on the worst-case input, does not provide any computational savings, we ask: Can the savings from multi-exit models be maliciously negated by input perturba- tions? As some natural inputs do require the full depth of the model, it may be possible to craft adversarial examples that delay the correct decision; it is unclear, however, how many inputs can be delayed with imperceptible perturba- tions. Furthermore, it is unknown if universal versions of these adversarial examples exist, if the examples transfer across multi-exit architectures and datasets. or if existing defenses (e.g. adversarial training) are effective against slowdown attacks.
Figure I: Simple to complex inputs. Some Tiny ImageNet images VGG- 16 model can correctly classify if computa- tion stops at the 1 5th and 14th layers.
Threat Model. We consider a new threat against DNNs, analogous to the denial-of-service (DoS) attacks that have been plaguing the Internet for decades. By imperceptibly perturbing the input to trigger this worst-case, the adversary aims to slow down the inferences and increase the cost of using the DNN. This is an important threat for many practical applications, which impose strict limits the responsiveness and resource usage of DNN models (e.g. in the Internet-of-Thing 2018),, because the adversary could push the victim outside these limits. For example, against commercial image classification system, such as Clarifai.com a slowdown attack might waste valuable computational resources. Against model partitioning scheme, such as Big-Little (De Coninck et al. 2015), it might introduce network latency by forcing excessive transmissions between local and remote models. A slowdown attack aims force the victim to do more work than the adversary. e.g. by amplifying the latency needed to process the sample or by crafting reusable perturbations. The adversary may have to achieve this with incomplete information about the multi-exit architecture targeted, the training data used by the victim or the classification task (see discussion in Appendix A
(Taylor et al.,
Our Contributions. To our best knowledge, we conduct the first study of the robustness of multi-exit architectures against adversarial slowdowns. To this end, we find that examples crafted by prior evasion attacks (Madry et al. 2017 Hu et al. (020) fail to bypass the victim model's early exits, and we show that an adversary can adapt such attacks to the goal of model slowdown by modifying its objective function. We call the resulting attack DeepSloth. We also propose an efficacy metric for comparing slowdowns across different multi-exit architectures. We experiment with three generic multi-exit DNNs (based on VGG16, ResNet56 and MobileNet) (Kaya et al.) 2019) and a specially- designed multi-exit architecture, MSDNets (Huang al. 2018), on two popular image classification benchmarks (CIFAR-10 and Tiny ImageNet). We find that DeepSloth reduces the efficacy of multi- exit DNNs by 90-100%, i.e., the perturbations render nearly all early exits ineffective. In a scenario typical for IoT deployments, where the model is partitioned between edge devices and the cloud, attack amplifies the latency by 1.5-5x, negating the benefits of model partitioning. We also show that it is possible to craft universal DeepSloth perturbation, which can slow down the model on either all or a class of inputs. While more constrained, this attack still reduces the efficacy by 5-45%. Further, we observe that DeepSloth can be effective in some black-box scenarios, where the attacker has limited knowledge about the victim. Finally, we show that a standard defense against adversarial pamplee--dveesariii training- inadequate against slowdowns. Our results suggest that further research will be required for protecting multi-exit architectures against this emerging security threat.
2

---

Published as a conference paper at ICLR 2021
2 RELATED WORK
Adversarial Examples and Defenses. Prior work on adversarial examples has shown that DNNs are vulnerable to test-time input perturbations (Szegedy et al. 2014 Goodfellow et al. 2015 Papernot et al. 2017] Carlini & Wagner Madry et al., 2018). An adversary who wants to maximize a model's error on specific test-time samples can introduce human-imperceepiible perturbations to these samples. Moreover, an adversary can also exploit surrogate model for launching the attack and still hurt an unknown victim (Athalye et al. [0188] Tramèr et al.
[011bb Inkawhich et al., (211)). This transferability leads to adversarial examples more practical black-box scenarios. Although
many defenses (Kurakin et al. 2016: Xu et al. 1017,Song et al.] 2018 Liao et al. 2018 Lecuyer
al. 2019) have been proposed against this threat, adversarial training (AT) has become the
frontrunner (addy et al. 2018). In Sec [ we evaluate the vulnerability of multi-exit DNNs to adversarial slowdowns in white-box and black-box scenarios. Sec we show that standard AT and its simple adaptation to our perturbations are not sufficient for preventing slowdown attacks.
Efficient Input-Adaptive Inference. Recent input-adaptive DNN architectures have brought two seemingly distant goals closer: achieving both high predictive quality and computational efficiency, There are two types of input-adaptive DNNs: adaptive neural networks (AdNNs) and multi-exit architectures. During the inference, AdNNs (Wang et al. 2018 Figurnov et al. 2017) dynamically skip certain part of the model to reduce the number of computations. This mechanism can be used only for ResNet-based architectures as they facilitate skipping within a network. On the other hand, multi-exit architectures (Teerapittaynoon et al. 2016 Huang et al., 20118) Kaya et al. (019)) introduce multiple side branches- or early-exits- to a model. During the inference on an input sample, these models can preemptively stop the computation altogether once the stopping criteria are met one of the branches. Kaya et al. (2019) have also identified that standard, non-adaptive DNNs are susceptible to overthinking, i.e., their inability to stop computation leads to inefficient inferences on many inputs.
Haque et al. (2020) presented attacks specifically designed for reducing the energy -mergyefffciency of AdNNs by using adversarial input perturbations. However, work studies new threat model that an adversary causes slowdowns on multi-exit architectures. By imperceptibly perturbing the inputs, our attacker can (i) introduce network latency to infrastructure that utilizes multi-exit architectures and (ii) waste the victim's 's computational resources. To quantify this vulnerability, we define a new metric to measure the impact of adversarial input perturbation on different multi-exit architectures (Sec 3 In Sec we also study practical attack scenarios and the transferability of adversarial input perturbations crafted by our attacker. Moreover, we discuss the potential defense mechanisms against this vulnerability, by proposing simple adaptation of adversarial training (Sec (6. To the best of our knowledge, our work is the first systematic study of this new vulnerability.
Model Partitioning. Model partitioning has been proposed bring DNNS resource-constrained devices (De Coninck et al. 2015) Taylor et al. 2018).. These schemes split a multi-exit model into sequential components and deploy them in separate endpoints, e.g., small, local on-device part and a large, cloud-based part. For bringing DNNs to the Internet of Things (IoT), partitioning is instrumental as it reduces the transmissions between endpoints, major bottleneck. In Sec 5.1, on a partitioning scenario, we show that our attack can force excessive transmissions.
EXPERIMENTAL SETUP
Datasets. We use two datasets:
CIFAR-10 (Krizhevsky et al., 2009) and Tiny-ImageNNt (Tiny). For testing the cross-domain transferability of our attacks, we use the CIFAR- 100 dataset.
Architectures and Hyper-parameters. To demonstrate that the vulnerability to adversarial slow- downs is common among multi-exit architectures, we experiment two recent techniques: Shallow- Deep Networks (SDNs) (Kaya et al. 2219) and MSDNets (Huang et al., 2018). These architectures were designed for different purposes: SDNs are generic and can convert any DNN into a multi-exit model, and MSDNets are custom designed for efficiency. We evaluate an MSDNet architecture (6 exits) and three SDN architectures, based on VGG-16 (Simonyan & Zisserman (014)) (14 exits), ResNet-56 (He et al. 2016) (27 exits), and MobileNet (Howard et al. 2017) (14 exits).
Metrics. We define the early-exit capability (EEC) curve of a multi-exit model to indicate the fraction of the test samples that exit early specific fraction of the model's full inference cost.
3

---

Published as a conference paper at ICLR 2021
Figure 2 shows the EEC curves of our SDNs Tiny ImageNet, assuming that the computation stops when there is a correct classification at an exit point. For example, VGG- 16 1e-11---bsse based SDN model can conrectly classify ~50% of the samples using ~50% of its full cost. Note that this stopping criterion is impractical; in Sec 4 we will discuss the practical ones.
We define the early-exit efficacy, efficacy in short, to quantify a model's 's ability of utilizing its exit points. The efficacy of multi-exit model is the area under its EEC curve, estimated via the trapezoidal rule. An ideal efficacy for a model is close to 1, when most of the input samples the computation stops very early; models that do not use their early exits have 0 efficacy. A model with low efficacy generally exhibits higher latency; in partitioned model, the low efficacy will cause more input transmissions to the cloud, and the la- tency is further amplified by the network round trips. A multi-exit model's efficacy and accuracy dictated by its stopping criteria, which we dis- cuss in the next section. As for the classification performance, we report the Top-1 accuracy on the test data.
innymmegenee--gp111/// 59 resnet56(ACC 53.5) Mobilenet(ACC 59.5;
Frac. Instances That Exitt (Cumulativv)
"99]6bn(ACC::7. EFCY 54)
resnet56/Acc:75.u EFCY:0.46)
mobilenetIACC::6.55 EFCY:0. 56)
0.6
0.4
0.2
0.0
0.0
0.4
0.8
Frac. Comp. Cost Over the Full Network
The EEC curves. Each curve shows the fraction of test samples model classifies using a certain fraction of its full inference cost. 'EFCY' is short for the model's s efficacy.
Figure 2:
4 ATTACKING THE MULTI-EXIT ARCHITECTURES
Setting. We consider the supervised classification setting with standard feedforward DNN archi- tectures. A DNN model consists of N blocks, or layers, that process the input sample, E gd from beginning to end and produce a classification. A classification, F(x, 0) RT" is the predicted probability distribution of x belonging to each label y E M = {1,..., m.. Here, 0 denotes the tunable parameters, or the weights, of the model. The parameters are learned on a training set D that contains multiple (2,, 91) pairs; where y; is the ground-truth label of the training sample 2,. We use 0 to denote the parameters at and before the jth block; i.e., 0g C 0z+1 and 0x = 0. Once a model is trained, its performance is then tested on set of unseen samples, S.
Multi- Exit Architectures. A multi-exit model contains K exit points internal classifiers attaheh attached to model's hidden blocks. We use F; to denote the jth exit point, which is attached the jth block. Using the output of the jth (j < N) block on v, F; produces an internal classification, i.e., Fx, 0,), which we simply denote as (z). In our experiments, we set K = N for SDNs, i.e., one internal classifier at cach block and K 6 for MSDNets. Given F()2,, multi-exit model uses deterministic criteria to decide between forwarding æ to compute F (x) and stopping for taking the early-exit at this block. Bypassing early-exits decreases a network's efficacy each additional block increases the inference cost. Note that multi-exit models process each sample individually, not in batches.
Practical Stopping Criteria. Ideally, a multi-exit model stops when it reaches a correct classification
at an exit point, i.e., argmax JEM Fx)(2)) 9: y; is the ground-truth label. However, for unseen samples, this impractical as y is unknown. The prior work has proposed two simple strategies to judge whether d y: F,(x)'s entropy (Teerapitayanoo et al. 2016 Huang et al. 2018 or its confidence (Kaya et al. 2019). Our attack (see Sec 4.3) leverages the fact that a uniform (z has both the highest entropy and the lowest confidence. For generality, we experiment with both confidence-barn waweneee-aaaed-aa SDNs and entropy-based -MSDn- strategiies
A strategy selects confidence, entropy, thresholds, T:, that determine whether the model should take the ht exit for an input sample. Conservative T)'s lead to fewer early exits and the opposite hurts the accuracy as the estimate of whether 인; = y becomes unreliable. As utility is a major practical concern, we set T,'s for balancing between efficiency and accuracy. On a holdout set, we set the thresholds to maximize model's 's efficacy while keeping its relative accuracy drop (RAD) over its maximum accuracy within 5% and 15%. We refer to these two settings as RAD<5% and RAD<1159 Table 2 (first segment) shows how accuracy and efficacy change in each setting.
4

---

Published as a conference paper at ICLR 2021
4.1 THREAT MODEL
We consider an adversary who aims decrease the early-exit efficacy victim model. The attacker crafts an imperceptible adversarial perturbation, v E 12d that, when added to a test-time sample S, prevents the model from taking early-exits,
Adversary's 's Capabilities. The attacker is able to modify the victim's test-time samples to apply the perturbations, e.g., by compromising a camera that collects the data for inference. To ensure the imperceptibility, we focus &o norm bounded perturbations as they (i) are well are studied; (ii) have successful defenses (Madry et al. (2188;; (iii) have prior extension to multi-exit models (Hu et al. 2020); and (iv) are usually the most efficient craft. We show results on &g and & perturbations in Appendix C In line with the prior work, we bound the perturbations as follows: for CIFAR- 10, | oo
0.03 (Madry et al. 2017). ||el|1 S (Tramèr & Boneh 2019) and ||0||| S 0.35 (Chen et al. 2017 ; for Tiny ImageNet, |00| S c 0.03 Yang et al. 2019), ,, | S 16 and ||l||2 S 0.6.
Adversary's 's Knowledge. To assess the security vulnerability of multi-exit architectures, we study white-box scenarios, i.e., the attacker knows all the details of the victim model, including its D and 0. Further, Sec [.2 we study more practical black-box scenarios, i.e., the attacker crafts v on a surrogate model and applies it to unknown vietim model.
Adversary's Goals. We consider three DeepSloth variants, (i) the standard, (ii) the universal and (iii) the class-universal. The adversary, in (i) crafts different for each x E S; in (ii) crafts a single for all x E S; in (iii) crafts single for a target class i E M. Further, although the adversary does not explicitly target it; we observe that DeepSloth usually hurts the accuracy. By modifying the objective function describe in Sec 4.3 we also experiment with DeepSloth variants that can explicitly preserve or hurt the accuracy, in addition to causing slowdowns.
4.2 STANDARD ADVERSARIAL ATTACKS DO NOT CAUSE DELAYS
To motivate DeepSloth, we first evaluate whether previous adversarial attacks have any effect on the efficacy of multi-exit models. These attacks add imperceptible perturbations to victim's test-time samples to force misclassifications. We experiment with the standard PGD attack (Madry et al. 2017): PGD-avg and PGD-mi variants against multi-exit models (Hu al.] 2020) and the Universal Adversarial Perturbation (UAP) attack that crafts single perturbation for all test samples (Moosavi Dezfooli et al. 2017). Table summarizes
findings that these attacks, although they hurt the accuracy, fail to cause any meaningful decrease in efficacy. In many cases, we observe that the attacks actually increase the efficacy. These experiments help us to identify the critical elements of the objective function of an attack that decreases the efficacy.
Table 1:
Impact of existing evasion attacks on efficacy. Each entry shows a model's efficacy (left) and accuracy (right) when subjected to the respective attack. The multi-exit models are trained on CIFAR- 10 and use RAD=5% as their carly-exit strategy.
NETWORK
No ATTACK
PGD-20
PGD-20 (AvG.)
PGD-20 (MAx.)
UAP
VGG-16
0.77 89%
0.79
29%
0.85
10%
0.81 / 27%
0.71 68%
RESNET-56
0.52 / 87%
0.55 12%
0.82 / 1 1%
0.70 / 6%
0.55 44%
MOBILENET
0.83 87%
0.85 14%
0.93
3%
0.89 / 12%
0.77 60%
4.3 THE DEEPSLOTH ATTACK
The Layer- Wise Objective Function. Figure 3 shows that the attacks that only optimize for the final output, e.g., PGD or UAP, do not perturb the model's earlier layer representations. This does not bypass the early-exits, which makes these attacks ineffective for decreasing the efficacy. Therefore, we modify the objective functions adversarial example-cratii outputs of all Fili < K. For crafting l. & and f bounded perturbations, adapt the PGD (Madry (et al., 2017). the DDN (Rony et al.] (2019) and the SLIDE algorithms (Tramer Bonch 2019). respectively. Next, we describe how we modify the PGD algorithm- we modify the others similarly:
algorithms to incorporate the
5

---

Published as a conference paper at ICLR 2021
(0' + sgn
ot+ II Bolloo <e
V. Σ Σ ((F( DE OCiCK
Here, t is the current attack iteration; is the step size; II is the projection operator that enforces ||l| < and L is the cross-entropy loss function. The selection of D' determines the type of the attack. For the standard variant: D' {xr J. i.e., single test-time sample. For the universal variant: D' = =D, iee. the whole training set. For the class-universal variant against the target class i E M: D' {(x, y) E Dly i), i.e., the training set samples from the 7t class. Finally, y is the target label distribution our objective pushes (z) towards. Next, we explain how we select y.
Pushing (z) Towards a Uniform Distribution. Despite including all F,, attacks such as PGD- avg and PGD-max (Hu al., 2020) still fail to decrease efficacy. How these attacks select reflects their goal of causing misclassifications and, therefore, they trigger errors in early-exits, ie., argmaXje F2(00 (x) y # y. However, as the carly-exits still have high confidence, or low entropy, the model still stops its computation early. We select ÿ as a uniform distribution over the class labels, i... g(t) = 1/m. This ensures that (x + u) bypasses common stopping criteria as uniform F; (x) has both the lowest confidence and the highest entropy.
5 EMPIRICAL EVALUATION
Here, we present the results for foo DeepSloth against two SDNs-VGG-16 and MobileNet-based and against the MSDNets. In the Appendix. we report the hyperparameters: the l and & attacks; the results on ResNet-56-basee we observe that l -bunded perturbations are more effective for slowdowns. The optimization challenges might explain this, & and &g attacks are usually harder to optimize (Carlini & Wagner 2017: Tramer & Boneh] (211)). Unlike objectives for misclassifications, the objective for slowdowns involves multiple loss terms and optimizes over all the output logits.
SDNs; the cost of the attacks; and some perturbed samples. Overall,
5.1 WHITE-BOX SCENARIOS
Perturbations Eliminate Early-Exits. Table 2 (second segment) shows that the victim models have- ~ 0 efficacy on the samples perturbed by DeepSloth. Across the board, the attack makes the early-exit completely ineffective and force the victim models to forward all input samples till the end. Further, DeepSloth also drops the victim's 's accuracy by 75-99%, comparable to the PGD attack. These results give an answer our main research question: the multi-exit mechanisms are vulnerable and their benefits can be maliciously offset by adversarial input perturbations. In particular, as SDN modification mitigates overthinking standard, non-adaptive DNNs (Kaya et al. 2019). DeepSloth also leads SDN-based models to overthink almost all samples by forcing extra computations.
Note that crafting a single perturbation requires multiple back back-propagations through the model and more floating points operations (FLOPs) than the forward pass. The high cost of crafting, relative to the computational damage to the victim, might make this vulnerability unattractive for the adversary. In the next sections, we highlight scenarios where this vulnerability might lead to practical exploitation. First, we show that in an IoT-like scenarios, the input transmission is a major bottleneck and DeepSloth can exploit it. Second, we evaluate universal DeepSloth attacks that enable the adversary to craft the perturbation only once and reuse it on multiple inputs.
Attacking an IoT Scenario. Many IoT scenarios, e.g., health monitoring for elderly (Park et al., 2017), require collecting data from edge devices and making low-latency inferences on this data. However, complex deep learning models are impractical for low-power edge devices, such as an Arduino, that are common in the IoT scenarios (Chen & Ran 2019). For example, standard hardware, an average inference takes MSDNet model on Tiny ImageNet 35M FLOPs and ~ 10ms.
A potential solution is sending the inputs from the edge to a cloud model, which then returns the prediction. Even in our optimistic estimate with a nearby AWS EC2 instance, this back-and-fon fortforth introduces ~ 1mmm latency per inference. Model partitioning alleviates this bottleneck by splitting multi-exit model into two; deploying the small first part at the edge and the large second part at the cloud (De Coninck al. 2015). The edge part sends input only when its prediction does not meet

---

Published as a conference paper at ICLR 2021
Table 2: The effectiveness loe DeepSloth. 'RAD<5,15% columns the results each early-exit setting. Each entry includes the model's efficacy (left) and accuracy (right). The Mtass-universa attack's 's results are an average of 10 classes. 'TI': :iny ImageNet and 'C10": CIFAR- 10.
NETWORK
MSDNET
VGG16
MOBILENET
SET.
RAD<5%
RAD<15%
RAD<5%
RAD<15%
RAD=5%
RAD< %
BASELINE (No ATTACK)
C10 TI
85%
0.89 / 85%
0.77 88% 0.89 / 79% 0.51 / 52%
0.83 87%
0.92 / 79%
0.64 / 55%
0.83 50%
0.39 57%
0.42 57%
0.59 / 51%
DEEPSLOTH
C10 TI
0.06 / 17%
0.06 17%
0.01 13%
0.04 / 16% 0.01 12%
0.06 / 16%
0.06/ 7%
0.06 7%
0.00/ 2%
0.01/ 2%
0.02 6%
0.04 / 6%
UNIVERSAL DEEPSLOTH
C10 TI
0.85 / 65%
0.85/ / 65%
0.62 65%
0.86 / 60%
0.73 / /61%%%%%% 11%
0,90 / 59%
0.58 / 46%
0.81 / 41%
0.31 /7%
0.44 / 44%
0.33 / 47%
0.51 / 43%
CLASS-UNIVERSAL DEEFSLOTH
C10 TI
0.82 / 32%
0.82 / 32%
0.47 35% 0.78 / 33% 0.33 / 27%
0.60 / 30%
0.85 / 27%
0.41 21%
0.71 17%
0.20 28%
0.21 27%
0.38 / 25%
the stopping criteria. For example, the first early-exit of MSDNets sends only 5% and 67% of all test samples, on CIFAR-10 10 and Tiny ImageNet, respectively. This leads to a lower average latency per inference, i.e., from Ims down 0.5ms and 7.4ms, respectively.
The adversary we study uses DeepSloth perturbations to force the edge part to send all the input samples to the cloud. For the victim, we deploy MSDNet models that we split into two parts at their first exit point. Targeting the first part with DeepSloth forces it to send 96% and 99.97% of all test samples to the second part. This increases average inference latency ~ ms and invalidates the benefits of model partitioning. In this scenario, perturbing each sample takes ~2ms on a Tesla V-100 GPU, Le., the time adversary spends is amplified by 1.5-5x the victim's 's latency increase.
Reusable Universal Perturbations. The universal attacks, although limited, are a practical as the adversary can reuse the same perturbation indefinitely to cause minor slowdowns. Table 2 (third segment) shows that they decrease the efficacy by 3-21% % and the accuracy by 15-25% over the baselines. Having a less conservative early-exit strategy, e.g.. RAD<15%, increases the resilience to the attack the cost of accuracy. Further, MSDNets are fairly resilient with only 3-9% efficacy drop; whereas SDNs are more vulnerable with 12-21% % drop. The attack is also slightly more effective on the more complex task, Tiny ImageNet, the early-exits become easier to bypass. Using random noise as a baseline, i.c., ~ (d(-e,, -e,,) E), we find that at most it decreases the efficacy by ~3%.
In the universal attack, we observe phenomenon: it pushes the samples towards a small subset of all classes. For example, ~ 17% of the perturbed samples are classified into the 'bird' class of CIFAR-I0; up from ~00% for the clean samples. Considering certain classes are distant in the feature space, e.g.. 'truck' and 'bird'; we expect the class-universal variant to be more effective. The results in Table 2 (fourth segment) confirm our intuition. We see that this attack decreases the baseline efficacy by 8-50% and the accuracy by 50-65%. We report the average results across multiple classes; however, we observe that certain classes are slightly more vulnerable to this attack.
Feature Visualization of DeepSloth. In Figure to shed light how DeepSloth differs from prior attacks, egg., PGD and PGD-avg, we visualize a model's 's hidden block (layer) features on the original and perturbed test-time samples. We observe that in earlier block (lefi panel), DeepSloth seems to disrupt the original features slightly more than the PGD attacks. Leaving carlier representations intact prevents PGDs from bypassing the early-exits. The behaviors of the attacks diverge in the middle blocks (middle panel). Here, DeepSloth features remain closer to the original features than prior attacks. The significant disruption of prior attacks leads high-confidence misclassification and fails to bypass early-exits. In the later block (right panel), we see that the divergent behavior persists.
Preserving or Hurting the Accuracy with DeepSloth. Here, we aim to answer whether DeepSloth can be applied when the adversary explicitly aims to cause prevent misclassifications, while still
7

---

Published as a conference paper at ICLR 2021
Org. (3)
Adv. (PaO)
Adv. (PC22-a0g)
Adv. (Dhrn)
PGD)
10un5)
Adv. 10uns)
Figure 3: Visualising features against attacks using UMAP. VGG-16's 3rd (leff), 8th (middle), and 14th (right) hidden block features on CIFAR-I0's 'dog' class (Best viewed in color, zoomed in).
causing slowdowns. Our main threat model has no explicit goal regarding misclassifications that hurt the user of the model, i.e., who consumes the output of the model. Whereas, slowdowns additionally hurt the executor or the owner of the model through the computations and latency increased at the cloud providers. In some ML-in-the-clous scenarios, where these two are different actors, the adversary might aim to target only the exccutor or both the executor and the user. To this end, we modify our objective function to push (i(x:) towards a slightly non-uniform distribution, favoring either the ground truth label for preventing misclassifications or a wrong label for causing them. We test this idea our VGG-16-based SDN model on CIFAR-10 in RAD<5% setting. We see that DeepSloth for preserving the accuracy leads to 81 % accuracy with 0.02 efficacy and DeepSloth for hurting the accuracy leads to 4% accuracy with 0.01 efficacy- the original DeepSloth led 13% accuracy with 0.01 efficacy. These results show the flexibility of DeepSloth and how it could be modified depending on the attacker's goals.
5.2 TOWARDS A BLACK-BOX ATTACK: TRANSFERABILIT OF DEEPSLOTH
Transferability of adversarial examples imply that they can still hurt model that they were not crafted on (Tramèr et al. 2017a] Liu et al. 2017). Even though white-box attacks are important to expose the vulnerability, black-box attacks, by requiring fewer assumptions,
more practical. Here, on four distinct scenarios, we investigate whether DeepSloth is transferable. Based on the scenario's constraints, we (i) train a surrogate model; (ii) craft the DeepSloth samples on it; and (iii) use these samples on the victim. We run these experiments on CIFAR-10 10 in the RAD<5%.
Cross-Architee rcciteccure, First, we relax the assumption that the attacker knows the victim architecture. We evaluate the transfcrability between a VGG-16-based SDN the same D. We find that the samples crafted on the MSDNet can slowdown the SDN: reducing its efficacy to 0.63 (from 0.77) and accuracy to 78% (from 88%). Interestingly, the opposite seems not to be the case: on the samples crafted against the SDN, the MSDNet still has 0.87 efficacy (from 0.89) and 73% accuracy (from 85%). This hints that DeepSloth transfers if the adversary uses an effective multi-exit models as the surrogate.
MSDNet-all all trained using
Limited Training Set Knowledge. Second, we relax the assumption that the attacker knows the victim's 's training set, D. Here, the attacker only knows random portion of D, i.c., 10%, 25%, and 50%. We use VGG- 16 architecture for both the surrogate and victim models. In the 10%, 25% and 50% settings, respectively, the attacks reduce the victim's 's efficacy to 0.66, 0.5, 0.45 and 0.43 (from 0.77); its accuracy to 81%, 73%, 72% and 74% (from 88%). Overall, the more limited the adversary's D is, the less generalization ability the surrogate has and the less transferable the attacks are.
Third, we relax the assumption that the attacker exactly knows the victim's 's task. Here, the attacker uses Dj y to train the surrogate, different from the victim's 's D altogether. We use a (GG-16-bbaee victim model on 110.R-10. This transfer attack reduces the victim's is efficacy to 0.63 (from 0.77) and its accuracy to 83% (from 88%). We see that the cross-domain attack might be more effective than the limited D scenarios. This makes DeepSloth particularly dangerous as the attacker, without knowing the victim's 's D, can collect similar dataset and still slowdown the victim. We hypothesize the transferability of earlier layer features in CNNs Yosinski et al., 2014) enables the perturbations attack to transfer from one domain to another, long they are similar enough.
VGG-16 on CIFAR-100 as the surrogate and attack
8

---

Published as a conference paper at ICLR 2021
Crus-Mecchnaissm. Finally, we test the scenario where the victim uses completely different mechanism than a multi-exit architecture to implement input adaptiveness, í.e., SkipNet (Wang et al. 2018). A SkipNet. modified residual network, selectively skips convolutional blocks based on the activations of the previous layer and, therefore, does not include any internal classifiers. We use a pre-trained SkipNet on CIFAR-10 10 that reduces the average computation for each input sample by ~50% over an equivalent ResNet and achieves ~94% accuracy. We then feed DeepSloth samples crafted on MSDNet this SkipNet, which reduces its average computational saving to ~32% (36% less effective) and its accuracy to 37%. This result suggests that the two different mechanisms have more in common than previously known and might share the vulnerabilityy. We believe that understanding the underlying mechanisms through which adaptive models save computation is an important research question for future work.
6 STANDARD ADVERSARIAL TRAINING IS NOT A COUNTERMEASURE
In this section, examine whether defender can adapt standard countermeasure against adversar- ial perturbations, adversarial training (AT) (Madry et al. 2018), to mitigate our attack. AT decreases a model's 's sensitivity perturbations that significantly change the model's outputs. While this scheme is effective against adversarial examples that aim to trigger misclassifications; it is unclear whether using our DeepSloth samples for AT can also robustify a multi-exit model against slowdown attacks.
To evaluate, we train our multi-exit models as follows. We first take a base network-VGG-16-20 train it CIFAR-10 10 PGD--0 adversarial examples. We then convert the resulting model into a multi-exit architecture, using the modification from (Kaya et al., 2019). During this conversion, we adversarially train individual exit points using PGD-10, PGD-IO (avg.), PGD-10 (max.), and DeepSloth; similar to (Hu et al. 2020). Finally, we measure the efficacy and accuracy of the trained models against PGD-20, PGD- 20 (avg.), PGD-20 (max.), and DeepSloth, on CIFAR-10 1 test-set.
Table 3: Evaluating adversarial training against slowdown attacks. Each entry includes the model's efficacy score (left) and accuracy (right). Results are on CIFAR-IO, in the RAD+5% setting.
ADV. TRAINING
No ATTACK
PGD-20
PGD-20 (AvG.)
PGD-20 (MAx.)
DEEPSLOTH
UNDEFENDED
0.77 89%
0.79 / 29%
0.85 10%
0.81 / 27%
0.01 13%
PGD-10
0.61 72%
0.55 / 38%
0.64 / 23%
0.58 / 29%
0.33 / 70%
PGD-10 (Avc.)
0.53 / 72%
0.47 / 36%
0.47 / 35%
0.47 / 35%
0.32 / 70%
PGD-10 (MAx.)
0.57 72%
0.51 / 37%
0.54 / 30%
0.52 34%
0.32 / 70%
OURS
0.74 72%
0.71 / 38% 0.82 14%
0.77 / 21 0.58 / 28%
0.44 67%%
OURS + PGD-10
0.61 / 73%
0.55 / 38%
0.63 23%
0.33 70%
Our results in Table verify that AT provides resilience against all PGD attacks. Besides, AT provides some resilience to our attack: DeepSloth reduces the efficacy to ~0.32 on robust models vs. 0.01 on the undefended one. However, we identify trade-off between the robustness and efficiency of multi-exits. Compared to the undefended model, on clean samples, we see that robust models have lower efficacy 0.77 vs. 0.53~0.61. We observe that the model trained only with DeepSloth samples (Ours) can recover the efficacy on both the clean and our DeepSloth samples, but this model loses its robustness against PGD attacks. Moreover, when we train model on both our DeepSloth samples and PGD-10 (Ours + PGD-10), 10), the trained model suffers from low efficacy. Our results imply that a defender may require an tht defense, such as flagging the users whose queries bypass the early-exits more often than clean samples for which the multi-exit network was calibrated.
CONCLUSIONS
This work exposes the vulnerability of input-adaptive inference mechanisms against adversarial slowdowns. As a vehicle for exploring this vulnerability systematically, we propose DeepSloth, an attack that introduces imperceptible adversarial perturbations to test-time inputs for offsetting the computational benefits of multi-exit inference mechanisms. We show that white-box attack, which perturbs each sample individually, eliminates any computational savings these mechanisms provide. We also show that it is possible to craft universal slowdown perturbations, which can be

---

Published as a conference paper at ICLR 2021
reused, and transferable samples, in black-box setting. Moreover, adversarial training, standard countermeasure for adversarial perturbations, is not effective against DeepSloth. Our analysis suggests that slowdown attacks realistic, yet under-appreciated, threat against adaptive models.
ACKNOWLEDGMENT
We thank the anonymous reviewers for their feedback. This research was partially supported by the Department of Defense.
REFERENCES
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses adversarial examples. Jennifer Dy and Andreas Krause (eds.). Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 274-283, Stockholmsmässan, Stockholm Sweden, 10-15 Jul 2018. PMLR. URL http: //proseedings.oo mlr press/ v80 1a//0//taalyeeeoe html
Ron Banner, Itay Hubara, Flad Hoffer, and Daniel Soudry. Scalahle methods for 8-hit training of neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, Neural Information Processing Systems
N. Cesa-Bianchi, and Garnett (eds.). Advances
31, pp. 5145-5153. Curran Associates, Inc.. 2018. URL http:: http:::/pppprr..iip.. httpppprrr.ips.ccppppper./ 7761 1161-scalable lable methods- 8-bit teaitttrriiinng of netuneural nww.wkkettoosssoooooooooooooeeee pdd
N. Carlini and D. Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 39-57, 2017.
Jiasi Chen and Xukan Ran. Deep learning with edge computing: A review. Proceedings of the IEEE, 107(8): 1655-1674, 2019.
Pin- Yu Chen, Yash Sharma, Huan Zhang, Jinfeng Yi, and Cho-Jui Hsich. Ead: elastic-ne net attacks to deep neural networks via adversarial examples. arXiv preprint 1iXiv:1799.04 2017.
Elias De Coninck, Tim Verbelen, Bert Vankeirsbilck,, Steven Bohez, Pieter Simoens, Piet Demeester, and Bart Dhoedt. Distributed neural networks internet things: The big-little approach. International Internet of Things Summit, pp. 484-492. Springer, 2015.
Michael Figurnov, Maxwell D. Collins, Yukun Zhu, Li Zhang, Jonathan Huang, Dmitry Vetrov, and Ruslan Salakhutdinov. Spatially Adaptive Computation Time for Residual Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 2017.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and hamessing adversarial examples. In International Conference Learning Representations, 2015. URL
1412
6572
Song Han, Huizi Mao, William Dally. Deep compression: Compressing deep neural networks with pruning. trained quantization and huffmma coding. arXiv preprint arXiv:1510.00149, 2015.
Mirazul Haque, Anki Chauhan. Cong Liu, and Wei Yang. Ilfo: Adversarial attack on adaptive neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
K. He, X. Zhang, S. Ren, and J. Sun. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
Geoffrey Hinton, Li Deng, Dong Yu, George Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara Sainath, al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine, 29(6): 82-97, 2012.
Lu Hou, Zhiqi Huang, Lifeng Shang, Xin Jiang, Xiao Chen, and Qun Liu. DynaBERT: Dynamic BERT with Adaptive Width and Depth. In Advances in Neural Information Processing Systems, 2020.
Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang. Tobias Weyand, Marco Andreetto, and Hartwig Adam. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications, 2017.
10

---

Published as a conference paper at ICLR 2021
C. Hu, W. Bao, D. Wang. and F. Liu. Dynamic adaptive dan surgery for inference acceleration on the edge. In IEEE INFOCOM 2019- IEEE Conference on Computer Communications, pp. 1423-1431, 2019, doi: 10.1 (U9/INFOC...2211..77777777
Ting-Kur Kuei Hu, Tianlong Chen, Haotao Wang. and Zhangyang Wang. Triple wins: Boosting accuracy, robustness and efficiency together by enabling input-adaptive inference. In International Conference on Learning Representations, 2020.
net Etrreemiuudduggeaaer.r
Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens van der Maaten, and Kilian Weinberger. Multi- scale dense networks for resource efficient image classification. In International Conference on Learning Representations, 2018. URLitttps://ppeteeiiie wwt.::tt////ote 2
Nathan Inkawhich, Wei Wen, Hai (Helen) Li, and Yiran Chen. Feature space perturbations yield more transferable adversarial examples. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Weiwen Jiang, Edwin H.-M. Sha, Xinyi Zhang, Lei Yang. Qingfeng Zhuge, Yiyu Shi, and Jingtong Hu. Achieving super-linear speedup across multi fpga for real-time dnn inference. ACM Trans. Embed, Compul. Syst. 18(5s), October 2019. ISSN 1539-9087. doi: 10. 14/33358192. URI. wttpthttt///ooo...
3. T 14/3356 99]
Yiping Kang, Johann Hauswald, Cao Gao, Austin Rovinski, Trevor Mudge, Jason Mars, and Lingjia Tang. Neurosurgeon: Collaborative intelligence between the cloud and mobile edge, In Proceedings of the Twenty- Second International Conference
Architectural Support for Programming Languages and Operating Systems, ASPLOS 17, pp. 615 629, New York, NY, USA. 2017. Association for Computing Machinery, ISBN 450444444 doi: N0.144//303769730336666 o 3037698
Yigitcan Kaya, Sanghyun Hong. and Tudor Dumitraş. Shallow- Networks: Understanding and mitigating network overthinking, In Proceedings of the 2019 International Conference Machine Learning (ICML), Long Beach, CA, Jun 2019.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning Multiple Layers of Features from Tiny Images. 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp. 1097-1105, 2012.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning scale. arXiv preprint arXiv:1611.01236. 2016.
M. Lecuyer, V. Atlidakis, R. Geambasu, D. Hsu, and S. Jana. Certified robustness adversarial examples with differential privacy. In 20/9 IEEE Symposiun on Security and Privacy (SP), pp. 656-672, 2019.
En Li. Zhi Zhou, and Xu Chen. Edge intelligence: On-demand deep learning model co-inference with device- edge synergy. In Proceedings of the 2018 Workshop on Mobile Edge Communications, MECOMM' 18, pp. 31-36, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 9781 458455999. doi: 0444522295566322556666 URL / /o1. 100 1ee1111433222 32255/222955 9556.65666666 322 62
Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets. CoRR, abs/1608.08 10, 2016. URL http:://eat /ttp:v/aaavvvvvva org/ abs /ww////s/000 /008 08710
Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL htttpss///ppeeeeee pet/forrne ? Sy.-Ssyyys...
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks, arXiv preprint arXiv:1706.0660 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018. URL RtLLhtttpp.:/opeeeviieee http://wwww...t/eoommee//aaaoeee
Seyed-Mohsen Dezsovi-Dezfooli,, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
11

---

Published as a conference paper at ICLR 2021
N. Papernot, McDaniel, S. Jha, M. Fredrikson, Z. B. Celik, and A. Swami. The limitations of deep learning in adversarial settings. In 2016 JEEE European Symposium on Security and Privacy (EuroS P). pp. 372-387, 2016.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha. Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM Asia conference on computer and communications security, pp- 506-519, 2017.
Se Jin Park, Murali Subramaniyam, Scoung Eun Kim, Seunghee Hong, Joo Hycong Lee, Chan Min Jo, and Youngscob Sco, Development of the elderly healthcare monitoring system with iot. In Advances in Human Factors and Ergonomics in Healtheare. pp. 309-315. Springer. 2017.
Jérôme Rony, Luiz G Hafemann, Luiz S Oliveira, Ismail Ben Ayed, Robert Sabourin, and Eric Granger. Decoupling direction and norm efficient gradient-based 12 adversarial attacks and defenses. Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 4322-4330, 2019.
Karen Simonyan and Andrew Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition, 2014.
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. In International Conference on Learning Representations, 2018,
net forom//oommmme 1aw.mmmm1-rrrGGGGGGo
C. Szegedy, Wei Liu, Yangqing Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Ra- binovich. Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 1 1-9, 2015.
Christian Szegedy, Wojeiech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, 2014. wtt.http:://rxxww.or/////
Mingxing Tan and Quoe Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.). Proceedings of the 36th International Conference on Machine Learning. volume 97 of Proceedings of Machine Learning Research, pp. 6105-6114, Long Beach, California, USA, 09-15 Jun 2019. PMLR. URL http::tt:::pppeeeeeeeee wttp::s..........1/.////// pwwpss/s9/taaalll
ml
html
Ben Taylor, Vicent Sanz Marco, Willy Wolff. Yehia Elkhatib, and Zheng Wang. Adaptive deep learning model selection on embedded systems. ACM SIGPLAN Notices, 53(6):31-43, 2018.
S. Teerapittayanon. B. McDanel, and H. T. Kung. Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd International Conference on Pattern Recognition (ICPR), pp.
2016
Tiny. Tiny ImageNet Visual Recognition Challenge. Accessed: 2020-09-28.
htw.aug--_mmagcnteteooouuuaa com/
Florian Tramèr and Dan Boneh. Adversarial training and robustness for multiple perturbations. In Advances in Neural Information Processing Systems, pp. 5866-5876. 2019.
Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Bonch, and Patrick McDaniel. The space of transferable adversarial examples. arXiv preprint arXiv: /74.00333, 2017a.
Florian Tramèr, Nicolas Papernot, lan Goodfellow, Dan Bonch, and Patrick McDaniel. The space of transferable adversarial examples. arXiv preprint 17414:1704.03453, 2017b.
Xin Wang. Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph E. Gonzalez. SkipNet: Learning Dynamic Routing in Convolutional Networks. In The European Conference on Computer Vision (ECCV), September 2018.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks, arXiv preprint 174.011004.01155, 2017.
Yuzhe Yang. Guo Zhang, Dina Katabi, and Zhi Xu. Me-net: Towards effective adversarial robustness with matrix estimation. arXiv preprint arXiv: 1905: 1191, 2019.
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In Advances in neural information processing systems, pp. 3320-3328, 2014.
12

---

Published as a conference paper at ICLR 2021
Li Zhou, Hao Wen, Radu Teodorescu, and David H.C. Du. Distributing deep neural networks with containerized partitions at the edge. 2nd USENIX Workshop on Hot Topics in Edge Computing (HotEdge 19), Renton, WA, July 2019. USENIX Association.
atesent icn/ zhou
Wangchunshu Zhou, Canwen Xu, Ge, Julian McAuley, Ke Xu, and Furu Wei. BERT Loses Patience: Fast and Robust Inference with Early Exit. In Advances in Neural Information Processing Systems, 2020.
13

---

Published as a conference paper at ICLR 2021
A MOTIVATING EXAMPLES
Here, we discuss two exemplary scenarios where an adversary can exploit the slowdown attacks.
(Case 1) Attacks on cloud-based ΣoT applications. In most cases, cloud-based IoT applications, such as Apple Siri, Google Now, or Microsoft Cortana, run their DNN inferences in the cloud. This cloud-only approach puts all the computational burden on cloud servers and increases the communications between the servers and IoT devices. In consequence, recent work (Kang et al.)
2017 Li al. 2018 Zhou al. 2019) utilizes multi-exit architectures for bringing computationally expensive models, e.g. language models (Zhou et al.) [2220] Hou et al.) 2020). in the cloud to IoT (or mobile) devices. They split a multi-exit model into two partitions and deploy each of them to a server and loT devices, respectively. Under this scheme, the cloud server only takes care of complex imputs that the shallow partition cannot correctly classify at the edge. As result, one can reduce the computations in the cloud and decrease communications between the cloud and edge. On the other hand, our adversary, by applying human-imperen imperceptible perturbations, can convert simple inputs into complex inputs. These adversarial inputs will bypass early-exits and, as result, reduce (or even offset) the computational and communication savings provided by prior work. Here, a defender may deploy DoS defenses such as firewalls rate-limiting. In this setting, the attacker may not cause DoS because defenses keep the communications between the server and IoT devices under certain-level. Nevertheless, the attacker still increases: (i) the computations at the edge (by making inputs skip early-exits) and (ii) the number of samples that cloud servers process. Recall that a VGG-16 SDN model classifies 90% of clean CIFAR-IO instances correctly at the first exit. If the adversarial examples crafted by the attacker bypass only the first exit, one can easily increase the computations on IoT devices and make them send requests the cloud.
(Case 2) Attacks on real-time DNN inference for resource- and time-constrained scenarios. Recent work on the real-time systems (Hu et al. 2019, Jiang et al., 2019) harnesses multi-exit architectures and model partitioning and time-constrained scenarios. Hu et al. (201)) showed a real-world prototype of an optimal model partitioning. which is based on self-driving car video dataset, can improve latency and throughput of partitioned models on the cloud and edge by 6.5-14x, respectively.
solution to optimize real-time DNN inference for resource-
However, the prior work does not consider the danger of slowdown attacks; our threat model has not been discussed before in the literature. Our results in Sec suggest that slowdown can be induced adversarially, potentially violating real-time guarantees. For example, our attacker can force partitioned models on the cloud and edge to use maximal computations for inference. Further, the same adversarial examples also require the inference results from the model running on the cloud, which potentially increases the response time of the edge devices by 1.5-5x. Our work showed that multi-exit architectures should be used with caution in real-time systems.
B HYPERPARAMETERS
In our experiments, we use the following hyperparameters to craft adversarial perturbations.
-assd DeepSloth. We find that -ased DeepSloth does not require careful tuning. For the standard attack. we set the total number of iterations to 30 and the step size to a = 0.002. For the modified attacks for hurting or preserving the accuracy, we set the total number of iterations to 75 and the step size to 0.001. We compute the standard perturbations using the entire 10k test-set samples in CIFAR-10 10 and Tiny Imagenet. For the universal variants, we set the total number of iterations to 12 and reduce the initial step size of a 0.005 by factor of 10 every 4 iterations. To compute a universal perturbation, we use randomly chosen 250 (CIFAR-10 10) and 200 (Tiny Imagenet) training samples.
based DeepSloth. For both the standard and universal attacks, we set the total number of iterations to 550 and the step size y to 0.1. Our initial perturbation has the nn-m of 1.0. Here, we use the same number of samples for crafting the standard and universal perturbations as the based attacks.
bbasdd DeepSloth. For our standard based DeepSloth, we set the total number of iterations to 250, the step size 0.5, and the gradient sparsity to 99. For the universal variants, reduce the total number of iterations to 100 and set the gradient sparsity to 90. Other hyperparameters remain the same. We use the same number of samples the based attacks, craft the perturbations.
14

---

Published as a conference paper at ICLR 2021
C EMPIRICAL EVALUATION OF L AND & DEEPSLOTH
Table 4 and Table shows the effectiveness of -based and b--based DeepSloth attacks, respectively,
Table 4: The effectiveness
DeepSloth. 'RAD<3,15% columns list the results in each early-exit setting. Each entry includes the model's efficacy score (leff) and accuracy (right). The class-universal
attack's results are an average of 10 classes. 'TI' is Tiny Imagenet and 'C10' is CIFAR-IO.
NETWORK
MSDNET
VGG16
MOBILENET
SET.
RAD<5%
RAD<15%
RAD<5%
RAD<15%
RAD<5%
RAD<15%
BASELINE (No ATTACK)
C10 TI
0.89/859)
0.89 / 85% 0.83 / 50%
0.77/899 0.39 57%
0.89 79% 0.51 / 52%
0.83 87% 0.42 57%
0.92 / 79% 0.59 / 51%
0.64 55%
DEEPSLOTH
C10 TI
0.36/51%
0.35 5/15% % 0.51 40%
0.12/ 36%
0.34 / 45%
0.18 / % 0.08 33%
0.49 / 53%
0.23 37%
0.08 22%
0.15 25%
0.19 / 35%
UNIVERSAL DEEPSLOTH
C10
0.89 83%
0.89 83%
0.75 85% 0.88 / 75% 0.51 52%
0.82 85%
0.92 / 77%
TT
0.64 / 55%
0.83 50%
0.38 57%
0.41 57%
0.59 / 51
CLASS-UNIVERSS
DEEPSLOTH 0.69 78% 0.86 / 67% % 0.50 58%
C10 TI
0.88 / 73%
0.88 73%
0.76 / 74%
0.89 / 65%%
0.64 / 54%
0.83 / 49%
0.39 59%
0.41 60%
0.58 / 53%
Table 5: The effectiveness of lg DeepSloth. R.D55 columns list the results in each early-exit setting. Each entry includes the model's efficacy score (left) and accuracy (right). The glass-universa. attack's results are average of 10 classes. 'TT' is Tiny Imagenet and 'C10' is CIFAR-10.
NETWORK
MSDNET
VGG16
MOBILENET
SET.
RAD<5%
R5D<15%
RAD<5%
RAD<15%
RAD<5%
RAD<15%
BASELINE (No ATTACK)
C10
0.89 / 85% 0.64 55%
0.89 058
0.77 89% 0.89 / 79% 0.51 52%
87% 0.83 88%%
0.92 / 79%
0.83 50%
0.39 57%
0.42 57%
0.59 51%
DEEPSLOTH
C10 TI
0.52 64%
0.52 64% 0.52 / 44%
0.22 60% 0.45 62% 0.21 / 36%
0.23 /6%
0.48 / 55%
0.24 / 42%
0. 35%
0.12 38%
0.25 / 40%
UNIVERSAL DEEPSLOTH
C10 TI
0189
0.89 / 81 % 0.82 48%
0.75 / 87% 0.88 / 76% 0.81 84%
0.92 / 76%
0.63 / 54%
0.38 / 56% 0.51 / 52%
0.41 56%
0.58 / 51%
UNISS-UUIVERSAL DREPSLOTH
C10
0.88 / 73%
0.88 73%
0.71 81% 0.86 / 70% 0.50 57%
0.76 76%
0.89 / 66% 0.58 / 53%
0.64 53%
0.83 49%
0.38 57%
0.41 58%
Our results show that the é and baseed attacks are less effective than the -ased attacks. In contrast to the -based attacks that eliminate the efficacy of victim multi-exit models, the é1- and ég- based attacks reduce the efficacy of the same models by 0.24~0.65. Besides, the accuracy drops caused by f1- and based attacks are in 6~21%, 26, smaller than that of c.c- based DeepSloth (75~99%). Moreover, we see that the universal variants of é and l basdd attacks can barely reduce the efficacy of multi-exit models decrease the efficacy up 0.08 and the accuracy by 12%.
15

---

Published as a conference paper at ICLR 2021
D EMPIRICAL EVALUATION OF DEEPSLOTH ON RESNET56
Table Gishows the the effectiveness of our DeepSloth attacks on ResNet56-base models.
Table 6: The effectiveness of DeepSloth on the ResNet-based models. 1RAD<5,15% columns list the results in each early-exit setting. Each entry includes the model's efficacy score (left) and accuracy (right). The class-universal attack's 's results are an average of 10 classes.
NETWORK
RESNET (Loo)
RESNET (41)
RESNET (62)
SET.
RAD<5%
RAD<15%
RAD<5%
RAD<15%
||
RAD=5%
RAD<15%
BASELINE (No ATTACK)
C10 TI
0.52 / 87%
0.69 / 80%
752% 0125/51%
0.69 / 80% 0.39 / 46%
0.51 / 87%
0.69 / 80%
0.25 / 51%
0.39 46%
0.25 51%
0.39 / 46%
DEEPSLOTH
C10 TI
0.00 19% 0.00/ 7%
0.01
19%
0.05 43%
0.18 / 47% 0.06 455 %
0.17 / 48%
0.01/ 7%
0.04 27%
0.10 / 28%
0.05 34%
0.13 / 35%
UNIVERSAL DEEPSLOTH
C10 TI
0.35 / 63%
0.59 60%
0.49 84% 0.68 75% 0.39 / 46%
0.48 85%
0.67 / 76%
0.25 25%
0.34 / 37% %
0.25 51%
0.25 51 %
0.38 / 46%
CLASS--NIVERSAA UNIVERSAL DEEPSLOTH
C10
0.23 / 33%
0.48 29%
0.39 / 70%
0.60 / 61 %
0.39 711 % 0.23 50%
0.60 / 61 %
TI
0.11 221%
0.23 18%
0.23 51% 0.36 / 46%
0.36 / 46%
Our results show that ResNet56 based models are vulnerable to all the foo. l2. and l fi-aeeed DeepSloth attacks. Using our -oe-based DeepSloth, the attacker can reduce the efficacy of the victim models to 0.00~~001 and the accuracy by 39~688. Besides, the b,, and fi-based attacks also decrease the efficacy 0.04~-0.18 and the accuracy by 11~44%. Compared the results on MSDNet, VGG16, and MobileNet in Table and the same attacks are more effective. The universal variants decrease the efficacy up to 0.21 and the accuracy up to 24%. particular, the l2. and -based attacks (on CIFAR- 10 models) are effective than the same attacks on MSDNet, VGGI6, and MobileNet models.
E COST OF CRAFTING DEEPSLOTH SAMPLES
In Table we compare the cost of DeepSloth with other attack algorithms on a VGG16-based CIFAR- 10 model-executed GPU. For the universal DeepSloth, we measure the execution time for crafting perturbation using batch (250 samples) of the training set. For the other attacks, we measure the time for perturbing the whole test set of CIFAR-10. Our DeepSloth takes roughly the same time the PGD and PGD-avg attacks and significantly less time than the PGD-max attack. Our
ATTACKS
TIME (SEC.)
a single Nvidía Tesla-V100
PGD-20
38
PGD-20 (AVG.)
48
PGD-20 (MAX.)
475
DEEPSLOTH
44
UNIVERSAL DEEPSLOTH
2
Table 7: Time it takes to craft attacks. universal DeepSloth takes only 2 seconds (10x faster than DeepSloth) only uses 250 samples.
F ADVERSARIAL EXAMPLES FROM STANDARD ATTACKS AND DEEPSLOTH
In Figure 4 visualize the adversarial examples from the PGD, UAP and our DeepSloth attacks.
16

---

Published as a conference paper at ICLR 2021
Standard Attacks
DeepSloth
Original
PGD
PGD (avg.)
PGD (max.)
UAP
Per-sample
Universal
Class-specific
Eir of the Perturbations (on Average)
0.03
0.03
0.03
0.03
0.03
0.03
0.03
Figure 4: Adversarial examples from the standard and our DeepSloth attacks. The leftmost column shows the clean images. In the next four columns, we show adversarial examples from PGD, PGD (avg.), PGD (max.), and UAP attacks, respectively. The last four columns include adversarial examples from the three variants of DeepSloth. Each row corresponds to each sample, and the last row contains the average norm of the perturbations over the eight samples in each attack.
17


Reviews:
[{"review_id": "xrwbVDB-67", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper defines a new problem named slowdown adversarial attack to decrease inference efficiency of early multi-exit networks.\n\nPros:\n1. Slowdown attack on efficient-inference networks in an innovative new problem with wide real-world applications.\n2. The proposed method, basically pushing the model prediction to uniform distribution, is straightforward and intuitively correct.\n3. The universal/class-universal variances of slowdown attack provide more flexibility of the proposed method, and the blackbox experiments show good generalization ability of slowdown attack across models and datasets.\n4. Last but not least, the authors show that traditional adversarial training is not a cure for the newly proposed slowdown attack, showing the challenging character of this new threat model.\n\nComments:\n1. Limitation in application scenario: From the prospective of an attacker, one may want to jointly attack inference efficacy and accuracy. Is it possible to design a more generalized setting, where the adversarial attacker can generate images that both slowdown inference and leads to wrong prediction? The current method pushes model predictions to uniform distribution, so it may be ineffective to generate adversarial images to fool classifiers. This is validated by the results, where the slowdown adversarial images have an 5% or 15% lower accuracy compared with clean images. \n2. Any discussions on how to conduct slowdown attacks on other efficient-inference models (e.g., SkipNet [1])?\n3. Please consider releasing your codes.\n\n[1] SkipNet: Learning Dynamic Routing in Convolutional Networks.", "extra_scores": {}, "extra_text": {}}, {"review_id": "qp5ZCIcIt8F", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper aims to indicate a new direction to conduct an adversarial attack: to delay the inference time/raise the computational costs of a ''multi-exit'' model.\n\nHowever, both the motivation and novelty are lacking, I will argue for the rejection if other reviewers wish to accept it.\n\nFrom the perspective of the industry:\n1. We do not actually use the so-called ''multi-exit'' architectures, because they could be difficult to optimize, quantize, assemble, and so on. For mobile devices, we deploy different ''single-exit'' architectures to meet their computing capacity and balance the performance. And it is even trivial to maintenance these architectures because we have developed automation tools.\n2. On the cloud servers, the deployment is redundant, such that the service will still work stably, even all the queries require the maximal flops. \n3. After all, it is easy to limit one's Queries Per Second (QPS), and nothing need to be worried about.\n\nFrom the perspective of academia:\n1. This paper only employs the PGD attack and the adversarial training on a specific task, the novelty is lacking.\n2. In section 5.2, if one knows the architecture and the training set and the task, how could this configuration being called ''black-box''.\n\nUPDATE:\n\nIn the rebuttal, the authors emphasize two facts:\n1. There exist such IoT scenarios and real-time systems that employ multi-exit architectures (including those that employ cloud computation.).\n2. The slowdown attack is effective in these scenarios.\n\nHowever, to prove this method works in practice, it is not a simple \"1 then 2\", you need to show us \"1 and 2\". That is, you do actually deploy any system described in [1,2,3,4,5,6,7,8], and provide a feasible approach to attack with your method, and report the actual damage caused by your method, and convince the readers the damage is significantly severe compared to the efforts spent for causing the damage. \nOtherwise, it is only an application of PGD with a different loss function.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "_O7wlzXJTDw", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies adversarial attack and defense for adaptive multi-exit network. Adaptive multi-exit network is, by itself, a pretty new and under-studied topic, let alone the adversarial study on top of it. This paper proposes a simple-yet-effective DeepSloth attack based on layerwise loss function. It also proposed an efficacy metric for better evaluation. The experiments are conducted on four multi-exit networks and two datasets: cifar10 and tiny imagenet. In the appendix, there is also evaluation on different norms of attacks. The results on white-box and black-box attack demonstrate the effectiveness: DeepSloth not only hurts the accuracy (also achieved by baselines), but also hurts the efficacy (only achieved by DeepSloth). To reduce the computation burden of performing the attack, the authors did two things: 1) model partitioning in scenario of IOT, and 2) universal attack across a dataset. At the end, the authors adapt AT to multi-exit networks and demonstrate that AT is effective in general and DeepSloth is further helping AT for better robustness.\n\nOverall, I think this paper is pretty complete, containing both attack side and defense side. The proposed method is simple and effective, and the empirical results look good.\n\nMy main concern is about the usefulness. The main gain over baseline attack is that DeepSloth also hurts efficacy. In practical use case, two factors lead to bad results to users: 1) inference timeout; 2) wrong prediction. So if we already got wrong prediction, why we care efficacy (which may cause inference timeout)?\n\nSome other minor points.\n\n1. Section 3: metrics. This is novelty of this paper, so I think should be emphasized somewhat. \n\n2. From Appendix, why DeepSloth is half time versus PGD-(avg)?\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "HKMqyhen98B", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nSummary of the paper\n=================\n\nThis paper studies a new category of adversarial attacks, i.e., attackers that try to slow-down multi-exit DNNs using adversarial examples. The paper extended adversarial attacks to perform the slow-down attack and showed that the attacks could slow-down multi-exit DNNs by 1.5x - 5.0x. Additionally, the paper experimentally answers many questions such as (1) the effectiveness of adversarial training against the attack, (2) input-agnostic attack, and (3) cross-architecture/domain transferability.\n\nStrongness\n=========\n\n- The paper suggests a new threat model.\n- The paper studies the problem from multiple aspects such as new-attacks, the effectiveness of existing and new defenses, and transferability.\n\nOverall, I think the paper suggested a new and interesting problem and also performed sufficient experiments and analyses for the new threat.\n\nWeakness\n========\n\nSome of the motivation of the attack is unclear. I am not convinced that attackers are motivated to use the slow-down attack instead of performing DDoS on cloud applications. And I do not think multi-exit models are used on IoT applications if the target is a real-time system (The reviewer is not a specialist in real-time systems, and please correct it if it is wrong). The reviewer believes that more discussion and motivational examples of the new thread model will increase this paper's impact.\n\nClarity\n=====\n\nThis paper is well organized and clear.\n\nRelation to prior work\n================\n\nTo the best of the reviewer's knowledge, the paper is correctly placed in the literature. However, it is not likely but possible that the reviewer is missing some important prior work.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper discusses a new threat model for multi-exit DNNs: attacks against efficiency of inference. The proposed attack increases the inference time of such networks by the factor of 1.5-5, while at the same reducing the accuracy of attacked networks. Unlike classical adversarial examples, the new type of attack cannot be thwarted by adversarial training.

Overall, the paper exhibits a novel contribution, is well written and methodically sound. Its practical motivation is somewhat weak, as it is currently unclear for which applications such attacks may be feasible. However, the novelty of the threat model addressed by this paper makes it an interesting methodical contribution. 
Decision: Accept (Spotlight)