SlowGuess/NeurIPS_2023_OCR huggingface dataset:
BERT Lost Patience
Won't Be Robust to Adversarial Slowdown
Zachary Coalson, Gabriel Ritter, Rakesh Bobba, and Sanghyun Hong  
Oregon State University  
{coalsonz, ritterg, bobbar, hongsa}@oregonstate.edu
In this paper, we systematically evaluate the robustness of multi-exit language models against adversarial slowdown. To audit their robustness, we design a slowdown attack that generates natural adversarial text bypassing early-exit points. We use the resulting WAFFLE attack as a vehicle to conduct a comprehensive evaluation of three multi-exit mechanisms with the GLUE benchmark against adversarial slowdown. We then show our attack significantly reduces the computational savings provided by the three methods in both white-box and black-box settings. The more complex a mechanism is, the more vulnerable it is to adversarial slowdown. We also perform a linguistic analysis of the perturbed text inputs, identifying common perturbation patterns that our attack generates, and comparing them with standard adversarial text attacks. Moreover, we show that adversarial training is ineffective in defeating our slowdown attack, but input sanitization with a conversational model, e.g., ChatGPT, can remove perturbations effectively. This result suggests that future work is needed for developing efficient yet robust multi-exit models. Our code is available at: https://github.com/ztcoalson/WAFFLE
A key factor behind the recent advances in natural language processing is the scale of language models pre-trained on a large corpus of data. Compared to BERT [5] with 110 million parameters that achieves the GLUE benchmark score [36] of  $81\%$  from three years ago, T5 [29] improves the score to  $90\%$  with  $100 \times$  more parameters. However, pre-trained language models with this scale typically require large memory and high computational costs to run inferences, making them challenging in scenarios where latency and computations are limited.
To address this issue, input-adaptive multi-exit mechanisms [32, 41, 45, 40, 42, 48, 21, 44] have been proposed. By attaching internal classifiers (or early exits) to each intermediate layer of a pre-trained language model, the resulting multi-exit language model utilizes these exits to stop its forward pass preemptively, when the model is confident about its prediction at any exit point. This prevents models from spending excessive computation for "easy" inputs, where shallow models are sufficient for correct predictions, and therefore reduces the post-training workloads while preserving accuracy.
In this work, we study the robustness of multi-exit language models to adversarial slowdown. Recent work [10] showed that, against multi-exit models developed for computer vision tasks, an adversary can craft human-imperceptible input perturbations to offset their computational savings. However, it has not yet been shown that the input-adaptive methods proposed in language domains are susceptible to such input perturbations. It is also unknown why these perturbations cause slowdown and how similar they are to those generated by standard adversarial attacks. Moreover, it is unclear if existing defenses, e.g., adversarial training [43], proposed in the community can mitigate slowdown attacks.
Our contributions. To bridge this gap, we first develop WAFFLE, a slowdown attack that generates natural adversarial text that bypasses early-exits. We illustrate how our attack works in Figure 1.
Based on our finding that existing adversarial text attacks [16, 43] fail to cause significant slowdown, we design a novel objective function that pushes a multi-exit model's predictions at its early-exits toward the uniform distribution. WAFFLE integrates this objective into existing attack algorithms.
Second, we systematically evaluate the robustness of three early-exit mechanisms [41, 45, 21] on the GLUE benchmark against adversarial slowdown. We find that WAFFLE significantly offsets the computational savings provided by the mechanisms when each text input is individually subject to perturbations. We also show that methods offering more aggressive computational savings are more vulnerable to our attack.
Third, we show that WAFFLE can be effective in black-box scenarios. We demonstrate that our attack transfers, i.e., the adversarial texts crafted with limited knowledge about the victim cause slowdown across different models and multi-exit mechanisms. We are also able to find universal slowdown triggers, i.e., input-agnostic sequences of words that reduces the computa
Figure 1: Illustrating adversarial slowdown. Replacing the word "He" with "Her" makes the resulting text input bypass all 11 ICs (internal classifiers) and leads to misclassification. The text is chosen from the Corpus of Linguistic Acceptability.
tional savings of multi-exit language models when attached to any text input from a dataset.
Fourth, we conduct a linguistic analysis of the adversarial texts WAFFLE crafts. We find that the effectiveness of the attack is not due to the amount of perturbations made on the input text, but rather how perturbations are made. Specifically, we find two critical characteristics present in a vast majority of successful samples: (1) subject-predicate disagreement, meaning that a subject and corresponding verb within a sentence do not match, and (2) the changing of named entities. These characteristics are highlighted in [31], where it was shown that BERT takes both into account when making predictions.
Fifth, we test the effectiveness of potential countermeasures against adversarial slowdown. We find that adversarial training [12, 43] is ineffective against WAFFLE. The defended multi-exit models lose efficacy, or they lose significant amounts of accuracy in exchange for aggressive computational savings. In contrast, we show that input sanitization can be an effective countermeasure. This result suggests that future work is needed to develop robust yet effective multi-exit language models.
Adversarial text attacks on language models. Szegedy et al. [34] showed that neural network predictions can be fool-ed by human-imperceptible input perturbations and called such perturbed inputs adversarial examples. While earlier work in this direction studied these adversarial attacks on computer vision models [2, 25], there has been a growing body of work on searching for adversarial examples in language domains as a result of language models gaining more traction. However, adversarial texts are much harder to craft due to the discrete nature of natural language. Attacks on images leverage perturbations derived from computing gradients, as they are composed of pixel values forming a near-continuous space, but applying them to texts where each word is in a discrete space is not straightforward. As a result, diverse mechanisms for crafting natural adversarial texts [7, 30, 20, 16, 8, 9, 19] have been proposed. In this work, we show that an adversary can exploit the language model's sensitivity to input text perturbations to achieve a completely different attack objective, i.e., adversarial slowdown. A standard countermeasure against the adversarial input perturbation is adversarial training that augments the training data with natural adversarial texts [13, 47, 15, 23, 43]. We also show that adversarial training and its adaptation to our slowdown attacks are ineffective in mitigating the vulnerability to adversarial slowdown.
Input-adaptive efficient neural network inference. Neural networks are, while accurate, computationally demanding in their post-training operations. Kaya et al. [17] showed that overthinking is one problem—these models use all their internal layers for making predictions on every single input even for the "easy" samples where a few initial layers would be sufficient. Prior work [35, 17, 11] proposed multi-exit architectures to mitigate the wasteful effect of overthinking. They introduce multiple internal classifiers (i.e., early exits) to a pre-trained model and fine-tune them on the training data to
make correct predictions. During the inference on an input, these early-exits enable input-adaptive inference, i.e., a model stops running forward if the prediction confidence is sufficient at an exit.
Recent work [32, 41, 45, 40, 42, 48, 21, 44] adapted the multi-exit architectures to language models, e.g., BERT [5], to save their computations at inference. DeeBERT [41] and FastBERT [22] have been proposed, both straightforward adaptations of multi-exit architectures to language models. Zhou et al. [45] proposed patience-based early exits (PABEE) and showed that one could achieve efficiency, accuracy, and robustness against natural adversarial texts. Liao et al. [21] presented PastFuture that makes predictions from a global perspective, considering past and future predictions from all the exits. However, no prior work studied the robustness of the computational savings that these mechanisms offer to adversarial slowdown [10]. We design a slowdown attack to audit their robustness. More comparisons of our work to the work done in computer vision domains are in Appendix A.
3 Our Auditing Method: WAFFLE Attack
We consider an adversary who aims to reduce the computational savings provided by a victim multi-exit language model. To achieve this goal, the attacker performs perturbations to a natural test-time text input  $x \in S$ . The resulting adversarial text  $x'$  needs more layers to process for making a prediction. This attack potentially violates the computational guarantees made by real-time systems harnessing multi-exit language models. For example, the attacker can increase the operational costs of the victim model or push the response time of such systems outside the expected range.
Just like language models deployed in the real-world that accept any user input, we assume that the attacker has the ability to query the victim model with perturbed inputs. We focus on the word-level perturbations as they are well studied and efficient to perform with word embeddings [28]. But it is straightforward to extend our attack to character-level or sentence-level attacks by incorporating the slowdown objective we design in Sec 3.2 into their adversarial example-crafting algorithms.
To assess the slowdown vulnerability, we comprehensively consider both white-box and black-box settings. The white-box adversary knows all the details of the victim model, such as the training data and the model parameters, while the black-box attacker has limited knowledge of the victim model.
3.2 The Slowdown Objective
Most adversarial text-crafting algorithms iteratively apply perturbations to a text input until the resulting text  $x'$  achieves a pre-defined goal. The goal here for the standard adversarial attacks is the untargeted misclassification of  $x'$ , i.e.,  $f_{\theta}(x') \neq y$ . Existing adversarial text attacks design an objective (or a score) function that quantifies how the perturbation of a word (e.g., substitution or removal) helps the perturbed sample achieve the goal. At each iteration  $t$ , the attacker considers all feasible perturbations and chooses one that minimizes the objective the most.
We design our score function to quantify how close a perturbed sample  $x'$  is to causing the worst-case slowdown. The worst-case we consider here is that  $x'$  bypasses all the early exits, and the victim model classifies  $x'$  at the final layer. We formulate our score function  $s(x', f_{\theta})$  as follows:
$$
s \left(x ^ {\prime}, f _ {\theta}\right) = \sum_ {0 <   i \leq K} \left(1 - \frac {1}{N - 1} \mathcal {L} \left(F _ {i} \left(x ^ {\prime}\right), \hat {y}\right)\right)
$$
Here, the score function takes  $x'$  as the perturbed text and  $f_{\theta}$  as the victim model. It computes the loss  $\mathcal{L}$  between the softmax output of an  $i$ -th internal classifier  $F_{i}$  and the uniform probability distribution  $\hat{y}$  over classes. We use  $\ell_1$  loss.  $K$  is the number of early-returns, and  $N$  is the number of classes.
The score function  $s$  returns a value in [0, 1]. It becomes one if all  $F_{i}$  is close to the uniform distribution  $\hat{y}$ ; otherwise, it will be zero. Unlike conventional adversarial attacks, our score function over iterations pushes all  $F_{i}$  to  $\hat{y}$  (i.e., the lowest confidence case). Most early-exit mechanisms stop forward pass if  $F_{i}$ 's prediction confidence is higher than a pre-defined threshold  $T$ ; thus,  $x'$  that achieves the lowest confidence bypasses all the exit points.
3.3 The WAFFLE Attack
We finally implement WAFFLE by incorporating the slowdown objective we design into existing adversarial text attacks. In this work, we adapt two existing attacks: TextFooler [16] and A2T [43]. TextFooler is by far the strongest black-box attack [38], and A2T is a gradient-based white-box attack. In particular, A2T can craft natural adversarial texts much faster than black-box attacks; thus, it facilitates adversarial training of language models. We discuss the effectiveness of this in Sec 7.
We describe how we adapt TextFooler for auditing the slowdown risk in Algorithm 1 (see Appendix for our adaptation of A2T). We highlighted our adaptation to the original Textfooler in blue.
Algorithm 1 WAFFLE (based on TextFollower)
Input: a text input  $x = \{w_{1}, w_{2}, \dots, w_{n}\}$ , its label  $y$ , the victim model  $f_{\theta}$ , its early exits  $F_{i}$ , sentence similarity function  $Sim(\cdot)$ , its threshold  $\epsilon$ , word embeddings  $E$  over the vocabulary  $V$ , and the attack success threshold  $\alpha$ .
Output: a natural adversarial text  $x'$
1:  $x^{\prime}\gets x$
2: for each word  $w_{i}$  in  $x$  do
3: Compute the importance  $I_{w_i}$
4: end for
5: Compose a set  $W$  of all words  $w_{i} \in x$  sorted by the descending order of their importance
6: Remove the stop words from the set  $W$
7: for each word  $w_{i}$  in  $W$  do
8: Initiate the set of substitute candidates  $C$  by computing the top  $N$  synonyms; we compute the cosine similarity between  $E_{w_i}$  and  $E_{w'}$ , where  $w' \in V$
9:  $C \gets \mathrm{POSFile}(C)$
10:  $C_{final}\gets \{\}$
11: for  $c_k$  in  $C$  do
12:  $x^{temp} \gets \text{Replace } w_{j} \text{ with } c_{k} \text{ in } x'$
13: if  $\operatorname{Sim}(x^{temp}, x') > \epsilon$  then
14: Add  $c_k$  to  $C_{\text{final}}$
15:  $s_k \gets f_\theta(x^{temp})$
16: end if
17: end for
18: if  $\exists c_{k}$  whose score is  $s_k\geq \alpha$  then
19: Keep the candidates  $c_k \in C_{final}$
20:  $c^* \gets \operatorname{argmax}_{c \in C_{final}} \operatorname{Sim}(x, x_{w_j \to c}^{temp})$
21:  $x^{\prime}\gets \mathrm{Replace}w_{j}$  with  $c^*$  in  $x^{\prime}$
22: return  $x^{\prime}$
23: else if  $s_k(x') > \min s_k$  then
24:  $c^* \gets \operatorname{argmax}_{c_k \in C_{final}} s_k$
25:  $x^{\prime}\gets \mathrm{Replace}w_{j}$  with  $c^*$  in  $x^{\prime}$
26: end if
27: end for
28: return  $x'$
(line 1-6) Compute word importance. We first compute the importance of each word  $w_{i}$  in a text input  $x$ . TextFollower removes each word from  $x$  and computes their influence on the final prediction result. It then ranks the words based on their influence. In contrast, we rank the words based on the increase in the slowdown objective after each removal. By perturbing only a few words, we can minimize the alterations to  $x$  and keep the semantic similarity between  $x'$  and  $x$ . Following the original study, we filter out stop words, e.g., 'the' or 'when', to minimize sentence structure destruction.
(line 7-9) Choose the set of candidate words for substitution. The attack then works by replacing a set of words in  $x$  with the candidates carefully chosen from  $V$ . For each word  $w_{i} \in x$ , the attack collects the set of  $C$  candidates by computing the top  $N$  synonyms from  $V$  (line 8). It computes the cosine similarity between the embeddings of the word  $w_{i}$  and the word  $w' \in V$ . We use the same embeddings [28] that the original study uses. TextFooler only keeps the candidate words with the same part-of-speech (POS) as  $w_{i}$  to minimize grammar destruction (line 9).
(line 10-28) Craft a natural slowdown text. We then iterate over the remaining candidates and substitute  $w_{i}$  with  $c_{k}$ . If the text after this substitution  $x^{temp}$  is sufficiently similar to the text before it,  $x'$ , we store the candidate  $c_{k}$  into  $C_{final}$  and compute the slowdown score  $s_{k}$ . In the end, we have a perturbed text input  $x'$  that is similar to
the original input within the  $\epsilon$  similarity and the slowdown score  $s_k$  (line 10-17). To compute the semantic similarity, we use Universal Sentence Encoder that the original study uses [3].
In line 20-26, if there exists any candidate  $c_k$  that already increases the slowdown score  $s_k$  over the threshold  $\alpha$  we choose the word with the highest semantic similarity among these winning candidates. However, when there is no such candidate, we pick the candidate with the highest slowdown score, substitute the candidate with  $w_i$ , and repeat the same procedure with the next word  $w_{i+1}$ . At the end (line 28), TextFooler does not return any adversarial example if it fails to flip the prediction. However, as our goal is causing slowdown, we use this adversarial text even when the score is  $s_k \leq \alpha$ .
4 Auditing the Robustness to Adversarial Slowdown
We now utilize our WAFFLE attack as a vehicle to evaluate the robustness of the computational savings provided by multi-exit language models. Our adaptations of two adversarial text attacks, TextFooer and A2T, represent the black-box and white-box settings, respectively.
Tasks. We evaluate the multi-exit language models trained on seven classification tasks chosen from the GLUE benchmark [36]: RTE, MRPC, MNLI, QNLI, QQP, SST-2, and CoLA.
Multi-exit mechanisms. We consider three early-exit mechanisms recently proposed for language models: DeeBERT [41], PABEE [45], and Past-Future [21]. In DeeBERT, we take the pre-trained BERT and fine-tune it on the GLUE tasks. We use the pre-trained ALBERT [18] for PABEE and Past-Future. To implement these mechanisms, we use the source code from the original studies. We describe all the implementation details, e.g., the hyper-parameter choices, in Appendix.
Metrics. We employ two metrics: classification accuracy and efficacy proposed by Hong et al. [10]. We compute both the metrics on the test-set  $S$  or the adversarial texts crafted on  $S$ . Efficacy is a standardized metric that quantifies a model's ability to use its early exits. It is close to one when a higher percentage of inputs exit at an early layer; otherwise, it is 0. To quantify the robustness, we report the changes in accuracy and efficacy of a clean test-set  $S$  and  $S$  perturbed using WAFFLE.
4.1 Multi-exit Language Models Are Not Robust to Adversarial Slowdown
Attack	Metric	GLUE Task
RTE	MNLI	MRPC	QNLI	QQP	SST-2	CoLA
DeeBERT (BERT-base)
TF	Acc.	63 → 48	-	82 → 75	88 → 78	92 → 67	-	79 → 57
Eff.	0.34 → 0.32	-	0.35 → 0.32	0.35 → 0.33	0.36 → 0.40	-	0.34 → 0.20
A2T	Acc.	63 → 52	-	82 → 75	88 → 81	92 → 74	-	79 → 66
Eff.	0.34 → 0.32	-	0.35 → 0.33	0.35 → 0.35	0.36 → 0.41	-	0.34 → 0.29
WAFFLE (TF)	Acc.	63 → 51	-	82 → 61	88 → 62	92 → 69	-	79 → 70
Eff.	0.34 → 0.11	-	0.35 → 0.09	0.35 → 0.10	0.36 → 0.22	-	0.34 → 0.13
WAFFLE (A2T)	Acc.	63 → 57	-	82 → 75	88 → 78	92 → 83	-	79 → 73
Eff.	0.34 → 0.19	-	0.35 → 0.17	0.35 → 0.19	0.36 → 0.30	-	0.34 → 0.24
PABEE (ALBERT-base)
TF	Acc.	79 → 34	83 → 25	87 → 37	91 → 33	92 → 31	93 → 22	82 → 5
Eff.	0.24 → 0.22	0.28 → 0.17	0.32 → 0.21	0.31 → 0.18	0.37 → 0.27	0.37 → 0.26	0.32 → 0.23
A2T	Acc.	79 → 57	83 → 52	87 → 63	91 → 71	92 → 61	93 → 76	82 → 38
Eff.	0.24 → 0.22	0.28 → 0.21	0.32 → 0.26	0.31 → 0.27	0.37 → 0.31	0.37 → 0.32	0.32 → 0.23
WAFFLE (TF)	Acc.	79 → 57	83 → 38	87 → 47	91 → 51	92 → 67	93 → 50	82 → 48
Eff.	0.24 → 0.09	0.28 → 0.05	0.32 → 0.08	0.31 → 0.06	0.37 → 0.17	0.37 → 0.08	0.32 → 0.08
WAFFLE (A2T)	Acc.	79 → 72	83 → 69	87 → 73	91 → 82	92 → 79	93 → 85	82 → 60
Eff.	0.24 → 0.17	0.28 → 0.18	0.32 → 0.21	0.32 → 0.23	0.37 → 0.27	0.37 → 0.29	0.32 → 0.19
PastFuture (ALBERT-base)
TF	Acc.	74 → 41	86 → 42	88 → 36	92 → 52	92 → 50	-	-
Eff.	0.52 → 0.46	0.50 → 0.24	0.50 → 0.24	0.50 → 0.19	0.52 → 0.35	-	-
A2T	Acc.	74 → 58	86 → 59	88 → 58	92 → 74	92 → 64	-	-
Eff.	0.52 → 0.49	0.50 → 0.32	0.50 → 0.31	0.50 → 0.35	0.52 → 0.43	-	-
WAFFLE (TF)	Acc.	74 → 51	86 → 45	88 → 42	92 → 58	92 → 64	-	-
Eff.	0.51 → 0.17	0.50 → 0.05	0.50 → 0.15	0.50 → 0.07	0.52 → 0.25	-	-
WAFFLE (A2T)	Acc.	74 → 64	86 → 67	88 → 72	92 → 83	92 → 79	-	-
Eff.	0.52 → 0.36	0.50 → 0.26	0.50 → 0.29	0.50 → 0.33	0.52 → 0.39	-	-
Table 1: Robustness of multi-exit language models to our slowdown attacks. WAFFLE significantly reduces the computational savings offered by DeeBERT, PABEE, and PastFuture. In each cell, we report the accuracy (acc.) and efficacy (eff.) on a clean test set and its corresponding adversarial texts generated by the four attacks ( $\rightarrow$  denotes going from the clean test set to the adversarial texts).
Table 1 shows our evaluation results. Following the original studies, we set the early-exit threshold, i.e., entropy or patience, so that multi-exit language models have 0.33–0.5 efficacy on the clean test set (see Appendix for more details). We use four adversarial attacks: two standard adversarial attacks, TextFollower (TF) and A2T, and their adaptations: WAFFLE (TF) and WAFFLE (A2T). We perform
these attacks on the entire test-set and report the changes in accuracy and efficacy. In each cell, we include their flat changes and the values computed on the clean and adversarial data in parenthesis.
Standard adversarial attacks are ineffective in auditing slowdown. We observe that the standard attacks (TF and A2T) are ineffective in causing a significant slowdown. In DeeBERT, those attacks cause negligible changes in efficacy (-0.05-0.14), while they inflict a large accuracy drop (7%-25%). Against PABEE and PastFuture, we find that the changes are slightly higher than those observed from DeeBERT (i.e., 0.02-0.13 and 0.03-0.31). We can observe slowdowns in PastFuture, but this is not because the standard attacks are effective in causing slowdowns. This indicates the mechanism is more sensitive to input changes, which may lead to greater vulnerability to adversarial slowdown.
WAFFLE is an effective auditing tool for assessing the slowdown risk. We show that our slowdown attack can inflict significant changes in efficacy. In DeeBERT and PABEE, the attacks reduce the efficacy 0.06–0.26 and 0.07–0.29, respectively. In PastFuture, we observe more reduction in efficacy 0.13–0.45. These multi-exit language models are designed to achieve an efficacy of 0.33–0.5; thus its reduction up to 0.29–0.45 means a complete offset of their computational savings.
The more complex a mechanism is, the more vulnerable it is to adversarial slowdown. WAFFLE causes the most significant slowdown on PastFuture, followed by PABEE and DeeBERT. PastFuture stops forwarding based on the predictions from past exits and the estimated predictions from future exits. PABEE also uses patience, i.e., how often we observe the same decision over early-exits. They enable more aggressive efficacy compared to DeeBERT, which only uses entropy. However, this aggressiveness can be exploited by our attacks, e.g., introducing inconsistencies over exit points; thus, PABEE needs more layers to make a prediction.
4.2 Sensitivity to Attack Hyperparameter
The key hyperparameter of our attack, the attack success threshold  $(\alpha)$ , determines the magnitude of the scores pursued by WAFFLE while crafting adversaries. The score measures how uniform all output distributions of  $F_{i}$  are. A higher  $\alpha$  pushes WAFFLE to have a higher slowdown score before returning a perturbed sample. Figure 2 shows the accuracy and efficacy of all three mechanisms on QNLI against  $\alpha$  in [0.1, 1]. We show that as  $\alpha$  increases, the slowdown (represented as a decrease in efficacy) increases, and the accuracy decreases.
In addition, as  $\alpha$  increases, the rate of decrease in accuracy and efficacy decreases. Note that in PastFuture, when  $\alpha \geq 0.4$  the rate at which efficacy decreases drops by a large margin. The