SHIELD : Defending Textual Neural Networks against Multiple Black-Box Adversarial Attacks with Stochastic Multi-Expert Patcher (2022)

Abstract: Even though several methods have proposed to defend textual neural network (NN) models against black-box adversarial attacks, they often defend against a specific text perturbation strategy and/or require re-training the models from scratch. This leads to a lack of generalization in practice and redundant computation. In particular, the state-of-the-art transformer models (e.g., BERT, RoBERTa) require great time and computation resources. By borrowing an idea from software engineering, in order to address these limitations, we propose a novel algorithm, SHIELD, which modifies and re-trains only the last layer of a textual NN, and thus it “patches” and “transforms” the NN into a stochastic weighted ensemble of multi-expert prediction heads. Considering that most of current black-box attacks rely on iterative search mechanisms to optimize their adversarial perturbations, SHIELD confuses the attackers by automatically utilizing different weighted ensembles of predictors depending on the input. In other words, SHIELD breaks a fundamental assumption of the attack, which is a victim NN model remains constant during an attack. By conducting comprehensive experiments, we demonstrate that all of CNN, RNN, BERT, and RoBERTa-based textual NNs, once patched by SHIELD, exhibit a relative enhancement of 15%–70% in accuracy on average against 14 different black-box attacks, outperforming 6 defensive baselines across 3 public datasets. All codes are to be released.

## introduction adversarial text attack and defense. after being trained to maximize prediction performance, textual nn models frequently become vulnerable to adversarial attacks @xcite . in the nlp domain, in general, adversaries utilize different strategies to perturb an input sentence such that its semantic meaning is preserved while successfully letting a target nn model output a desired prediction. text perturbations are typically generated by replacing or inserting critical words (e.g., hotflip @xcite , textfooler @xcite ), characters (e.g., deepwordbug @xcite , textbugger @xcite ) in a sentence or by manipulating a whole sentence (e.g., scpna @xcite , gan-based @xcite ). since many recent nlp models are known to be vulnerable to adversarial black-box attacks (e.g., fake news detection @xcite , dialog systems @xcite , and so on), robust defenses for textual nn models are required. even though several papers have proposed to defend nns against such attacks, they were designed for either a specific type of attack (e.g., word or synonym substitution @xcite @xcite , misspellings @xcite , characterlevel @xcite , or word-based @xcite ). even though there exist some general defensive methods, most of them enrich nn models by re-training them with adversarial data augmented via known attack strategies @xcite @xcite or with external information such as knowledge graphs @xcite . however, these augmentations often induce substantial overhead in training or are still limited to only a small set of predefined attacks (e.g., @xcite ). hence, we are in search of defense algorithms that directly enhance nn models' structures (e.g., @xcite ) while achieving higher generalization capability without the need of acquiring additional data. motivation (fig. 1 ). different from white-box attacks, black-box attacks do not have access to a target model's parameters, which are crucial for achieving effective attacks. hence, attackers often the attacker assumes the model remains unchanged and (b) gives coherent signal during the iteration search, resulting in the true best attack: "dirty"→"dirrty". (c) a model patched with shield utilizes a weighted ensemble of 3 diverse heads depending on the input. therefore, the ensemble weights keep changing over time during adversaries' perturbation search processes -the line width represents the ensemble weights. (d) shield confuses the attacker with 3 varying distributions of the score, resulting in a sub-optimal attack "people"→"pe0ple". proposal. to address these challenges, we borrow ideas from software engineering where bugs can be readily removed by an external installation patch. specifically, we develop a novel neural patching algorithm, named as shield, which patches only the last layer of an already deployed textual nn model (e.g., cnn, rnn, transformers @xcite ) and transforms it into an ensemble of multi-experts or prediction heads (fig. 1c ). during inference, then shield automatically utilizes a stochastic weighted ensemble of experts for prediction depending on inputs. this will obfuscate adversaries' perturbation search, making black-box attacks much more difficult regardless of attack types, e.g., character or word level attacks (fig. 1c , d ). by patching only the last layer of a model, shield also introduces lightweight computational overhead and requires no additional training data. in summary, our contributions are as follows: ## the proposed method: shield we introduce stochastic multi-expert neural patcher (shield) which patches only the last layer of an already trained nn model f (x, θ) and transforms it into an ensemble of multiple expert predictors with stochastic weights. these predictors are designed to be strategically selected with different weights during inference depending on the input. this is realized by two complementary modules, namely (i) a stochastic ensemble (se) module that transforms f (•) into a randomized ensemble of different heads and (ii) a multi-expert (me) module that uses neural architecture search (nas) to dynamically learn the optimal architecture of each head to promote their diversity. ## discussion performance under budgeted attacks. shield not only improves the overall robustness of the patched nn model under a variety of black-box attacks, but also induces computational cost that can greatly discourage malicious actors to exercise adversarial attacks in practice. we define computational cost as # of queries on a target nn model that is required for a successful attack. since adversaries usually have an attack budget on # of model queries (e.g. a monetary budget, limited api access to the black-box model), the higher # of queries required, the less vulnerable a target model is to adversarial threats. a larger budget is crucial for genetic-based attacks because they usually require larger # of queries than greedy-based strategies. we have demonstrated in sec. 3.2 that shield is robust even when the attack budget is unlimited. fig. 2 shows that the performance of roberta after patched by shield also reduces at a slower rate compared to the base roberta model when the attack budget increases, especially under greedy-based attacks. ## limitations and future work in this paper, we limit the architecture of each expert to be an fcn with a maximum of 3 hidden layers (except the base model). if we include more options for this architecture (e.g., attention @xcite ), sub-models' diversity will significantly increase. the design of shield is modelagnostic and is also applicable to other complex and large-scale nns such as transformers-based models. especially with the recent adoption of transformer architecture in both nlp and computer vision @xcite sacrifice on running time. ## related work defending against black-box attacks. most of previous works (e.g., @xcite @xcite @xcite @xcite in adversarial defense are designed either for a specific type (e.g., word, synonymsubstitution as in certified training @xcite , misspellings @xcite ) or level (e.g., character or word-based) of attack. thus, they are usually evaluated against a small subset of (≤4) attack methods. despite there are works that propose general defense methods, they are often built upon adversarial training @xcite which requires training everything from scratch (e.g., @xcite @xcite or limited to a set of predefined attacks (e.g., @xcite ). although adversarial trainingbased defense works well against several attacks on bert and roberta, its performance is far out-weighted by shield ( contrast to previous approaches, shield addresses not the characteristics of the resulted perturbations from the attackers but their fundamental attack mechanism, which is most of the time an iterative perturbation optimization process (fig. 1 ). this allows shield to effectively defend against 14 different black-box attacks ( ensemble-based defenses. shield is distinguishable from previous ensemble-based defenses on two aspects. first, previous approaches such as dt @xcite , adp @xcite are mainly designed for computer vision. applying these models to the nlp domain faces a practical challenge where training multiple memory-intensive sota sub-models such as bert or roberta can be very costly in terms of space and time complexities. in contrast, shield enables to "hot-fix" a complex nn by replacing and training only the last layer, removing the necessity of re-training the entire model from scratch. second, previous methods (e.g., dt and adp) mainly aim to reduce the dimensionality of adversarial subspace, i.e., the subspace that contains all adversarial examples, by forcing the adversaries to attack a single fixed ensemble of diverse sub-models at the same time. this then helps improve the transferability of robustness on different tasks. however, our approach mainly aims to dilute not transfer but direct attacks by forcing the adversaries to attack stochastic, i.e., different, ensemble variations of sub-models at every inference passes. this helps shield achieve a much better defense performance compared to dt and adp across several attacks ( ## conclusion this paper presents a novel algorithm, shield, which consistently improves the robustness of textual nn models under black-box adversarial attacks by modifying and re-training only their last layers. by extending a textual nn model of varying architectures (e.g., cnn, rnn, bert, roberta) into a stochastic ensemble of multiple experts, shield utilizes differently-weighted sets of prediction heads depending on the input. this helps shield defend against black-box adversarial attacks by breaking their most fundamental assumption-i.e., target nn models remain unchanged during an attack. shield achieves average relative improvements of 15%-70% in prediction accuracy under 14 attacks on 3 public nlp datasets, while still maintaining similar performance on clean examples. thanks to its modeland domain-agnostic design, we expect shield to work properly in other nlp domains.