Universal Adversarial Attacks with Natural Triggers for Text Classification (2021)

Abstract: Recent work has demonstrated the vulnerability of modern text classifiers to universal adversarial attacks, which are input-agnostic sequences of words added to text processed by classifiers. Despite being successful, the word sequences produced in such attacks are often ungrammatical and can be easily distinguished from natural text. We develop adversarial attacks that appear closer to natural English phrases and yet confuse classification systems when added to benign inputs. We leverage an adversarially regularized autoencoder (ARAE) to generate triggers and propose a gradient-based search that aims to maximize the downstream classifier’s prediction loss. Our attacks effectively reduce model accuracy on classification tasks while being less identifiable than prior models as per automatic detection metrics and human-subject studies. Our aim is to demonstrate that adversarial attacks can be made harder to detect than previously thought and to enable the development of appropriate defenses.

## introduction adversarial attacks have recently been quite successful in foiling neural text classifiers @xcite . universal adversarial attacks @xcite are a sub-class of these methods where the same attack perturbation can be applied to any input to the target classifier. these attacks, being input-agnostic, point to more serious shortcomings in trained models since they do not require re-generation for each input. however, the attack sequences generated by these methods are often meaningless and irregular text (e.g., "zoning tapping fiennes" from @xcite ). while § equal contribution 1 our code is available at https://github.com/ hsuan-tung/universal_attack_natural_ trigger . in this paper, we design natural attack triggers by using an adversarially regularized autoencoder (arae) @xcite , which consists of an auto-encoder and a generative adversarial network (gan). we develop a gradient-based search over the noise vector space to identify triggers with a good attack performance. our method -natural universal trigger search (nuts) -uses projected gradient descent with l 2 norm regularization to avoid using out-of-distribution noise vectors and maintain the naturalness of text generated. 2 our attacks perform quite well on two different classification tasks -sentiment analysis and natural language inference (nli). for instance, the phrase combined energy efficiency, generated by our approach, results in a classification accuracy of 19.96% on negative examples on the stanford sentiment treebank @xcite . furthermore, we show that our attack text does better than prior approaches on three different measures -average word frequency, loss under the gpt-2 language model @xcite , and errors identified by two online grammar checking tools (scr; che). a human judgement study shows that up to 77% of raters find our attacks more natural than the baseline and almost 44% of humans find our attack triggers concatenated with benign inputs to be natural. this demonstrates that using techniques similar to ours, adversarial attacks could be made much harder to detect than previously thought and we require the development of appropriate defenses in the long term for securing our nlp models. 2020 ) develop natural attacks to cause semantic collisions, i.e. make texts that are semantically unrelated judged as similar by nlp models. universal attacks universal attacks are input-agnostic and hence, word-replacing and embedding-perturbing approaches are not applicable. @xcite and @xcite concurrently proposed to perform gradient-guided searches over the space of word embeddings to choose attack triggers. in both cases, the attack triggers are meaningless and can be easily detected by a semantic checking process. in contrast, we generate attack triggers that appear more natural and retain semantic meaning. in computer vision, gans have been used to create universal attacks @xcite . concurrent to our work, @xcite design label-consistent natural triggers to attack fact checking models. they first predict unigram triggers and then use a language model conditioned on the unigram to generate natural text as the final attack, while we generate the trigger directly. ## universal adversarial attacks with natural triggers we build upon the universal adversarial attacks proposed by @xcite . to enable natural attack triggers, we use a generative model which produces text using a continuous vector input, and perform a gradient-guided search over this input space. the resulting trigger, which is added to benign text inputs, is optimized so as to maximally increase the loss under the target classification model. problem formulation consider a pre-trained text classifier f to be attacked. given a set of benign input sequences {x} with the same ground truth label y, the classifier has been trained to predict f (x) = y. our goal is to find a single inputagnostic trigger, t, that when concatenated foot_1 with any benign input, causes f to perform an incorrect classification, i.e., f ([t; x]) = y, where ; represents concatenation. in addition, we also need to ensure the trigger t is natural fluent text. ## experiments we demonstrate our attack on two tasks -sentiment analysis and natural language inference. we use the method of @xcite as a baseline foot_2 and use the same datasets and target classifiers for comparison. for the text generator, we use an arae model pre-trained on the 1 billion word dataset @xcite . for both our attack (nuts) and the baseline, we limit the vocabulary of attack trigger words to the overlap of the classifier and arae vocabularies. we generate triggers using the development set of the tasks and report results on test set (results on both sets in appendix). ## conclusion we developed universal adversarial attacks with natural triggers for text classification and experimentally demonstrated that our model can generate attack triggers that are both successful and appear natural to humans. our main goals are to demonstrate that adversarial attacks can be made harder to detect than previously thought and to enable the development of appropriate defenses. future work can explore better ways to optimally balance attack success and trigger quality, while also investigating ways to detect and defend against them.