Assessing Robustness of Text Classification through Maximal Safe Radius Computation (2020)

Abstract: Neural network NLP models are vulnerable to small modifications of the input that maintain the original meaning but result in a different prediction. In this paper, we focus on robustness of text classification against word substitutions, aiming to provide guarantees that the model prediction does not change if a word is replaced with a plausible alternative, such as a synonym. As a measure of robustness, we adopt the notion of the maximal safe radius for a given input text, which is the minimum distance in the embedding space to the decision boundary. Since computing the exact maximal safe radius is not feasible in practice, we instead approximate it by computing a lower and upper bound. For the upper bound computation, we employ Monte Carlo Tree Search in conjunction with syntactic filtering to analyse the effect of single and multiple word substitutions. The lower bound computation is achieved through an adaptation of the linear bounding techniques implemented in tools CNN-Cert and POPQORN, respectively for convolutional and recurrent network models. We evaluate the methods on sentiment analysis and news classification models for four datasets (IMDB, SST, AG News and NEWS) and a range of embeddings, and provide an analysis of robustness trends. We also apply our framework to interpretability analysis and compare it with LIME.

## introduction deep neural networks (dnns) have shown great promise in natural language processing (nlp), outperforming other machine learning techniques in sentiment analysis @xcite , language translation @xcite , speech recognition @xcite and many other tasks foot_0 . despite these successes, concerns have been raised about robustness and interpretability of nlp models @xcite . it is known that dnns are vulnerable to adversarial examples, that is, imperceptible perturbations of a test point that cause a prediction error @xcite . in nlp this issue manifests itself as a sensitivity of the prediction to small modifications of the input text (e.g., replacing a word with a synonym). in this paper we work with dnns for text analysis and, given a text and a word embedding, consider the problem of quantifying the robustness of the dnn with respect to word substitutions. in particular, we define the maximal safe radius (msr) of a text as the minimum distance (in the embedding space) of the text from the decision boundary, i.e., from the nearest perturbed text that is classified differently from the original. unfortunately, computation of the msr for a neural network is an np-hard problem and becomes impractical for real-world networks @xcite . as a consequence, we adapt constraint relaxation techniques @xcite @xcite developed to compute a guaranteed lower bound of the msr for both convolutional (cnns) and recurrent neural networks (rnns). furthermore, in order to compute an upper bound for the msr we adapt the monte carlo tree search (mcts) algorithm @xcite to word embeddings to search for (syntactically and semantically) plausible word substitutions that result in a classification different from the original; the distance to any such perturbed text is an upper bound, albeit possibly loose. we employ our framework to perform an empirical analysis of the robustness trends of sentiment analysis and news classification tasks for a range of embeddings on vanilla cnn and ltsm models. in particular, we consider the imdb dataset @xcite , the stanford sentiment treebank (sst) dataset @xcite , the ag news corpus dataset @xcite and the news dataset @xcite . we empirically observe that, although generally nlp models are vulnerable to minor perturbations and their robustness degrades with the dimensionality of the embedding, in some cases we are able to certify the text's classification against any word substitution. furthermore, we show that our framework can be employed for interpretability analysis by computing a saliency measure for each word, which has the advantage of being able to take into account non-linearties of the decision boundary that local approaches such as lime @xcite cannot handle. in summary this paper makes the following main contributions: related work. deep neural networks are known to be vulnerable to adversarial attacks (small perturbations of the network input that result in a misclassification) @xcite @xcite . the nlp domain has also been shown to suffer from this issue @xcite @xcite @xcite . the vulnerabilities of nlp models have been exposed via, for example, small character perturbations @xcite , syntactically controlled paraphrasing @xcite , targeted keywords attacks @xcite , and exploitation of back-translation systems @xcite . formal verification can guarantee that the classification of an input of a neural network is invariant to perturbations of a certain magnitude, which can be established through the concept of the maximal safe radius @xcite or, dually, minimum adversarial distortion @xcite . while verification methods based on constraint solving @xcite and mixed integer programming @xcite can provide complete robustness guarantees, in the sense of computing exact bounds, they are expensive and do not scale to real-world networks because the problem itself is np-hard @xcite . to work around this, incomplete approaches, such as search-based methods @xcite or reachability computation @xcite , instead compute looser robustness bounds with much greater scalability, albeit relying on the knowledge of nontrivial lipschitz constants. in this work, we exploit approximate, scalable, linear constraint relaxation methods @xcite @xcite , which do not assume lipschitz continuity. in particular, we adapt the cnn-cert tool @xcite and its recurrent extension popqorn @xcite to compute robustness guarantees for text classification in the nlp domain. we note that nlp robustness has also been addressed using interval bound propagation @xcite . ## robustness quantification of text classification against word substitutions in text classification an algorithm processes a text and associates it to a category. raw text, i.e., a sequence of words (or similarly sentences or phrases), is converted to a sequence of real-valued vectors through an embedding e : @xmath0, which maps each element of a finite set w (e.g., a vocabulary) into a vector of real numbers. there are many different ways to build embeddings @xcite @xcite , nonetheless their common objective is to capture relations among words. furthermore, it is also possible to enforce into the embedding syntactic/semantic constraints, a technique commonly known as counter-fitting @xcite , which we assess from a robustness perspective in section 3. each text is represented univocally by a sequence of vectors @xmath1, . . . , x m ), where @xmath2, x @xmath3, padding if necessary. in this work we consider text classification with neural networks, hence, a text embedding x is classified into a category @xmath4, through a trained network n : r d•m [0,1] → r |c| , i.e., @xmath5, where without any loss of generality we assume that each dimension of the input space of n is normalized between 0 and 1. we note that pre-trained embeddings are scaled before training, thus resulting in a l ∞ diameter whose maximum value is 1. thus, the lower and upper bound measurements are affected by normalization only when one compares embeddings with different dimensions with norms different from l ∞ . in this paper robustness is measured for both convolutional and recurrent neural networks with the distance between words in the embedding space that is calculated with either l 2 or l ∞ -norm: while the former is a proxy for semantic similarity between words in polarized embeddings (this is discussed more in details in the experimental section), the latter, by taking into account the maximum variation along all the embedding dimensions, is used to compare different robustness profiles. ## experimental results we use our framework to empirically evaluate the robustness of neural networks for sentiment analysis and news classification on typical cnn and lstm architectures. while we quantify lower @xcite , the ag news corpus @xcite and the news dataset @xcite : details are in ## conclusions we introduced a framework for evaluating robustness of nlp models against word substitutions. through extensive experimental evaluation we demonstrated that our framework allows one to certify certain architectures against single word perturbations and illustrated how it can be employed for interpretability analysis. while we focus on perturbations that are syntactically coherent, we acknowledge that semantic similarity between phrases is a crucial aspect that nonetheless requires an approach which takes into account the context where substitutions happen: we will tackle this limitation in future. furthermore, we will address robustness of more complex architectures, e.g., networks that exploit attention-based mechanisms @xcite .