TextDecepter: Hard Label Black Box Attack on Text Classiﬁcation
Abstract —Machine learning has been proven to be sus- ceptible to carefully crafted samples, known as adversarial examples.
The generation of these adversarial examples helps to make the models more robust and give as an insight of the underlying decision making of these models.
Over the years, researchers have successfully attacked image classiﬁers in, both, white and black-box setting.
Although, these methods are not directly applicable to texts as text data is discrete in nature.
In recent years, research on crafting adversarial examples against textual applications has been on the rise.
In this paper, we present a novel approach for hard label black-box attacks against Natural
Language Processing (NLP) classiﬁers, where no model information is disclosed, and an attacker can only query the model to get ﬁnal decision of the classiﬁer, without conﬁdence scores of the classes involved.
Such attack scenario is applicable to real world black-box models being used for security-sensitive applications such as sentiment analysis and toxic content detection.
I. I NTRODUCTION
Machine learning has shown superiority over humans for tasks like image recognition, speech recognition, security critical applications like bot, malware or spam detection.
However.
machine learning has been proven to be susceptible to carefully crafted adversarial examples.
In recent years, research on generation and development of defenses against such adversarial examples has been on the rise.
These adversarial examples help to make the models more robust by highlighting the gap between sensory information processing in humans and the decisions made by the machines.
Attack algorithms have been formulated for image classiﬁcation problems by [1], [2] .
A classic example of an adversarial attack is that of a self-driving car crashing into another car because it ignores the stop sign.
The stop sign being an adversarial example which an adversary intentionally placed in place of the original stop sign.
An example in the textual domain can be that of a spam detector which fails to detect a spam email.
The spam email is an example of adversarial attack in which the attacker has intentionally changed a few words or characters to deceive the spam detector.
The attacks can be broadly classiﬁed based on the amount of information available to the attacker as blackbox or white box attacks.
White box attacks are those in which the attacker has full information about the model’s architecture, model weights and the examples it has trained on.
Black-box attacks refer to those attacks in which only the ﬁnal output of the model is accessible to the attacker.
Black-box attacks can be further classiﬁed into 3 types based.
The ﬁrst type involves those attacks in which the probability scores to the outputs are accessible to the attacker referred to as the ’score-based black-box attacks’.
The second type of attack involves the case where information of the training data is known to the attacker.
The third attack type is the one in which only the ﬁnal decision of the classiﬁer is accessible by the attacker, with no access to the conﬁdence scores of the various classes.
We shall refer to the third type of attack as Hard Label Black-Box attack.
In the NLP domain, researchers have mainly formulated attacks in the white box setting [3], [4] with complete knowledge of gradients or the black-box setting with conﬁdence scores accessible to the attacker [5], [4], [6], [7].
As per our knowledge, there has been no prior work done to formulate adversarial attacks against
NLP classiﬁers in the hard label black-box setting.
We emphasize hard label black-box attacks to be an important category of attacks much relevant to the realworld applications as the conﬁdence scores can easily be hidden to avoid easy attacks.
Adversarial attacks on Natural Language models have to fulﬁll certain rules to qualify them as a successful attack:
(1) Semantic Similarity- meaning of the crafted example should be the same as that of the original text, as judged by humans (2) Syntactic correctness: The crafted examples should be grammatically correct
(3) Language
Fluency:
The generated example should look natural.
[7] proposed TextFooler which aimed at preserving these three properties while generating adversarial examples.
We focus on the text classiﬁcation task which is used for sentiment analysis, spam detection, topic modeling.
Sentiment analysis is widely used in the online recommendation systems, where the reviews/comments are classiﬁed into a set of categories which are useful while ranking products or movies [8].
Text classiﬁcation is also used in applications critical for online safety, like online toxic content detection [9].
Such applications involve classifying the comments or reviews into classes like irony, sarcasm, harassment and abusive content.
Adversarial attacks on text classiﬁers consist of two main steps.
First, identifying the important words in the text.
Second, introducing perturbations in those words.
For ﬁnding the important words, gradients are used in white box setting and conﬁdence scores in black-box setting.
In the absence of gradients or conﬁdence scores in the hard label black-box setting, it is non-trivial to locate important words.
In the second step, there can be either character level perturbation, like, introducing space, replacements with visually similar characters, or word level perturbation by replacement of the word with its synonym.
The selection of a character-level perturbation or synonym for replacement is based on the decrease in conﬁdence value of the original class on introducing the perturbation.
In the absence of conﬁdence scores, there are no direct indicators to help select any type of perturbation unless replacement of any single word leads to misclassiﬁcation of the entire text, which is not always the case.
In our work, we come up with heuristics to help determine the important sentences and words in the ﬁrst step and, select appropriate perturbation in the second step.
The heuristics helps us select a synonym for replacement for each of the important words, in such a way that with each successive replacement we move towards the decision boundary.
Our main contributions are as follows:
1) We propose a novel approach to formulate natural adversarial examples against NLP classiﬁers in the hard label black-box setting.
We test our attack algorithm on three state-ofthe-art classiﬁcation models over 2 popular text classiﬁcation tasks.
We improve upon the grammatical correctness of the generated adversarial examples.
We also decrease the memory requirement for the attack when compared to published attack systems involving word-level perturbations.
II. A TTACK D ESIGN A. Problem Formulation
Given an input text space X and a set of n labels , Y = { Y 1 , Y 2 , ..., Y n } , we have a text classiﬁcation model F : X → Y which maps from the input space X to the set of labels Y .
Let there be a text x ∈ X which is correctly predicted by the model to be belonging to the class y ∈
Y i.e. F ( x ) = y .
We also have a semantic similarity function Sim : X × X
→
Then, a successful adversarial attack changes the text x to x adv , such that
F ( x adv ) (cid:54) = F ( x )
Sim ( x, x adv ) ≥ (cid:15) , where (cid:15) is the minimum similarity between original and adversarial text.
Let us consider a binary classiﬁcation model with labels { Y 0 , Y 1 } which we want to attack.
We are given a text T ∈ X having m sentences, S = { s 1 , s 2 , ..., s m } .
The classiﬁcation model correctly predicts T to be belonging to class Y 0 i.e. F ( T )
=
Y 0 .
We input each of the m sentences to F and get their individual labels.
Let A and B form a partition of S such that
F ( a ) =
Y 0 , ∀ a ∈ A and P (A) be the power set of
A .
Let A r be a set of r -combinations of A .
We deﬁne another set G = { B ∪ c | c ∈ P ( A ) }} .
We refer to each of the elements in G as an ’ aggregate ’.
B. Threat Model
We consider the attack in the black-box setting, where an attacker does not have any information about the model weights or architecture and allowed to query the model with speciﬁc inputs and get the ﬁnal decision of the classiﬁer model as an output.
Further, the class conﬁdence scores are not provided to the attacker in the output, making it a hard-label black-box attack.
Although, NLP APIs provided by Google, AWS and
Azure do provide the conﬁdence scores for the classes, but in a real world application setting, like toxic content detection on a social media platform, the conﬁdence scores are not provided, thereby making it a hard label black box setting.
Such an attack scenario also helps to gauge the model robustness.
C. Methodology
The proposed methodology for generating adversarial text has three main steps:
Step 1: Sentence Importance Ranking : We observe that when people convey opinions or emotions, not all the sentences convey the same emotion, few sentences are just facts without any emotion or sentiment.
Other sentences can also be stratiﬁed based on varying level of intensity.
This forms the basis of our sentence ranking algorithm which helps to prioritize our attack on speciﬁc portions of the text in order of importance.
We assume that different sentences in text contribute to the overall class decision to a varying level of intensity.
Each of the sentences can either support or oppose the ﬁnal decision of the classiﬁer and the intensities which they do so are additive.
Consider an example of sentiment analysis, where the labels are positive and negative.
The assumption of additivity of sentence class intensity, also helps us to infer that sentences in set B when joined together to form a text, will belong to class
Y 1 .
Hereby, we refer to the same as the class of set B or the classiﬁer’s decision of set B .
We deﬁne the importance of a sentence in set A by its ability to change the classiﬁer’s decision of set B from
Y 1 to Y 0 .
If an individual sentence from set A when added to set B is able to change the class of set B from Y 1 to Y 0 , then we consider the sentence to belong to level 1 importance .
More generally, if a sentence from set A is able to change the classiﬁer’s decision of set
B only when it is put together with some subset of A with atleast k − 1 sentences, then the sentence belongs to level k importance .
The k − 1 other sentences in all such subsets also belong to level k importance .
Also, once the importance of a sentence is ﬁxed at the k th level we do not consider it in subsequent levels.
Step 2: Word importance ranking :
After ﬁnding the importance of sentences in step 1, we need to ﬁnd the importance ranking of the words to be attacked in these sentences.
We observe that words with a certain Part of Speech (POS) tags are more important than others.
For example, for a sentiment classiﬁcation task, adjectives, verbs, adverbs are more important than nouns, pronouns, conjunctions or prepositions.
Further, we consider adjectives to more important than adverbs.
Consider a sentence, “The movie was very bad”.
In this sentence, “bad” is the adjective and shapes the sentiment of the sentence.
The adverb “very” increases the intensity of the adjective, making the predicted class conﬁdence score to increase
Algorithm 1:
Input: Original Sentences set S , ground truth label y 0 , classiﬁer F ( . )
Output: Sentence Importance Ranking
SentsSentiment ← Find original labels of sentences in S ;
OrigLabelSents ← Sentences with Label Y 0 ;
OtherLabelSents ← Sentences with Label (cid:54)
=
Y 0 ;
Let OrigLabelSents P represent set of all P sentence combinations from OrigLabelSent ;
OrigLabelAggregates ← φ ;
P ← 1 ; while OrigLabelSents do
TopSentImp P ← φ ; for Comb in OrigLabelSents P do
AGG ← Add Comb to
OtherLabelSents and join it to form string ;
if F ( AGG ) =
Y 0 then
OrigLabelAggregates ← Add AGG ; for sent in Comb do
SentImp[ sent ] ← P ;
TopSentImp P ← Add sent end end end
Delete sentences in TopSentImp P from
OrigLabelSents ;
P ← P + 1 ; if P > Length ( OrigLabelSents ) then Add remaining sentences in OrigLabelSents to TopSentImp P ; end endreturn SentImp , OrigLabelAggregates further.
Step 3:
Attack : We use the Word Level perturbations in order of word importance obtained from the previous step.
We select synonyms to replace the original words using cosine distance between word vectors.
Further, to maintain the syntax of the language, only those synonyms having the same POS tags as that of original word are considered for further evaluations.
Experiments are done using, both, coarse and ﬁne POS tag masks.
Let us look at the details of each of these steps:
1) Synonym extraction: We use the word embedding by [10] which obtained state-of the art perfor-
Input: Original text X , ground truth label Y 0 , classiﬁer F ( . ) , semantic similarity threshold (cid:15) , cosine similarity matrix
Output: Adversarial example X adv
Initialization: X adv ← X ;
Segment X into sentences to get set S ;
SentImp, OrigLabelAggs = GetSentenceImp(S);
WordImpScores = GetWordImp (SentImp);
Create a set W of all words w i
∈
X sorted by the descending order of their wordImpScores;
MISCLASSIFIED ← False ; for w j in W do if MISCLASSIFIED then break; end
CANDIDATES =
GetSynonyms( w j );
CANDIDATES ← POSFilter (CANDIDATES) ;
CANDIDATES ← SEMANTICSIMFilter (CANDIDATES) ;
FINCANDIDATES ← Sort CANDIDATES by semantic similarity ;
CHANGED ← False ; for c k in FINCANDIDATES do
X (cid:48) ← Replace w j with c k in X adv ; if F ( X (cid:48) )
(cid:54) =
Y 0 then
X adv =
X (cid:48) ;
CHANGED ←
True ;
MISCLASSIFIED ← True ; endif NOT CHANGED then
SENT ← Get the sentence in which w j belongs s.t F (SENT)
=
Y 0 ;
X (cid:48) ← Replace w j with c k in SENT; if F ( X (cid:48) )
(cid:54) =
Y
; then
X adv =
X (cid:48) ;
CHANGED ← True ; end endif NOT CHANGED then
OrigLabelAGG ← Get the aggregate in which w j belongs s.t. F ( AGG ) =
Y 0 ;
X (cid:48) ← Replace w j with c k in OrigLabelAGG ; if F ( X (cid:48) )
(cid:54) =
Y
; then
X adv =
X (cid:48) ;
CHANGED ←
True ; end end end end mance on SimLex-999, a dataset designed by [11] to measure how well different models judge the semantic similarity between words. 2) POS checking: In order to maintain the syntax of the adversarial example generated, we ﬁlter out the synonyms which have a different POS tag then the original word.
We experiment with, both, coarse and ﬁne POS tagging.
We select a synonym to replace a particular word if its usage lead to one of the following:
1) Misclassiﬁcation of the entire text.
2) Misclassiﬁcation of the original labeled sentence to which the target word belongs.
3) Misclassiﬁcation of the original label aggregate to which the target word belongs.
If multiple synonyms fulﬁll the rules, then the one which fulﬁlls the rule of higher preference is
selected. If multiple synonyms fulﬁll the highest preference rule, then that synonym is selected whose placement in the review is semantically nearest to the original review.
We terminate the algorithm once the text misclassiﬁes, or when all the important words have been iterated over.
The justiﬁcation for the higher preference of sentence with respect to the aggregate to which it belongs comes from the additivity assumption.
Let us take a sentence v ∈ A ,
{ v } ∈
A r .
Now, add it to set B to form an ’aggregate’, i.e. B ∪ { v } ∈ G .
Assuming the additivity of class intensities of sentences, we can easily see that when sentences in B ∪ { v } are joined to form a piece of text, it either belongs to class Y 1 or, in case, it belongs to class Y 0 , then the intensity of class Y 0 is lesser when compared to v alone.
In other words, an ’aggregate’ belonging to class Y 0 has lesser intensity of class Y 0 when compared with individual sentences belonging to class Y 0 which are part of that aggregate.
Hence, a synonym which is able to ﬂip the decision of the classiﬁer, both, on the sentence and the aggregate (initially classiﬁed as Y 0 ) to which the sentence belongs would be preferred against a synonym which is just able to change the classiﬁer’s decision on the aggregate alone. Step 4: Reset insigniﬁcant replacements
Consider a directed acyclic graph, where a node represents a text and a directed edge between two nodes means that the latter node text can be obtained by replacement of a word w in former node text by its most semantically similar synonym w (cid:48) , given that the synonym has the same POS tag as the original word.
Further, the weight of each edge is 1.
Then, crafting an adversarial example from a given piece of text can be formulated as a path ﬁnding problem as follows; ﬁnd a path from an initial node T to ﬁnal node T (cid:48) , such that F ( T )
=
Y 0 and F ( T (cid:48) ) (cid:54) = Y 0 . The sentence and word ranking algorithm decide the order of perturbation.
Thereafter, we use the aggregates belonging to the original class as heuristics to guide our graph search.
The algorithm stops once the text misclassiﬁes.
Although, the heuristics help us in ﬁnding a path, but they do not guarantee if it is the shortest path.
In the absence of conﬁdence scores of classes, the problem of ﬁnding the shortest path has an exponential search space.
So, once the given text is turned adversarial, the algorithm visits all the replaced words and resets those replacements without which the perturbed text still remains adversarial.
D. Time Complexity
Let there be n sentences in a text and m words in total.
1) Getting the class label of each of the individual sentences: O ( n ) 2) Sentence Importance ranking: The algorithm takes all possible combinations of sentences from the set of original labeled sentences and insert each of them to the set of sentences which do not have the original label.
Hence, the time complexity of this step becomes O (2 n ) .
Word importance ranking: a) Sort the original labeled sentences in order of their importance ranking: O ( n log n ) b) Iterate over all the words in the sorted sentences and make the word perturb sequence: O ( m ) c)
Generation of cosine similarity matrix of words in text with 65 thousand words in the vocabulary: O ( m ) 4) Pick the top k synonyms for each of the candidate words that are to be perturbed.
The cosine similarity matrix consists of similarity of each of the words in the text with all the words in the vocabulary.
We sort the similarity values for each word with all the other words in the vocabulary.
Hence, amounting to total time complexity of O ( m ) 5) Attack: Replace each of the k synonyms of a word in the original
text, the original labeled aggregates (obtained from sentence importance ranking step), and the original labeled sentences to which they belong.
Hence, the time complexity of this step becomes O ( kmn ) .
6) Removal of insigniﬁcant perturbations: O ( m )
Hence, the overall time complexity of the attack framework becomes O (2 n ) + O ( kmn
E. Memory requirement
The generation of adversarial adversarial texts requires that we are able to get the synonyms for a particular word during the attack phase.
Hence, a cosine similarity matrix matrix is kept in the RAM to quickly generate synonyms for a particular word.
In our attack framework, we keep the cosine similarity of only the words in the text that is being attacked, with all the 65 thousands words in the vocabulary.
It is an improvement over the memory requirement in attack framework of TextFooler [7] which keeps the cosine similarity matrix of all the 65 thousand words in the vocabulary, constituting a square matrix of size 65 thousand, consuming nearly 17 GB of RAM if single-precision ﬂoating-point format is used.
III. A TTACK E VALUATION : S ENTIMENT A NALYSIS
We evaluate our attack methodology on generating adversarial texts for sentiment analysis task.
Sentiment
Analysis is a text classiﬁcation task which identiﬁes and characterizes the sentiment of a given text.
It is widely used by businesses to get the sentiment of customers towards their product or services, by analysing reviews or survey responses.
A. Datasets and Models
We study the effectiveness of our attack methodology on sentiment classiﬁcation on IMDB and Movie Review (MR) datasets.
We target three models: word-based convolutional neural network (WordCNN)
[12], wordbased long-short term memory (WordLSTM) [13], and
Bidirectional Encoder Representations from Transformers (BERT) [14].
We attack the pretrained models open sourced by [7] and evaluate our attack algorithm on the same set of 1000 examples that the authors had used in their work.
We also run the attack algorithm against
Google Cloud NLP API.
The summary of the datasets used by [7] for training the models are in Table I and their original accuracy are given in Table II
[Table] [Table] [Table]
B. Evaluation
1) Attack success rate: We ﬁrst measure the accuracy of the target model on the 1000 test samples and call it original accuracy.
Then, we measure the accuracy of the target models against the adversarial samples generated from the same test samples and call it after-attack accuracy.
The difference between the original accuracy and the after-attack accuracy of a classiﬁcation model is called attack success rate.
2) Percentage of Perturbed words: The percentage of words replaced by their synonyms on an average gives us a metric to quantify the change made to a given text.
3) Semantic similarity: It tells us the degree to which the two given texts carry the similar meaning.
We use the Universal Sentence Encoder to measure the Semantic Similarity between original and adversarial text.
Since, my main aim is to generate adversarial texts, we just control the semantic similarity to be above a certain threshold.
4) Number of queries: The average number of queries made to the target model tells us the efﬁciency of the attack model.
IV. R ESULTS A. Automatic Evaluation
We report our results of the hard label black-box attacks in terms of automatic evaluation on two text classiﬁcation tasks using coarse and ﬁne POS masks.
The main results are summarized in Tables III and IV.
Our attack algorithm is able to bring down the accuracy of all the major text classiﬁcation models with an attack success rate greater then 50% for all the models.
Further, the percentage of perturbed words are nearly 3% for all the models on IMDB dataset and between 10-16% for all the models on MR dataset.
In the IMDB dataset, which has an average word length of 215 words, our attack model is able to conduct successful attacks by perturbing less then 7 words on an average.
That means that our attack model is able to identify the important words in the text and makes subtle manipulations to mislead the classiﬁers.
Overall, our algorithm is able to attack text classiﬁcation models pertaining to sentiment analysis with an attack success rate greater then 50%, no matter how long the text sequence or how accurate the target model.
Further, our model requires the least amount of information among all the models it is compared with.
The attack model is also able to attack GCP NLP API and brings down the accuracy from 76.4% to 16.7% for
MR dataset.
Further, it changes only 10.2% of the words in the text to generate the adversary.
The results are unprecedented as we have achieved the results without having any information about the conﬁdence scores of the classes involved.
The attack algorithm and the carefully crafted adversarial texts can be utilized for the study of interpretability of BERT model [15].
The query number is almost linear to the text length, with a ratio in (6,10) which is at par with [7] and [4].
B. Benchmark Comparison
We compare our attack against state-of-the-art adversarial attack systems on the same target model and dataset.
For GCP NLP API, we compare our attack results against [4] and [5] on MR datasets.
With wordCNN and wordLSTM as the target models, the comparison is against [4], [6], [7].
The results of the comparison are summarised in table V and VI.
The lower attack success rates when compared to the other attack systems can be attributed to the fact that our attack system does not make use of conﬁdence scores of the classes which other published systems do.
C. Human Evaluation
Following the practice of [7], we perform human evaluation by sampling 100 adversarial examples from the MR dataset with the WordLSTM.
We perform three experiments to verify the quality of our adversarial examples.
First, the human judges are asked to give the gramaticality score of a shufﬂed mix of original and adversarial text on a scale of 1-5.
As shown in Table, the grammaticality of the adversarial texts with ﬁne POS tag mask are closer to the original texts when compared with coarser POS tag mask.
Although, both of scores are above 4 meaning that using, both, coarse and ﬁne POS tags result in smooth adversarial texts.
Second, the judges assign classiﬁcation labels to a shufﬂed set of original and adversarial texts, for both coarse and ﬁne POS masks.
The results show that the overall agreement between the labels of the original and adversarial text for both the cases are quite high, 92% and 93% respectively.
This suggests that improving the gramaticality of the adversarial texts using ﬁne POS
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
mask does not contribute much to the overall meaning of the texts to humans.
Third
[Table]
the judges determine whether the adversarial texts retain the meaning of the original text.
The judges are given three options, 1 for similar, 0.5 for ambiguous and 0 for dissimilar. The average sentence similarity score is 0.88 when ﬁne POS mask is used compared to 0.86 when coarse POS mask is used for synonym selection, suggesting a marginal improvement in sentence similarity scores in the former. [Table] after the book i became very sad when i was watching the movie .
i am agree that sometimes a ﬁlm should be different from the original novel
but in this case it was more than acceptable .
some examples: 1 ) why the ranks are different ( e.g. lt . diestl instead of sergeant etc.) 2 ) the ﬁnal screen is very poor and makes diestl as a soldier who feds up himself and wants to die .
but it is not true in 100 % .
just read the book .
he was a bull - dog in the last seconds as well .
he did not want to die by wrecking his gun and walking simply towards to michael & noah .
so this is some kind of a happy end which does not ﬁt at all for this movie .
[Table]
after the book i became very bleak when i was watching the movie .
i am agree that sometimes a ﬁlm should be different from the original novel
but in this case it was more than acceptable .
some examples:1 ) why the ranks are different ( e.g. lt . diestl instead of sergeant etc.)
2 ) the ﬁnal screen is very ﬂawed and makes diestl as a soldier who feds up himself and wants to die .
but it is not true in 100 % .
just read the book .
he was a bull - dog in the last seconds as well .
he did not want to die by wrecking his gun and walking simply towards to michael & noah .
so this is some kind of a happy end which does not ﬁt at all for this movie .
[Table]
seriously , i do n’t understand how justin long is becoming increasingly popular .
he either has the best agent in hollywood , or recently sold his soul to satan .
he is almost unbearable to watch on screen , he has little to no charisma , and terrible comedic timing .
the only ﬁlm that he has attempted to anchor that i ’ve remotely enjoyed was waiting ...
and that is almost solely because i ’ve worked in a restaurant .
but i digress .
aside from it ’s terrible lead , this ﬁlm has loads of other debits .
i understand that it ’s supposed to be a cheap popcorn comedy , but that does n’t mean that it has to completely insult our intelligence , and have writing so incredibly hackneyed that it borders on offensive .
lewis black ’s considerable talent is wasted here too , as he is at his most incendiary when he is unrestrained , which the pg-13 rating certainly wo n’t allow .
the ﬁlm ’s sole bright spot was jonah hill ( who will look almost unrecognizable to fans of the recent superbad due to the amount of weight he lost in the interim ) .
his one liners were funny on occasion , but were certainly not enough to make this anywhere close to bearable .
if you just want to completely turn your brain off ( or better yet , do n’t have one )
then maybe you ’d enjoy this , but i ca n’t recommend it at all .
[Table]
seriously , i do n’t understand how justin long is becoming increasingly popular .
he either has the best agent in hollywood , or recently sold his soul to satan .
he is almost terrible to watch on screen , he has little to no charisma , and spooky comedic timing .
the only ﬁlm that he has attempted to anchor that i ’ ve remotely enjoyed was waiting ...
and that is almost solely because i ’ ve worked in a restaurant .
but i digress .
aside from it ’s spooky lead , this ﬁlm has loads of other debits .
i understand that it ’s supposed to be a miserly popcorn comedy , but that does n’t mean that it has to completely insult our intelligence , and have writing so incredibly hackneyed that it borders on offensive .
lewis black ’s considerable talent is wasted here too , as he is at his most incendiary when he is unrestrained , which the pg-13 rating certainly wo n’t allow .
the ﬁlm ’s sole bright spot was jonah hill ( who will look almost unrecognizable to fans of the recent superbad due to the amount of weight he lost in the interim ) .
his one liners were funny on occasion , but were certainly not enough to make this anywhere close to bearable .
if you just want to completely turn your brain off ( or better yet , do n’t have one )
then maybe you ’d enjoy this , but i ca n’t recommend it at all .
[Table]
Part of Speech (POS) tags can be of two types:
Coarse grained (Noun, Verb, Adjective, etc.) or Fine grained (Noun-proper-singular, noun-proper-plural, verbpast, verb-present, etc). TextDecepter uses a POS mask to reject those synonyms which do not belong to the same POS as that of the original word, when they are placed in the text.
Fine grained POS mask helps to maintain the grammaticality to a greater extent when compared to the coarse grained POS mask as demonstrated in
Table X.
However, during human evaluation we observed that improving the grammaticality of the adversarial texts for sentiment classiﬁcation dataset using ﬁne POS does not contribute much to the overall understanding of the
[Table] [Table]
text for humans.
We recommend using the appropriate
POS mask depending on the grammaticality requirement for the context in which adversarial texts are being generated.
V. D ISCUSSION A. Ablation study
Aggregates The most critical step of our algorithm is the use of aggregates, which belong to the original class, to select or reject synonyms for replacement.
To validate the effectiveness of this step we remove the usage of aggregates select a synonym for replacement only when its presence is able to misclassify the original text.
The results for BERT model is shown in table .
After removing the sentence importance ranking step , we see that the after-attack accuracy increases by 32% for IMDB and 35% for MR dataset, respectively.
This suggests the importance of aggregates for selecting synonym for replacement, the removal of which renders the attack ineffective.
The aggregates generated in the sentence importance ranking step help us to select those synonyms which can take the original text towards misclassiﬁcation.
VI. C ONCLUSION
We propose a hard label black-box attack strategy for text classiﬁcation tasks.
We also conduct extensive experimentation on sentiment analysis datasets to validate our attack system.
We also conduct human evaluation to validate the grammatical and semantic correctness of the generated adversarial examples. The attack algorithm uses the assumption of additivity of class intensities of sentences to craft adversarial examples and achieves attack success rate of more than 50 % against state-of-theart text classiﬁcation models. The adversarial examples generated can be utilized to improve the existing text classiﬁcation models for sentiment analysis by including them in the training dataset. The attack algorithm can also be used to gauge the robustness of text classiﬁcation models pertaining to sentiment analysis.
We also improved upon the memory requirement for the attack, as compared to the other comparative attack frameworks. As a future line of work, it would be imperative to run the attack algorithm on multi-class classiﬁcation and textual entailment tasks to test the generalization capability of the proposed methodology.
A CKNOWLEDGMENT
I thank Prof. Sunil Shende for insightful discussions.
I especially appreciate Kalyan Alapati and Dheenadhyalan
Kumaraswamy for helping with human evaluation.