V ert A ttack: Taking Advantage of Text Classifiers’ Horizontal Vision (2024)

Abstract: Text classification systems have continuouslyimproved in performance over the years. How-ever, nearly all current SOTA classifiers have asimilar shortcoming, they process text in a hor-izontal manner. Vertically written words willnot be recognized by a classifier. In contrast,humans are easily able to recognize and readwords written both horizontally and vertically.Hence, a human adversary could write problem-atic words vertically and the meaning wouldstill be preserved to other humans. We simulatesuch an attack, VertAttack. VertAttack identifieswhich words a classifier is reliant on and thenrewrites those words vertically. We find thatVertAttack is able to greatly drop the accuracyof 4 different transformer models on 5 datasets.For example, on the SST2 dataset, VertAttackis able to drop RoBERTa’s accuracy from 94 to13%. Furthermore, since VertAttack does notreplace the word, meaning is easily preserved.We verify this via a human study and find thatcrowdworkers are able to correctly label 77%perturbed texts perturbed, compared to 81% ofthe original texts. We believe VertAttack offersa look into how humans might circumvent clas-sifiers in the future and thus inspire a look intomore robust algorithms.

## introduction automatic text classifiers have seen a continual increase in helping websites moderate and monitor products or people. though they are helpful to reduce the work load of humans, they can be subject to problems like bias @xcite and are vulnerable to adversarial attacks @xcite . research into text adversarial attacks has been on the rise in recent years. the reasons range from testing classifiers' robustness @xcite to privacy concerns @xcite . current state-of-the-art (sota) attacks largely fall into character based attacks and word-based 1. propose an attack (vertattack) to mimic how humans may subvert automatic classifiers. this attack exploits current classifiers' glaring weakness (inability to "read" vertical text). we find that, on average, vertattack is able to drop classifiers' accuracy to 36.6% accuracy, which is lower than bert-attack (47.5%) and textbugger (63.2%). the success of vertattack shows a vulnerability in classifiers which humans may leverage to easily defeat them. we share code and perturbed texts for future research. ## threat model the examined threat model follows from prior research @xcite @xcite . we assume blackbox knowledge of a classifier. that is, vertattack has no internal knowledge of the classifier, but has access to the probabilities and label output by the model. vertattack uses this for feedback (section 4.1). with prior research, there is an assumption that the feedback classifier is the same as the target classifier. however, websites rarely share the exact 112 classifier used for moderating texts. thus, we also 113 examine the cases of where the feedback classifier first the attack finds which word most helps i really hate this restaurant, it deserves no love. classifier 1. word selection _ really hate this restaurant, it deserves no love. i really hate this restaurant, it deserves no _ … 2. word transformation i really h this restaurant, it a t e deserves no love. check transformed text i really h this restaurant, it a t e deserves n love. o classifier incorrect? yes: output final text no: continue modifying 3. check and repeat figure 2: vertattack basic overview. a word to transform is first selected from the input text and then transformed vertically. the classifier assists in providing feedback in the form of class probabilities. the process is repeated until the classifier misclassifies the text. the classifier. we employ a greedy search method 144 (algorithm 1). in previous work this has been 145 referred to as word importance (jin et al., 2020) or 146 greedy selection (hsieh et al., 2019). the method 147 removes one word 1 at a time and checks the change 148 in classification probability from the original text. 149 each word is removed and then replaced until all 150 probabilities are calculated. the word that causes 151 the highest drop in probability is chosen as the 152 word to be transformed. 153 4.2 word transformation 154 algorithm 2 word transformation input: text, perturb positions output: text m # lines ← max length of words to be modified k ← 0 while @xmath0 once a word is selected it is then transformed 155 vertically (algorithm 2). first, the number of lines 156 needed (ie. length of word) for each selected word 157 1 here a word is defined as a token separated by whitespace. is calculated. next, we iterate through each word of the original text. if a word is a non-selected word, then it is simply added to the final text. if the word is a selected word, then only the character of the corresponding line is chosen. for example, if "happy" is selected, and the line number is 2, then "a" is added to the final text. for all lines that only consist of whitespace and the vertical characters, the required whitespace is calculated by the length of each non-selected word. finally, we add a width constraint to the algorithm for practicality. the transformation is only run on that width (number of words) at a time and all text is combined at the end. for example, if there are 100 words and the width constraint is 10, then only 10 are modified at a time. once the transformations are applied, the classifier is queried again to see if the transformed text causes the classifier to misclassify. if so, the final text is produced. if not, then the algorithm repeats, however, this time the words that have been selected already are removed as candidates during the selection step. ## experimental setup to test the effectiveness of vertattack, we evaluate the attack against several transformer classifiers across datasets examined in previous attack papers foot_0 @xcite @xcite . ## human study to investigate humans' understanding of vertattack's texts, we employed human crowdworkers to label a sampled set of texts which were perturbed by vertattack. specifically, we randomly sampled 100 of the 1000 texts from the rotten tomatoes dataset. we then asked crowdworkers to read the text and decide the sentiment of the text (positive or negative). for each text, we employed 3 crowdworkers 4 , and took the majority vote of the labels. it should be noted that no instructions to read the texts vertically were given. more information on the instructions can be found in appendix a. the confusion matrix of results is in humans were able to identify sentiment correctly, 77% of the time, far greater than the 7 -26% of the automated classifiers. this confirms that unlike the automated classifiers, humans are well prepared to read text in non-traditional manners. for comparison, we also ran the same study with on the original, unperturbed 100 texts. this is also in vertattack's texts have more to do with the difficulty of some of the texts rather than due to perturbation. to further investigate how vertattack performs in the adversarial text space, we compare to two other attacks, bert-attack (li et al., 2020) and textbugger (li et al., 2019) 5 . bert-attack is similar to vertattack as it is a word based attack. to select a word, bert-attack finds the importance score of a word by masking each word (one at a time) and comparing to the original logits. for replacement, bert-attack relies on bert to give suggestions via its mlm training. textbugger is a character based attack which tests inserting, deleting, swapping, or substituting characters. we run both attacks on the same 1000 examples from the rotten tomatoes dataset. the results can be seen in ## malicious use -offensive language to confirm the main results and demonstrate how vertattack may be used maliciously, we apply ver-tattack to "offensive" texts. we take a subset of olid's @xcite test set, labeled off (offensive). this results in 260 texts. we leveraged pretrained classifiers from huggingface 6 , trained on olid training data. we examine 3 variations of transformer models, bert, albert, and xlnet @xcite . the full results are in ## initial defenses we investigate some initial steps automated classifiers might take to mitigate vertattack's effectiveness. since vertattack introduces whitespace, simple solutions might be to reduce that whitespace. thus, we look at three different approaches. first, we simply remove extraneous whitespace and limit at most 1 space between each token, denoted as simple. second, we leverage a text segmentation library 9 to remove whitespace and re-combine words, denoted as segment. finally, we assume the classifier has learned the algorithm for vertattack and thus reverses it. that is, the classifier attempts to recombine vertical characters into words before classification. this is denoted as reverse. the full algorithm can be found in the appendix (appendix c). ## enhancing vertattack with chaff as demonstrated, if the classifier knows this type of attack is occurring, it can strongly mitigate it by reversing the algorithm. thus, we enhance ver-tattack by introducing chaff. specifically, rather than inserting only whitespace vertically, an alphabet character has a chance of being inserted. this occurs at a probability p. for example, if @xmath1, then there is a 10% probability that rather than whitespace, a character is inserted in the vertical lines. note that to preserve readability we do not allow this for whitespace next to perturbed words (nor original whitespace). the full algorithm can be found in appendix c. we test chaff for @xmath2, 10, 20, 30, 60}. the main results against the reverse algorithm (section 11.2) are in