SlowGuess/NAACL_2021_OCR huggingface dataset:
Samson Tan\* Shafiq Joty\*
$^{\S}$ Salesforce Research Asia National University of Singapore Nanyang Technological University
{samson.tan,sjoty}@salesforce.com
Multilingual models have demonstrated impressive cross-lingual transfer performance. However, test sets like XNLI are monolingual at the example level. In multilingual communities, it is common for polyglots to code-mix when conversing with each other. Inspired by this phenomenon, we present two strong black-box adversarial attacks (one word-level, one phrase-level) for multilingual models that push their ability to handle code-mixed sentences to the limit. The former uses bilingual dictionaries to propose perturbations and translations of the clean example for sense disambiguation. The latter directly aligns the clean example with its translations before extracting phrases as perturbations. Our phrase-level attack has a success rate of  $89.75\%$  against  $\mathrm{XLM - R_{large}}$ , bringing its average accuracy of 79.85 down to 8.18 on XNLI. Finally, we propose an efficient adversarial training scheme that trains in the same number of steps as the original model and show that it improves model accuracy. $^1$
The past year has seen incredible breakthroughs in cross-lingual generalization with the advent of massive multilingual models that aim to learn universal language representations (Pires et al., 2019; Wu and Dredze, 2019; Conneau et al., 2020b). These models have demonstrated impressive cross-lingual transfer abilities: simply fine-tuning them on task data from a high resource language such as English after pretraining on monolingual corpora was sufficient to manifest such abilities. This was observed even for languages with different scripts and no vocabulary overlap (K et al., 2020).
However, transferring from one language to another is insufficient for NLP systems to understand multilingual speakers in an increasingly multilingual world (Aronin and Singleton, 2008). In many
(a) Aligned words across sentences
(b) Extracted candidate perturbations
我不知道 what aku supposed to use it for.
(c) Final multilingual adversary
Figure 1: BUMBLEBEE's three key stages of adversary generation: (a) Align words in the matrix (English) and embedded sentences (top: Indonesian, bottom: Chinese); (b) Extract candidate perturbations from embedded sentences; (c) Construct final adversary by maximizing the target model's loss.
multilingual societies (e.g., Singapore, Papua New Guinea, etc.), it is common for multilingual interlocutors to produce sentences by mixing words, phrases, and even grammatical structures from the languages in their repertoires (Matras and Sakel, 2007). This is known as code-mixing (Poplack et al., 1988), a phenomenon common in casual conversational environments such as social media and text messages.² Hence, it is crucial for NLP systems serving multilingual communities to be robust to code-mixing if they are to understand and establish rapport with their users (Tay, 1989; Bawa et al., 2020) or defend against adversarial polyglots.
Although gold standard data (Bali et al., 2014; Patwa et al., 2020) is important for definitively evaluating code-mixed text processing ability, such datasets are expensive to collect and annotate. The
$^{1}$ Code: github.com/salesforce/adversarial-polyglots
Examples of real code-mixing in Appendix A.
dizzying range of potential language combinations further compounds the immensity of such an effort.
We posit that performance on appropriately crafted adversaries could act as a lower bound of a model's ability to generalize to the distribution simulated by said adversaries, an idea akin to worst-case analysis (Divekar, 1984). For example, Tan et al. (2020b) showed that an NLP system that was robust to morphological adversaries was less perplexed by dialectal text exhibiting morphological variation. Likewise, if a system is robust to code-mixed adversaries constructed from some set of languages, it is reasonable to expect it to also perform better on real code-mixed text in those languages. While they may not fully model the intricacies of real code-mixing (Sridhar and Sridhar, 1980), we believe that they can be useful in the absence of appropriate evaluation data. Hence, we:
- Propose two strong black-box adversarial attacks targeting the cross-lingual generalization ability of massive multilingual representations (Fig. 1), demonstrating their effectiveness on state-of-the-art models for natural language inference and question answering. To our knowledge, these are the first two multilingual adversarial attacks.
- Propose an efficient adversarial training scheme that takes the same number of steps as standard supervised training and show that it creates more language-invariant representations, improving accuracy in the absence of lexical overlap.
Multilingual classifiers. Low resource languages often lack support due to the high cost of annotating data for supervised learning. An approach to tackle this challenge is to build cross-lingual representations that only need to be trained on task data from a high resource language to perform well on another under-resourced language (Klementiev et al., 2012). Artetxe and Schwenk (2019) presented the first general purpose multilingual representation using a BiLSTM encoder. Following the success of Transformer models (Vaswani et al., 2017), recent multilingual models like mBERT (Devlin et al., 2019), Unicoder (Huang et al., 2019), and XLM-R (Conneau et al., 2020a) take the pretraining  $\rightarrow$  fine-tuning paradigm into the multilingual realm by pretraining Transformer encoders on unlabeled monolingual corpora with various language modeling objectives before fine-tuning them
on task data from a high-resource language such as English. This is known as cross-lingual transfer.
Code-mixed text processing. Previous research on code-mixed text processing focused on constructing formal grammars (Joshi, 1982) and token-level language identification (Bali et al., 2014; Solorio et al., 2014; Barman et al., 2014), before progressing to named entity recognition and part-of-speech tagging (Ball and Garrette, 2018; AlGhamdi and Diab, 2019; Aguilar and Solorio, 2020). Recent work explores code-mixing in higher-level tasks such as question answering and task-oriented dialogue (Chandu et al., 2019; Ahn et al., 2020). Muller et al. (2020) demonstrate mBERT's ability to transfer to an unseen dialect by exploiting its speakers' tendency to code-mix.
A key challenge of developing models that are robust to code-mixing is the availability of code-mixed datasets. Hence, Winata et al. (2019) use a pointer-generator network to generate synthetically code-mixed sentences while Pratapa et al. (2018) explore the use of parse trees for the same purpose.
Yang et al. (2020) propose to improve machine translation with "code-switching pretraining", replacing words with their translations in a similar manner to masked language modeling (Devlin et al., 2019). These word pairs are constructed from monolingual corpora using cosine similarity. Sitaram et al. (2019) provide a comprehensive survey of code-mixed language processing.
Word-level adversaries. Modified inputs aimed at disrupting a model's predictions are known as adversarial examples (Szegedy et al., 2014). In NLP, perturbations can be applied at the character, subword, word, phrase, or sentence levels.
Early word-level adversarial attacks (Ebrahimi et al., 2018; Blohm et al., 2018) made use of the target model's gradients to flip individual words to trick the model into making the wrong prediction. However, while the perturbations were adversarial for the target model, perturbed word's original semantics was often not preserved. This could result in the expected prediction changing and making the model appear more brittle than it actually is.
Later research addressed this by searching for adversarial rules (Ribeiro et al., 2018) or by constraining the candidate perturbations to the  $k$  nearest neighbors in the embedding space (Alzantot et al., 2018; Michel et al., 2019; Ren et al., 2019; Zhang et al., 2019; Li et al., 2019; Jin et al., 2020). Zang
Original
Adversary
Prediction	P: The girl that can help me is all the way across town. H: There is no one who can help me.
P: alan girl that can help me is all the way across town. H: one who can help me.
Before: Contradiction After: Entailment
Original
Adversary
Prediction	P: We didn't know where they were going. H: We didn't know where the people were traveling to.
P: We didn't know where they were going. H: We didn't know where lesgens allaient.
Before: Entailment After: Neutral
Original
Adversary
Prediction	P: Well it got to where there's two or three aircraft arrive in a week and I didn't know where they're flying to.
H: There are never any aircraft arriving.
P: ⌒ΠΕΜ, ΠΟΙΙΟ Σο mahali there's two or three aircraft arrive in a week and I didn't know where they're flying to.
H:从来没有 aircraft arriving.
Prediction	Before: Contradiction After: Entailment
Table 1: BUMBLEBEE adversaries found for XLM-R on XNLI (P: Premise; H: Hypothesis).
et al. (2020) take another approach by making use of a annotated sememes to disambiguate polysemous words, while Tan et al. (2020a) perturb only the words' morphology and encourage semantic preservation via a part-of-speech constraint. Other approaches make use of language models to generate candidate perturbations (Garg and Ramakrishnan, 2020; Han et al., 2020). Wallace et al. (2019) find phrases that act as universally adversarial perturbations when pretended to clean inputs. Zhang et al. (2020) provide a comprehensive survey.
Summary. Existing work on pretrained multilingual models has highlighted their impressive zero-shot cross-lingual transfer ability, though some analyses (K et al., 2020) indicate this could be a result of exploiting lexical overlaps rather than an indication of true cross-lingual understanding. Although language-agnosticity is commonly measured via cross-lingual retrieval tasks such as LAReQA (Roy et al., 2020) and similarity search (Artetxe and Schwenk, 2019), we offer a different perspective in this paper by operationalizing it as a model's ability to handle code-mixing. Existing evaluations for code-mixed text processing focus on gold annotated data, but such datasets are (relatively) expensive to compile and face similar scarcity challenges as those for low-resource languages. Existing word-/phrase-level adversarial attacks probing the limits of model robustness have largely focused on monolingual (English) inputs. In contrast, our adversarial attacks are designed to test the robustness of multilingual models to adversarial code-mixers. Finally, we propose an efficient adversarial training scheme to improve the robustness of said models to code-mixed adversaries.
3 Generating Multilingual Adversaries
Code-mixing is a phenomenon where a multilingual speaker mixes words, and even grammatical
rules, from different languages in a single sentence. This is distinguished from code-switching, which occurs at the inter-sentential level (Kachru, 1978).
Extreme code-mixing. Inspired by the proliferation of real-life code-mixing and polyglots, we propose POLYGLOSS and BUMBLEBEE, two multilingual adversarial attacks that adopt the persona of an adversarial code-mixer. We focus on the lexical component of code-mixing, where some words in a sentence are substituted with their equivalents from another language in the interlocutor's repertoire. Borrowed words fall into two categories, none borrowing and loanwords, though distinguishing between them is beyond the scope of this work.
Since most code-mixers are bilinguals, natural code-mixed sentences tend to be constructed from two languages, with one language determining the syntax of the overall sentence (Poplack et al., 1988). However, in a world with an increasing number of multilingual societies, it is conceivable for codemixing to occur between more than two languages (Tan, 1988). We take this idea to the extreme to test multilingual representations for their robustness to such cross-lingual lexical variation.
Problem formulation. Given a target multilingual model  $\mathcal{M}$ , a clean example  $x$  with the label  $y$ , and a set of embedded languages  $\mathbb{L}$  from which to borrow words, we aim to generate the adversarial example  $x'$  that maximizes  $\mathcal{M}$ 's loss. Formally,
$$
x ^ {\prime} = \underset {x _ {c} \in X} {\arg \max } \mathcal {L} (y, \mathcal {M} \left(x _ {c}\right)), \tag {1}
$$
where  $x_{c}\in X$  is a candidate adversary generated by perturbing  $x$ ,  $\mathcal{M}$  is a task-specific neural model, and  $\mathcal{L}(\cdot)$  is the model's loss function.
3.1 POLYGLOSS: Word-Level Adversaries
To obtain a code-mixed adversary, we first generate candidate adversaries by substituting words in the
clean example with their equivalents from another language. These substitutions/perturbations can be generated by via machine translation or mined from bilingual dictionaries. Following Myers-Scotton (1997), we will refer to the original example's language as the matrix language and the perturbation's language as the embedded language.
Next, we perform beam search on the candidates to find the adversary that maximizes the target model's loss in a black-box manner (Alg. 2 in Appendix B.1). In our implementation, we also keep track of successful adversaries and return the ones with the highest and lowest losses. The former is a stronger adversary, while the latter often has fewer perturbations. More details are in Appendix B.1.
Orthographic preservation. When the embedded language uses a different script from the matrix language, code-mixers tend to transliterate borrowed words into the same script (Abuhakema, 2013; Bali et al., 2014). This still poses a significant challenge to multilingual models (Khanuja et al., 2020). We generally preserve the embedded language's script where possible to avoid unfairly penalizing the target model since there is often no standard way of transliterating words.
Scalable sense disambiguation. Due to the polysemous nature of many words, translating the right sense is crucial to preserving the word's (and sentence's) semantics. Common word sense disambiguation methods (Agirre and Edmonds, 2007) use a sense tagger trained on an annotated sense inventory such as WordNet (Miller, 1995). However, this approach requires individual taggers and sense inventories for each matrix and embedded language, making it a serious challenge to extend POLYGLOSS to low-resource languages.
Instead, we propose to filter candidate perturbations using the embedded language translation of the clean example. This is easily done by checking if the candidate perturbation exists in the translation. Since our examples tend to be single sentences, the probability of different senses of the same word occurring in a single sentence is generally low (Conneau et al., 2018; Popel et al., 2020). This approach only requires a machine translation (MT) system and no extra linguistic information, making it highly scalable as long as a supervised (or unsupervised) machine translation system is available. By using gold translations instead of machine translations, it is even possible to mostly
Algorithm 1 BUMBLEBEE
Require: Clean example-label pair  $(x,y)$ , Target Model  $\mathcal{M}$ , Embedded languages  $\mathbb{L}$
Ensure: Adversarial example  $x'$
$$
\begin{array}{l} T \leftarrow \text {T R A N S L A T E} (x, \text {t a r g e t - l a n g u a g e s} = \mathbb {L}) \\ \mathcal {L} _ {x} \leftarrow \operatorname {G E T L O S S} (\mathcal {M}, x, y) \\ B \leftarrow \left\{\left(\mathcal {L} _ {x}, x, 0\right) \right\} \quad \triangleright \text {I n i t i l i z e b e a m} \\ \mathbb {P} \leftarrow \text {A L I G N A N D E X T R A C T P H R A S E S} (x, T) \\ w h i l e \quad N O T E M P T Y (B) d o \\ \mathcal {L} _ {x _ {c}}, x _ {c}, i \leftarrow \operatorname {P O L L} (B) \\ C \leftarrow \operatorname {G E T C A N D I D A T E S} \left(x _ {c}, \mathbb {P} [ i ]\right) \\ \mathcal {L} \leftarrow \operatorname {G E T L O S S} (\mathcal {M}, C, y) \\ i \leftarrow i + 1 \\ \operatorname {U P D A T E B E A M} (B, \mathcal {L}, C, i) \\ e n d \quad w h i l e \\ x ^ {\prime} \leftarrow \operatorname {P o l l} (B) \\ \text {r e t u r n} x ^ {\prime} \\ \end{array}
$$
guarantee semantic preservation at the word-level.
3.2 BUMBLEBEE: Phrase-Level Adversaries
Although using bilingual dictionaries with our filtering method ensures that the semantics of a borrowed word matches the original, the dictionary's comprehensiveness determines the presence of sufficient candidate adversaries. In addition, POLY-GLOSS swaps words at the word level, which may hurt the naturalness of the resulting sentence since it is more common for code-mixers to borrow phrases than individual words (Abuhakema, 2013).
A solution to these issues is to replace phrases in the matrix sentence with their equivalents from the reference translations instead of using a dictionary lookup (Alg. 1). A key advantage of this approach is its flexibility and scalability to more languages since it only requires parallel bitexts from the matrix and embedded languages. With the advent of neural sequence-to-sequence models, such bitexts can be easily generated using publicly available MT models. However, a key challenge for this approach is extracting the matrix-embedded phrase pairs from the clean example and its translation. We follow common phrase-based machine translation methods and accomplish this by aligning the matrix and embedded sentences (Koehn, 2010). Implementation details can be found in Appendix B.2.
Syntactic preservation. To improve the adversaries' naturalness, we impose an equivalence constraint (Poplack, 1980), preventing a perturbation from being applied if it is from the same language as the previous word and will disrupt the syntax of the current phrase if applied (Winata et al., 2019). Such disruptions usually occur when borrowing words from languages with a different word order.
Model	XNLI-13	XNLI-31
Clean	PGuf.	PGfilt.	Clean	PGfilt.
XLM-Rlarge	81.10	6.06	28.28	80.60	8.76
XLM-Rbase	75.42	2.17	12.27	74.75	3.57
mBERTbase	67.54	2.15	9.24	66.56	3.11
Unicoderbase	74.98	1.99	11.33	74.28	3.73
Table 2: POLYGLOSS (PG) results (accuracy) on XNLI-13 and -31 test sets with beam width  $= 1$ . PG{filt,uf.} indicates whether the candidate perturbations were filtered using reference translations. Clean accuracy scores are the averages across all languages in the test set. Lower is better.
Model	XNLI-13	Standard XNLI
Clean	BB	Clean	Rand.	BB
XLM-Rlarge	81.10	11.31	79.85	75.04	8.18
XLM-Rbase	75.42	5.08	74.06	65.19	3.53
mBERTbase	67.54	6.10	65.66	59.17	4.45
Unicoderbase	74.98	4.81	73.69	65.55	3.61
We first evaluate POLYGLOSS and BUMBLEBEE on XNLI (Conneau et al., 2018), then evaluate the stronger attack on XQuAD (Artetxe et al., 2020). XNLI is a multilingual dataset for natural language inference (NLI) with parallel translations for each example in fifteen languages. Each example comprises a premise, hypothesis, and a label with three possible classes: {contradiction, neutral, entailment}. We construct two more datasets from XNLI: XNLI-13 and XNLI-32. XNLI-13 comprises all XNLI languages except Swahili and Urdu due to the lack of suitable dictionaries for POLYGLOSS. We then translate the English test set into eighteen other languages with MT systems to form XNLI-31, increasing the number of embedded languages POLYGLOSS can draw from. XQuAD is a multilingual dataset for extractive question answering (QA) with parallel translations in eleven languages. In the cross-lingual transfer setting, the models are trained on English data, MNLI (Williams et al., 2018) and SQuAD 1.1 (Rajpurkar et al., 2016), and tested on multilingual data, XNLI and XQuAD, respectively. We perturb the premise and hypothesis for NLI and only the question for QA. More experimental details can be found in Appendix D.
Matrix language. Although our attacks work with any language as the matrix language, we use English as the matrix language in our experiments
Table 3: BUMBLEBEE (BB) results on XNLI with beam width  $= 1$ . Clean accuracies are the averages across all languages in each test set. We include a random (Rand.) baseline by randomly (rather than adversarially) perturbing sentences and report the average across 5 seeds. Lower is better.
Model	Clean	BUMBLEBEE
XLM-Rlarge	75.64 / 61.39	35.32 / 22.52
XLM-Rbase	68.90 / 53.50	17.95 / 10.33
mBERTbase	64.66 / 49.47	20.66 / 11.68
Table 4: BUMBLEBEE results on XQuAD (F1/EM).
due to the availability of English  $\rightarrow$  T translation models and the prevalence of English as the matrix language in many code-mixing societies.
Models. We conduct our experiments on three state-of-the-art massive multilingual encoder models: XLM-RoBERTa, mBERT, and Unicoder, each pretrained on more than 100 languages.
From Tables 2 and 3, we observe that all the models are significantly challenged by adversarial code-mixing, though XLM- $\mathbf{R}_{\mathrm{large}}$  is the most robust to both attacks, likely due to having more parameters. However, even after filtering POLYGLOSS's candidate perturbations by the gold translations in XNLI-13, we observe an average drop in accuracy of  $80.01\%$ , relative to the models' accuracy on the clean XNLI-13. BUMBLEBEE induces even greater performance drops (average relative decrease of  $90.96\%$  on XNLI-13), likely due to its word aligner yielding more candidates than POLYGLOSS's dictionary lookup. Increasing the number of embedded languages POLYGLOSS can draw upon results in greater drops in model performance (average relative decrease in accuracy of  $93.66\%$  on XNLI-31).
BERT- vs. XLM-based. We notice that mBERT is more sensitive to intra-phrasal syntactic disruption than the XLM-based models. mBERT is the most robust to BUMBLEBEE out of all the base models when the equivalence constraint is in place, yet is the least robust to POLYGLOSS. However, the latter trend is replicated for BUMBLEBEE if we remove this constraint (Table 16 in Appendix G). A possible explanation is that XLM-R and Unicoder were trained on monolingual CommonCrawl (CC) data, while mBERT was trained on multilingual Wikipedia, which could be considered as aligned at the article level since there are articles on the same topic in different languages. Hence, it is possible that this helped to align the languages more accurately in the feature space but made it more sensitive to syntactic disruptions. However, many other hyperparameters differ between the two that could have also influenced their robustness. Hence, we leave a rigorous study of these factors to future
work. The higher performance of the XLM-based models on clean data can likely be attributed to the CC corpus being an order of magnitude larger than multilingual Wikipedia (Lauscher et al., 2020).
Candidate filtering. In the unfiltered setting, it is impossible for POLYGLOSS to discriminate between valid and invalid senses for a given context. Hence, a potential criticism is that the large difference in POLYGLOSS's success rate between the filtered and unfiltered settings could be attributed to the inappropriate senses of polysemous words being chosen and disrupting the semantics of the sentence. On the other hand, filtering perturbations with reference translations of the sentence shrinks the space of perturbations to  $\sim 1$  per language. Due to the dictionaries' non-exhaustive nature, not every word in the matrix sentence has an entry in the dictionary to begin with, making this filtering step a significant reduction of the space of candidates.
To determine the likely cause of the accuracy difference between the filtered and unfiltered settings in XNLI-13, we increase the number of languages available to POLYGLOSS to thirty-one. If the difference between the filtered and unfiltered settings were not due to a lack of sufficient candidates, we should observe only a minor difference between the filtered settings for both XNLI-13 and -31. However, we observe a  $69\%$  drop for  $\mathrm{XLM - R_{large}}$ , indicating that the former accuracy difference is likely due to the reduced number of valid candidates.