SlowGuess/NAACL_2021_OCR huggingface dataset:
Phrase-level adversaries. In addition to generating more fluent sentences (Table 1), extracting the candidate perturbations directly from the translations does away with the need for sense disambiguation and increases the number of perturbations per example since it is not limited to a static dictionary. The increased effectiveness of BUMBLEBEE compared to POLYGLOSS (1.13x) is further evidence that a key factor to the success of such adversarial attacks is the availability of sufficient candidates; increasing the dimensionality of the search space increases the probability that an adversarial example for the model exists (Goodfellow et al., 2015). We also include a non-adversarial baseline (Rand.) by sampling candidates from a uniform distribution instead of searching for the worst-case perturbations. Our results in Table 3 indicate that the worst-case performance of multilingual models on code-mixed data may be much lower than the scores reported on human-produced test sets since they were not
Model	Devanagari	Transliterated (Latin)
XLM-Rlarge	61.35	41.97
XLM-Rbase	48.62	30.01
mBERTbase	37.70	23.41
Unicoderbase	49.34	30.00
Table 5: BUMBLEBEE results on XNLIen,hi using both Devanagari and Latin scripts. Lower is better.
created in a targeted, adversarial fashion. Experiments on beam width and a proof of concept for fully unsupervised adversaries are in Appendix E.
Transliteration. Since real-life code-mixers often use a single script for the entire sentence, we now test the effect of transliteration on BumbleBEE's success rate for the English  $^+$  Hindi language pair. We accomplish this by transliterating all candidates from Devanagari into Latin using the dictionaries released by Roark et al. (2020). From Table 5, we see that transliteration significantly affects the robustness of all models, even the XLM-based ones which were pretrained on similar data.
XQuAD. We observe that both XLM-R and mBERT are significantly challenged by BUMBLE-BEE even though only the question was modified (Table 4). We did not experiment on Unicoder to reduce carbon costs since its performance was almost identical to XLM-Rbase in our XNLI experiments.
POLYGLOSS or BUMBLEBEE? As expected, inspection of individual adversarial examples revealed that BUMEBLBEE generated more natural sentences than POLYGLOSS since the languages used within phrases were more consistent (Table 1). However, incorrect alignments due to the word aligner's probabilistic nature could introduce occasional noise into the adversarial examples. For example, we found "the" (en) to be often aligned with "的" (zh) even though the former is an article and the latter a possessive. We observe that the aligner performs better when the sentences have similar word orders (e.g., English-French vs. English-Chinese) and we can expect the adversaries generated in these settings to be more natural. Hence, we recommend POLYGLOSS when greater preservation of word-level semantics is desired, and BUMEBLBEE when phrase-level perturbations are desired or bilingual dictionaries are unavailable.
Discussion. K et al. (2020) noted significant performance drops in XNLI accuracy for mBERT when the premise and hypothesis were in differ
ent languages (Fake English vs. {Hindi, Russian, Spanish}), theorizing this to be an effect of disrupting the model's reliance on lexical overlap. Our experiments in §4 and §5 lend support to this hypothesis. In Table 1, we see multiple examples where the prediction was flipped from "contradiction" to "entailment" simply by perturbing a few words. If the models did not rely on lexical overlap but performed comparisons at the semantic level, such perturbations should not have severely impacted their performance. Our results on QA also corroborate Lee et al. (2019)'s finding that models trained on SQuAD-style datasets exploit lexical overlap between the question and context.
5 Code-Mixed Adversarial Training
Finally, we propose code-mixed adversarial training (CAT), an extension of the standard adversarial training paradigm (Goodfellow et al., 2015), to improve the robustness of multilingual models to adversarial polygons. In standard adversarial training, adversarial attacks are run on the training set to generate adversaries for training. However, this makes adversarial training computationally expensive. Hence, we take inspiration from Tan et al. (2020a)'s method of randomly sampling perturbations from an adversarial distribution and generate code-mixed perturbations using word alignment.
To generate the code-mixed adversarial training set  $X^{\prime}$ , we first compute the adversarial distribution  $\mathcal{P}_{adv}$  by enumerating the perturbations per embedded language in all successful adversaries (§4). Formally,  $\mathcal{P}_{adv} = \{f_i\}_{i=1\dots|\mathbb{L}|}$ , where  $f_i = \frac{l_i}{\sum_{j=1}^{|\mathbb{L}|} l_j}$  and  $\mathbb{L}$  is the set of embedded languages.
Next, for each clean example  $x$ , we sample  $n$  languages from  $\mathcal{P}_{adv}$  before translating the example into the  $n$  languages and aligning the translations with  $x$ . For sentence-pair classification tasks like NLI, we use a per-sentence  $n$  to further increase variation. Intuitively, limiting  $n$  improves the example's naturalness and the algorithm's efficiency (the alignment is the most costly step). We then extract phrases from the aligned sentences, yielding our candidate perturbations  $\mathbb{P}$ . Next, we sample a perturbation with probability  $\rho$  from  $\mathbb{P}$  for each phrase in  $x$ . Reducing  $\rho$  yields more natural sentences since they will be less perturbed. Finally, we apply these perturbations to  $x$ , obtaining a CAT example  $x'$ . Doing this  $k$  times for all  $x$  in  $X$  and adding the result to  $X$  yields  $X'$  (Alg. 3 in Appendix G).
In contrast to running the adversarial attack on
the training set, sampling perturbations from a distribution does not guarantee that the resulting example will be adversarial to the model. This issue can be mitigated by increasing the number of CAT examples observed during training. However, this would increase the computational cost if we were to train the model for the same number of epochs. Hence, we set  $k$  to one less than the number of epochs XLM-Rbase was fine-tuned for in §4 and train the model for one epoch on the adversarial training set. This exposes the model to more variation in the same number of training steps.
Setting. We conduct our experiments on NLI with XLM-R<sub>base</sub> with no loss of generality. In §4, the model was trained for ten epochs. Hence, we set  $k = 9$ ,  $n = 2$ ,  $\rho = 0.5$  for CAT and train all models for a similar number of steps (60k) with the same hyperparameters as §4. We first test the models on the BUMBLEBEE adversaries generated in §4 before directly attacking the model. Next, we construct more realistic settings by running BUMBLEBEE with only 1-2 embedded languages from standard XNLI, Swahili (sw), Hindi (hi), and Urdu (ur). These languages were the lowest resourced in the pretraining data (Conneau et al., 2020a).
We also construct another non-adversarial test set from XNLI by randomly choosing hypotheses and premises from different languages (K et al., 2020). Since the original examples are individually monolingual, this test set will reveal if a model is simply exploiting lexical overlap rather than comparing the underlying concepts.
Finally, we run BUMBLEBEE with embedded languages not seen during task-specific training and from a different family (Austronesian) from the XNLI languages, Filipino (tl) and Indonesian (id). This zero-shot defense setting will reveal if CAT encourages the learning of more language-invariant representations, or is simply allowing the model to adapt to the adversarial distribution.
Baselines. Since training on languages in the test set takes us out of the cross-lingual transfer setting, we train a translate-train- $n$  baseline for a fair comparison. In this setting, we train on every  $x$  and its translations in the  $n$  languages sampled in CAT, regardless of whether they contributed words to the final CAT examples. We also include Ganin et al. (2016)'s domain adversarial neural network (DANN), which has been used for cross-lingual adaptation (Joty et al., 2017; Chen et al., 2018).
Condition/Method	Clean	CleanDL	Adv$4	Advsw	Advhi+ur	AdvXNLI	Advtl	Advid+tl
Cross-lingual transfer (from §4)	74.06	66.09	3.53	38.54	29.12	3.53	36.96	24.83
Translate-train-n	77.25	72.01	29.44	50.53	40.63	7.04	44.37	33.23
DANN (Ganin et al., 2016)	51.86	35.10	33.45	16.02	17.52	6.54	12.05	7.06
Code-mixed adv. training (CAT)	77.10	75.46	50.21	58.58	48.20	12.63	49.14	38.16
Table 6: Results on standard XNLI with XLM-Rbase. Clean refers to the combined test set of all languages, CleanDL to the variant where the hypothesis and premise of each example are from different languages, Adv $\S_4$  to the BUMBLEBEE adversaries from  $\S_4$ , and Adv $\{\mathrm{Igs}\}$  to new adversaries from English + the subscripted languages. Higher is better.
From Table 6, we observe that both training on fully translated data and on CAT examples improved accuracy on the non-adversarial test sets and robustness to code-mixed adversaries, compared to the cross-lingual transfer model that was only trained on English data. Similar to K et al. (2020), we found that disrupting the models' reliance on lexical overlap  $(\mathrm{Clean}_{\mathrm{DL}})$  hurt performance. The drop was particularly significant for the cross-lingual transfer (8 points) and translate-train- $n$  models (5.24 points). On the other hand, our CAT model only suffered a 1.5-point drop, indicating that the former two models likely rely heavily on lexical overlap to make predictions, while our CAT model may be using "deeper", more language-agnostic features. Crucially, our CAT model achieves similar to better clean accuracy than the baselines, contrasting with prior work showing that adversarial training hurts clean accuracy (Tsipras et al., 2019). Finally, our CAT model is  $>1.7x$  more robust to adversaries constructed from all fifteen XNLI languages than the translate-train- $n$  model. Although DANN-type training improved robustness to the previous BUMBLEBEE adversaries, clean performance was significantly degraded and BUMBLEBEE was able to find even more damaging adversaries upon attacking the model directly.
When attacked with 1-2 embedded languages that were seen during training, CAT also yields significant improvements in robustness over the baselines: a  $>7$  point increase compared to translate-train- $n$  and a  $>19$  point gain over the zero-shot transfer setting. In the zero-shot defense setting, CAT shows a  $>12$ -point gain over the zero-shot transfer model and a  $>4.7$ -point gain over the translate-train- $n$  model. We believe these results to be due to CAT encouraging the learning of language-invariant representations by exposing the model to cross-lingual lexical variation and preventing the model from exploiting lexical overlaps.
(a) Cross-Linguual Transfer
(b) CAT
Figure 2: t-SNE visualizations of XLM- $\mathbf{R}_{\mathrm{base}}$  representations fine-tuned using different methods.
6 Seeing is Believing
To further understand the effect of various finetuning methods on XLM- $\mathbf{R}_{\mathrm{base}}$ , we visualize the  $
$  vector from the layer before the classification head using t-SNE (Linderman et al., 2019). Here, all sentences from XNLI are passed through the representations individually. If a representation were  $100\%$  language-invariant, we should expect t-SNE to be unable to separate individual languages into their own clusters. Hence, the extent to which t-SNE is able to do so would indicate the amount of language-specific information in this last layer.
From Fig. 2a, we observe that for the cross-lingual transfer model (§4), t-SNE managed to organize the sentences from several languages (Chinese, Hindi, Thai, Urdu) into distinct clusters. This indicates that a significant amount of language-specific information remains in the vector representations of sentences from these languages. Visualizing the sequence-averaged embeddings makes this even clearer (Fig. 5 in Appendix G). Hence, while XLM-R may be multilingual, it appears to be structured as a space of individual language subspaces as opposed to a mixed, or language-invariant space. On the other hand, t-SNE was much less successful when given the representation trained with CAT (Fig. 2b). Mixing multiple languages in the same sentence and showing the model multiple variants of the same sentence likely encourages the model to refine its representation such that all variants of the same sentence are represented similarly, re
sulting in a more language-invariant representation. T-SNE plots of the other models are in Appendix G.
We acknowledge that our methods do not fully model real code-mixing since we do not learn the mixing patterns from real data and there are subtleties in real code-mixing we ignore for simplicity, e.g., accounting for the prestige of participating languages (Bhatia, 2011). In addition, it is impossible to guarantee the semantic preservation of a sentence generated by BUMBLEBEE due to the word aligner's statistical nature, though we can expect more accurate alignments to improve semantic preservation. Finally, while CAT improves robustness, there remains a significant gap between the robust and clean accuracies. In line with recent work challenging the Anglocentricity of crosslingual models (Anastasopoulos and Neubig, 2020; Liang et al., 2020), a promising direction of future work lies in investigating how the choice of matrix language affects model robustness.
Ensuring that multilingual models are robust to both natural and adversarial code-mixing is important in today's increasingly multilingual world if they are to allow their target users to fully express themselves in human-machine conversations and to defend against adversarial users attempting to evade toxicity/misinformation detection systems.
To approximate a lower bound for model performance on lexical code-mixing, we propose two strong black-box multilingual adversarial attacks and demonstrate their effectiveness on state-of-the-art cross-lingual NLI and QA models. The former generates perturbations from bilingual dictionaries and disambiguates between senses using sentence translations, while the latter generates perturbations by aligning sentences from different languages.
Next, we show that training on code-mixed data synthesized via word alignment improves clean and robust accuracy when models are prevented from exploiting lexical overlap without hurting clean accuracy. Crucially, we achieve this in the same number of steps as standard supervised training.
Finally, we use t-SNE visualizations to show that multilingual models are not necessarily language-invariant and that our code-mixed adversarial training scheme encourages language-invariance.
9 Broader Impact / Ethical Considerations
Adversarial attacks and defenses are double-edged swords. On one hand, adversarial examples expose the gaps in existing models and help to focus the research community's attention on flaws that need to be addressed before these models can be used reliably in noisy, real-world environments. On the other, the same adversarial attacks can be used by malicious actors to bypass toxicity/misinformation detection systems. Similarly, methods for improving adversarial robustness can be used to defend against malicious actors and improve robustness to natural noise or linguistic variation, yet they can also be used to strengthen automated censorship systems and limit freedom of speech. For example, our adversarial attacks could be used both as a lower bound for model performance on naturally occurring code-mixed text and to bypass misinformation detection systems while preserving the message's intelligibility for multilingual speakers. Our adversarial training method could be used to both improve machine understanding of code-mixers by making multilingual representations more language-invariant and suppress the freedom of speech of polyglots who could have been using code-mixing to evade censorship.
At the same time, technology strongly shapes our behavior (Reeves et al., 2019). Consequently, given the centrality of code-switching/mixing to many polyglots' lived experiences (Duff, 2015) and the positive correlations between multilingualism, code-switching, and creativity (Leikin, 2013; Kharkhurin and Wei, 2015; Fürst and Grin, 2018), we should ensure that the natural language technologies we build do not inhibit multilingual speakers from fully expressing themselves, e.g., by discouraging code-mixing due to non-understanding. In addition, studies have found that aphasic polyglots code-mix more frequently than neurotypical polyglots to cope with word-retrieval difficulties (Goral et al., 2019), making it important for natural language technologies to be robust to code-mixing if they are to be inclusive. Therefore, we include both adversary generation and defense methods to avoid tipping the balance too far in either direction.
We would like to thank Cynthia Siew (NUS Psychology), Greg Bennett and Kathy Baxter (Salesforce), Lav Varshney (UIUC Electrical and Computer Engineering), Min-Yen Kan (NUS Com
puter Science), and our anonymous reviewers for their invaluable feedback. We are also grateful to Guangsen Wang, Mathieu Ravaut, Soujanya Lanka, and Tuyen Hoang for contributing manually code-mixed sentences and Bari M Saiful for pointers on replicating the XLM-R results. Samson is supported by Salesforce and Singapore's Economic Development Board under its Industrial Postgraduate Programme.
Ghazi M Abuhakema. 2013. Code switching and code mixing in arabic written advertisements: Patterns, aspects, and the question of prestige and standardisation. The Internet Journal Language, Culture and Society.
Eneko Agirre and Philip Edmonds. 2007. Word Sense Disambiguation: Algorithms and Applications, 1st edition. Springer Publishing Company, Incorporated.
Gustavo Aguilar and Thamar Solorio. 2020. From English to code-switching: Transfer learning with strong morphological clues. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8033-8044, Online. Association for Computational Linguistics.
Emily Ahn, Cecilia Jimenez, Yulia Tsvetkov, and Alan Black. 2020. What code-switching strategies are effective in dialogue systems? Proceedings of the Society for Computation in Linguistics, 3(1):308-318.
Fahad AlGhamdi and Mona Diab. 2019. Leveraging pretrained word embeddings for part-of-speech tagging of code switching data. In Proceedings of the Sixth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 99-109.
Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial examples. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2890–2896, Brussels, Belgium. Association for Computational Linguistics.
Antonios Anastasopoulos and Graham Neubig. 2020. Should all cross-lingual embeddings speak English? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8658-8679, Online. Association for Computational Linguistics.
Larissa Aronin and David Singleton. 2008. Multilingualism as a new linguistic dispensation. International Journal of Multilingualism, 5(1):1-16.
Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. On the cross-lingual transferability of monolingual representations. In Proceedings of the 58th
Annual Meeting of the Association for Computational Linguistics, pages 4623-4637, Online. Association for Computational Linguistics.
Mikel Artetxe and Holger Schwenk. 2019. Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond. Transactions of the Association for Computational Linguistics, 7:597-610.
Kalika Bali, Jatin Sharma, Monojit Choudhury, and Yogarshi Vyas. 2014. "i am borrowing ya mixing?" an analysis of english-hindi code mixing in facebook. In Proceedings of the First Workshop on Computational Approaches to Code Switching, pages 116-126.
Kelsey Ball and Dan Garrette. 2018. Part-of-speech tagging for code-switched, transliterated texts without explicit language identification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3084-3089, Brussels, Belgium. Association for Computational Linguistics.
M Saiful Bari, Tasnim Mohiuddin, and Shafiq Joty. 2020. Multimix: A robust data augmentation framework for cross-lingual nlp. arXiv preprint arXiv:2004.13240.
Utsab Barman, Amitava Das, Joachim Wagner, and Jennifer Foster. 2014. Code mixing: A challenge for language identification in the language of social media. In Proceedings of the first workshop on computational approaches to code switching, pages 13-23.
Anshul Bawa, Pranav Khadpe, Pratik Joshi, Kalika Bali, and Monojit Choudhury. 2020. Do multilingual users prefer chat-bots that code-mix? let's nudge and find out! Proceedings of the ACM on Human-Computer Interaction, 4(CSCW1):1-23.
Tej K Bhatia. 2011. The multilingual mind, optimization theory, and hinglish. Chutneying English: The Phenomenon of Hinglish, pages 37-52.
Matthias Blohm, Glorianna Jagfeld, Ekta Sood, Xiang Yu, and Ngoc Thang Vu. 2018. Comparing attention-based convolutional and recurrent neural networks: Success and limitations in machine reading comprehension. In Proceedings of the 22nd Conference on Computational Natural Language Learning, pages 108-118.
Khyathi Chandu, Ekaterina Loginova, Vishal Gupta, Josef van Genabith, Günter Neumann, Manoj Chinnakotla, Eric Nyberg, and Alan W Black. 2019. Code-mixed question answering challenge: Crowdsourcing data and techniques. In Third Workshop on Computational Approaches to Linguistic Code-Switching, pages 29-38. Association for Computational Linguistics (ACL).