Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots (2021)

Abstract: Multilingual models have demonstrated impressive cross-lingual transfer performance. However, test sets like XNLI are monolingual at the example level. In multilingual communities, it is common for polyglots to code-mix when conversing with each other. Inspired by this phenomenon, we present two strong black-box adversarial attacks (one word-level, one phrase-level) for multilingual models that push their ability to handle code-mixed sentences to the limit. The former uses bilingual dictionaries to propose perturbations and translations of the clean example for sense disambiguation. The latter directly aligns the clean example with its translations before extracting phrases as perturbations. Our phrase-level attack has a success rate of 89.75% against XLM-R-large, bringing its average accuracy of 79.85 down to 8.18 on XNLI. Finally, we propose an efficient adversarial training scheme that trains in the same number of steps as the original model and show that it creates more language-invariant representations, improving clean and robust accuracy in the absence of lexical overlap without degrading performance on the original examples.

## introduction the past year has seen incredible breakthroughs in cross-lingual generalization with the advent of massive multilingual models that aim to learn universal language representations @xcite @xcite . these models have demonstrated impressive cross-lingual transfer abilities: simply fine-tuning them on task data from a high resource language such as english after pretraining on monolingual corpora was sufficient to manifest such abilities. this was observed even for languages with different scripts and no vocabulary overlap @xcite . however, transferring from one language to another is insufficient for nlp systems to understand multilingual speakers in an increasingly multilingual world @xcite . in many multilingual societies (e.g., singapore, papua new guinea, etc.), it is common for multilingual interlocutors to produce sentences by mixing words, phrases, and even grammatical structures from the languages in their repertoires @xcite . this is known as code-mixing @xcite , a phenomenon common in casual conversational environments such as social media and text messages. 2 hence, it is crucial for nlp systems serving multilingual communities to be robust to code-mixing if they are to understand and establish rapport with their users @xcite or defend against adversarial polyglots. although gold standard data @xcite is important for definitively evaluating code-mixed text processing ability, such datasets are expensive to collect and annotate. the dizzying range of potential language combinations further compounds the immensity of such an effort. we posit that performance on appropriately crafted adversaries could act as a lower bound of a model's ability to generalize to the distribution simulated by said adversaries, an idea akin to worstcase analysis @xcite . for example, @xcite showed that an nlp system that was robust to morphological adversaries was less perplexed by dialectal text exhibiting morphological variation. likewise, if a system is robust to codemixed adversaries constructed from some set of languages, it is reasonable to expect it to also perform better on real code-mixed text in those languages. while they may not fully model the intricacies of real code-mixing @xcite , we believe that they can be useful in the absence of appropriate evaluation data. hence, we: code-mixed text processing. previous research on code-mixed text processing focused on constructing formal grammars @xcite and tokenlevel language identification @xcite @xcite , before progressing to named entity recognition and part-of-speech tagging @xcite @xcite . recent work explores code-mixing in higher-level tasks such as question answering and task-oriented dialogue @xcite . muller et al. (2020) demonstrate mbert's ability to transfer to an unseen dialect by exploiting its speakers' tendency to code-mix. a key challenge of developing models that are robust to code-mixing is the availability of codemixed datasets. hence, winata et al. (2019) use a pointer-generator network to generate synthetically code-mixed sentences while pratapa et al. ( 2018 ) explore the use of parse trees for the same purpose. @xcite propose to improve machine translation with "code-switching pretraining", replacing words with their translations in a similar manner to masked language modeling @xcite . these word pairs are constructed from monolingual corpora using cosine similarity. @xcite provide a comprehensive survey of code-mixed language processing. word-level adversaries. modified inputs aimed at disrupting a model's predictions are known as adversarial examples @xcite . in nlp, perturbations can be applied at the character, subword, word, phrase, or sentence levels. early word-level adversarial attacks @xcite made use of the target model's gradients to flip individual words to trick the model into making the wrong prediction. however, while the perturbations were adversarial for the target model, perturbed word's original semantics was often not preserved. this could result in the expected prediction changing and making the model appear more brittle than it actually is. later research addressed this by searching for adversarial rules @xcite or by constraining the candidate perturbations to the k nearest neighbors in the embedding space @xcite @xcite @xcite . zang summary. existing work on pretrained multilingual models has highlighted their impressive zeroshot cross-lingual transfer ability, though some analyses @xcite indicate this could be a result of exploiting lexical overlaps rather than an indication of true cross-lingual understanding. although language-agnosticity is commonly measured via cross-lingual retrieval tasks such as lareqa @xcite and similarity search @xcite , we offer a different perspective in this paper by operationalizing it as a model's ability to handle code-mixing. existing evaluations for code-mixed text processing focus on gold annotated data, but such datasets are (relatively) expensive to compile and face similar scarcity challenges as those for low-resource languages. existing word-/phrase-level adversarial attacks probing the limits of model robustness have largely focused on monolingual (english) inputs. in contrast, our adversarial attacks are designed to test the robustness of multilingual models to adversarial code-mixers. finally, we propose an efficient adversarial training scheme to improve the robustness of said models to code-mixed adversaries. ## generating multilingual adversaries code-mixing is a phenomenon where a multilingual speaker mixes words, and even grammatical rules, from different languages in a single sentence. this is distinguished from code-switching, which occurs at the inter-sentential level @xcite . extreme code-mixing. inspired by the proliferation of real-life code-mixing and polyglots, we propose polygloss and bumblebee, two multilingual adversarial attacks that adopt the persona of an adversarial code-mixer. we focus on the lexical component of code-mixing, where some words in a sentence are substituted with their equivalents from another language in the interlocutor's repertoire. borrowed words fall into two categories, nonce borrowing and loanwords, though distinguishing between them is beyond the scope of this work. since most code-mixers are bilinguals, natural code-mixed sentences tend to be constructed from two languages, with one language determining the syntax of the overall sentence @xcite . however, in a world with an increasing number of multilingual societies, it is conceivable for codemixing to occur between more than two languages @xcite . we take this idea to the extreme to test multilingual representations for their robustness to such cross-lingual lexical variation. problem formulation. given a target multilingual model m, a clean example x with the label y, and a set of embedded languages l from which to borrow words, we aim to generate the adversarial example x that maximizes m's loss. formally, ## experiments we first evaluate polygloss and bumblebee on xnli @xcite , then evaluate the stronger attack on xquad @xcite . xnli is a multilingual dataset for natural language inference (nli) with parallel translations for each example in fifteen languages. each example comprises a premise, hypothesis, and a label with three possible classes: {contradiction, neutral, entail-ment}. we construct two more datasets from xnli: xnli-13 and xnli-32. xnli-13 comprises all xnli languages except swahili and urdu due to the lack of suitable dictionaries for polygloss. we then translate the english test set into eighteen other languages with mt systems to form xnli-31, increasing the number of embedded languages polygloss can draw from. xquad is a multilingual dataset for extractive question answering (qa) with parallel translations in eleven languages. in the cross-lingual transfer setting, the models are trained on english data, mnli @xcite and squad 1.1 @xcite , and tested on mulitlingual data, xnli and xquad, respectively. we perturb the premise and hypothesis for nli and only the question for qa. more experimental details can be found in appendix d. ## code-mixed adversarial training finally, we propose code-mixed adversarial training (cat), an extension of the standard adversarial training paradigm @xcite , to improve the robustness of multilingual models to adversarial polyglots. in standard adversarial training, adversarial attacks are run on the training set to generate adversaries for training. however, this makes adversarial training computationally expensive. hence, we take inspiration from tan et al. to generate the code-mixed adversarial training set x , we first compute the adversarial distribution p adv by enumerating the perturbations per embedded language in all successful adversaries ( §4). formally, p @xmath0...|l| , where next, for each clean example x, we sample n languages from p adv before translating the example into the n languages and aligning the translations with x. for sentence-pair classification tasks like nli, we use a per-sentence n to further increase variation. intuitively, limiting n improves the example's naturalness and the algorithm's efficiency (the alignment is the most costly step). we then extract phrases from the aligned sentences, yielding our candidate perturbations p. next, we sample a perturbation with probability ρ from p for each phrase in x. reducing ρ yields more natural sentences since they will be less perturbed. finally, we apply these perturbations to x, obtaining a cat example x . doing this k times for all x in x and adding the result to x yields x (alg. 3 in appendix g). in contrast to running the adversarial attack on the training set, sampling perturbations from a distribution does not guarantee that the resulting example will be adversarial to the model. this issue can be mitigated by increasing the number of cat examples observed during training. however, this would increase the computational cost if we were to train the model for the same number of epochs. hence, we set k to one less than the number of epochs xlm-r base was fine-tuned for in §4 and train the model for one epoch on the adversarial training set. this exposes the model to more variation in the same number of training steps. setting. we conduct our experiments on nli with xlm-r base with no loss of generality. in §4, the model was trained for ten epochs. hence, we set @xmath1, @xmath2, ρ = 0.5 for cat and train all models for a similar number of steps (60k) with the same hyperparameters as §4. we first test the models on the bumblebee adversaries generated in §4 before directly attacking the model. next, we construct more realistic settings by running bum-blebee with only 1-2 embedded languages from standard xnli, swahili (sw), hindi (hi), and urdu (ur). these languages were the lowest resourced in the pretraining data @xcite . we also construct another non-adversarial test set from xnli by randomly choosing hypotheses and premises from different languages @xcite . since the original examples are individually monolingual, this test set will reveal if a model is simply exploiting lexical overlap rather than comparing the underlying concepts. finally, we run bumblebee with embedded languages not seen during task-specific training and from a different family (austronesian) from the xnli languages, filipino (tl) and indonesian (id). this zero-shot defense setting will reveal if cat encourages the learning of more languageinvariant representations, or is simply allowing the model to adapt to the adversarial distribution. baselines. since training on languages in the test set takes us out of the cross-lingual transfer setting, we train a translate-train-n baseline for a fair comparison. in this setting, we train on every x and its translations in the n languages sampled in cat, regardless of whether they contributed words to the final cat examples. we also include @xcite 's domain adversarial neural network (dann), which has been used for cross-lingual adaptation @xcite . ## seeing is believing to further understand the effect of various finetuning methods on xlm-r base , we visualize @xmath3. here, all sentences from xnli are passed through the representations individually. if a representation were 100% language-invariant, we should expect t-sne to be unable to separate individual languages into their own clusters. hence, the extent to which t-sne is able to do so would indicate the amount of language-specific information in this last layer. from fig. 2a , we observe that for the crosslingual transfer model ( §4), t-sne managed to organize the sentences from several languages (chinese, hindi, thai, urdu) into distinct clusters. this indicates that a significant amount of languagespecific information remains in the vector representations of sentences from these languages. visualizing the sequence-averaged embeddings makes this even clearer (fig. 5 in appendix g). hence, while xlm-r may be multilingual, it appears to be structured as a space of individual language subspaces as opposed to a mixed, or language-invariant space. on the other hand, t-sne was much less successful when given the representation trained with cat (fig. 2b ). mixing multiple languages in the same sentence and showing the model multiple variants of the same sentence likely encourages the model to refine its representation such that all variants of the same sentence are represented similarly, re-sulting in a more language-invariant representation. t-sne plots of the other models are in appendix g. ## limitations and future work we acknowledge that our methods do not fully model real code-mixing since we do not learn the mixing patterns from real data and there are subtleties in real code-mixing we ignore for simplicity, e.g., accounting for the prestige of participating languages @xcite . in addition, it is impossible to guarantee the semantic preservation of a sentence generated by bumblebee due to the word aligner's statistical nature, though we can expect more accurate alignments to improve semantic preservation. finally, while cat improves robustness, there remains a significant gap between the robust and clean accuracies. in line with recent work challenging the anglocentricity of crosslingual models @xcite , a promising direction of future work lies in investigating how the choice of matrix language affects model robustness. ## conclusion ensuring that multilingual models are robust to both natural and adversarial code-mixing is important in today's increasingly multilingual world if they are to allow their target users to fully express themselves in human-machine conversations and to defend against adversarial users attempting to evade toxicity/misinformation detection systems. to approximate a lower bound for model performance on lexical code-mixing, we propose two strong black-box multilingual adversarial attacks and demonstrate their effectiveness on state-of-theart cross-lingual nli and qa models. the former generates perturbations from bilingual dictionaries and disambiguates between senses using sentence translations, while the latter generates perturbations by aligning sentences from different languages. next, we show that training on code-mixed data synthesized via word alignment improves clean and robust accuracy when models are prevented from exploiting lexical overlap without hurting clean accuracy. crucially, we achieve this in the same number of steps as standard supervised training. finally, we use t-sne visualizations to show that multilingual models are not necessarily languageinvariant and that our code-mixed adversarial training scheme encourages language-invariance.