Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots
Abstract
Multilingual models have demonstrated impressive cross-lingual transfer performance.
However, test sets like XNLI are monolingual at the example level.
In multilingual communities, it is common for polyglots to code-mix when conversing with each other.
Inspired by this phenomenon, we present two strong blackbox adversarial attacks (one word-level, one phrase-level) for multilingual models that push their ability to handle code-mixed sentences to the limit.
The former uses bilingual dictionaries to propose perturbations and translations of the clean example for sense disambiguation.
The latter directly aligns the clean example with its translations before extracting phrases as perturbations.
Our phrase-level attack has a success rate of 89.75% against XLM-R large , bringing its average accuracy of 79.85 down to 8.18 on XNLI.
Finally, we propose an efﬁcient adversarial training scheme that trains in the same number of steps as the original model and show that it improves model accuracy.
Introduction
The past year has seen incredible breakthroughs in cross-lingual generalization with the advent of massive multilingual models that aim to learn universal language representations (Pires et al., 2019; Wu and Dredze, 2019; Conneau et al., 2020b).
These models have demonstrated impressive cross-lingual transfer abilities: simply ﬁne-tuning them on task data from a high resource language such as English after pretraining on monolingual corpora was sufﬁcient to manifest such abilities.
This was observed even for languages with different scripts and no vocabulary overlap (K et al., 2020).
However, transferring from one language to another is insufﬁcient for NLP systems to understand multilingual speakers in an increasingly multilingual world (Aronin and Singleton, 2008).
In many multilingual societies (e.g., Singapore, Papua New
[Figure]
Guinea, etc.), it is common for multilingual interlocutors to produce sentences by mixing words, phrases, and even grammatical structures from the languages in their repertoires (Matras and Sakel, 2007).
This is known as code-mixing (Poplack et al., 1988), a phenomenon common in casual conversational environments such as social media and text messages.
2 Hence, it is crucial for NLP systems serving multilingual communities to be robust to code-mixing if they are to understand and establish rapport with their users (Tay, 1989; Bawa et al., 2020) or defend against adversarial polyglots.
Although gold standard data (Bali et al., 2014;
Patwa et al., 2020) is important for deﬁnitively evaluating code-mixed text processing ability, such datasets are expensive to collect and annotate.
The dizzying range of potential language combinations further compounds the immensity of such an effort.
We posit that performance on appropriately crafted adversaries could act as a lower bound of a model’s ability to generalize to the distribution simulated by said adversaries, an idea akin to worstcase analysis (Divekar, 1984).
For example, Tan et al.
(2020b) showed that an NLP system that was robust to morphological adversaries was less perplexed by dialectal text exhibiting morphological variation.
Likewise, if a system is robust to codemixed adversaries constructed from some set of languages, it is reasonable to expect it to also perform better on real code-mixed text in those languages.
While they may not fully model the intricacies of real code-mixing (Sridhar and Sridhar, 1980), we believe that they can be useful in the absence of appropriate evaluation data.
Hence, we:
• Propose two strong black-box adversarial attacks targeting the cross-lingual generalization ability of massive multilingual representations (Fig. 1), demonstrating their effectiveness on state-of-theart models for natural language inference and question answering.
To our knowledge, these are the ﬁrst two multilingual adversarial attacks.
• Propose an efﬁcient adversarial training scheme that takes the same number of steps as standard supervised training and show that it creates more language-invariant representations, improving accuracy in the absence of lexical overlap.
Related Work
Multilingual classiﬁers.
Low resource languages often lack support due to the high cost of annotating data for supervised learning.
An approach to tackle this challenge is to build cross-lingual representations that only need to be trained on task data from a high resource language to perform well on another under-resourced language (Klementiev et al., 2012).
Artetxe and Schwenk (2019) presented the ﬁrst general purpose multilingual representation using a BiLSTM encoder.
Following the success of Transformer models (Vaswani et al., 2017), recent multilingual models like mBERT (Devlin et al., 2019),
Unicoder (Huang et al., 2019), and XLM-R (Conneau et al., 2020a) take the pretraining → ﬁne-tuning paradigm into the multilingual realm by pretraining Transformer encoders on unlabeled monolingual corpora with various language modeling objectives before ﬁne-tuning them on task data from a high-resource language such as
English.
This is known as cross-lingual transfer.
Code-mixed text processing.
Previous research on code-mixed text processing focused on constructing formal grammars (Joshi, 1982) and tokenlevel language identiﬁcation (Bali et al., 2014;
Solorio et al., 2014; Barman et al., 2014), before progressing to named entity recognition and part-of-speech tagging (Ball and Garrette, 2018;
AlGhamdi and Diab, 2019; Aguilar and Solorio, 2020).
Recent work explores code-mixing in higher-level tasks such as question answering and task-oriented dialogue (Chandu et al., 2019;
Ahn et al., 2020).
Muller et al.
(2020) demonstrate mBERT’s ability to transfer to an unseen dialect by exploiting its speakers’ tendency to code-mix.
A key challenge of developing models that are robust to code-mixing is the availability of codemixed datasets.
Hence, Winata et al.
(2019) use a pointer-generator network to generate synthetically code-mixed sentences while Pratapa et al.
(2018) explore the use of parse trees for the same purpose.
Yang et al.
(2020) propose to improve machine translation with “code-switching pretraining”, replacing words with their translations in a similar manner to masked language modeling (Devlin et al., 2019).
These word pairs are constructed from monolingual corpora using cosine similarity.
Sitaram et al.
(2019) provide a comprehensive survey of code-mixed language processing.
Word-level adversaries.
Modiﬁed inputs aimed at disrupting a model’s predictions are known as adversarial examples (Szegedy et al., 2014).
In
NLP, perturbations can be applied at the character, subword, word, phrase, or sentence levels.
Early word-level adversarial attacks (Ebrahimi et al., 2018; Blohm et al., 2018) made use of the target model’s gradients to ﬂip individual words to trick the model into making the wrong prediction.
However, while the perturbations were adversarial for the target model, perturbed word’s original semantics was often not preserved.
This could result in the expected prediction changing and making the model appear more brittle than it actually is.
Later research addressed this by searching for adversarial rules (Ribeiro et al., 2018) or by constraining the candidate perturbations to the k nearest neighbors in the embedding space (Alzantot et al., 2018;
Michel et al., 2019;
Ren et al., 2019; Zhang et al., 2019; Li et al., 2019; Jin et al., 2020).
Zang et al.
(2020) take another approach by making use of a annotated sememes to disambiguate polysemous words, while Tan et al.
(2020a) perturb only
[Table]
the words’ morphology and encourage semantic preservation via a part-of-speech constraint.
Other approaches make use of language models to generate candidate perturbations (Garg and Ramakrishnan, 2020; Han et al., 2020).
Wallace et al.
ﬁnd phrases that act as universally adversarial perturbations when prepended to clean inputs.
Zhang et al.
(2020) provide a comprehensive survey.
Summary
Existing work on pretrained multilingual models has highlighted their impressive zeroshot cross-lingual transfer ability, though some analyses (K et al., 2020) indicate this could be a result of exploiting lexical overlaps rather than an indication of true cross-lingual understanding.
Existing evaluations for code-mixed text processing focus on gold annotated data, but such datasets are (relatively) expensive to compile and face similar scarcity challenges as those for low-resource languages.
Existing word-/phrase-level adversarial attacks probing the limits of model robustness have largely focused on monolingual (English) inputs.
In contrast, our adversarial attacks are designed to test the robustness of multilingual models to adversarial code-mixers.
Finally, we propose an efﬁcient adversarial training scheme to improve the robustness of said models to code-mixed adversaries.
Generating Multilingual Adversaries
Code-mixing is a phenomenon where a multilingual speaker mixes words, and even grammatical rules, from different languages in a single sentence.
This is distinguished from code-switching, which occurs at the inter-sentential level (Kachru, 1978).
Extreme code-mixing.
Inspired by the proliferation of real-life code-mixing and polyglots, we propose P OLY G LOSS and B UMBLEBEE , two multilingual adversarial attacks that adopt the persona of an adversarial code-mixer.
We focus on the lexical component of code-mixing, where some words in a sentence are substituted with their equivalents from another language in the interlocutor’s repertoire.
Borrowed words fall into two categories, nonce borrowing and loanwords, though distinguishing between them is beyond the scope of this work.
Since most code-mixers are bilinguals, natural code-mixed sentences tend to be constructed from two languages, with one language determining the syntax of the overall sentence (Poplack et al., 1988).
However, in a world with an increasing number of multilingual societies, it is conceivable for codemixing to occur between more than two languages (Tan, 1988).
We take this idea to the extreme to test multilingual representations for their robustness to such cross-lingual lexical variation.
Problem formulation.
Given a target multilingual model M , a clean example x with the label y , and a set of embedded languages L from which to borrow words, we aim to generate the adversarial example x (cid:48) that maximizes M ’s loss.
Formally, x (cid:48)
[Equation]
where x c ∈ X is a candidate adversary generated by perturbing x , M is a task-speciﬁc neural model, and L ( · ) is the model’s loss function.
3. P OLY G LOSS : Word-Level Adversaries
To obtain a code-mixed adversary, we ﬁrst generate candidate adversaries by substituting words in the clean example with their equivalents from another language.
These substitutions/perturbations can be generated by via machine translation or mined from bilingual dictionaries.
Following Myers-Scotton (1997), we will refer to the original example’s language as the matrix language and the perturbation’s language as the embedded language.
Next, we perform beam search on the candidates to ﬁnd the adversary that maximizes the target model’s loss in a black-box manner (Alg. 2 in Appendix B.1).
In our implementation, we also keep track of successful adversaries and return the ones with the highest and lowest losses.
The former is a stronger adversary, while the latter often has fewer perturbations.
More details are in Appendix B.1.
Orthographic preservation.
When the embedded language uses a different script from the matrix language, code-mixers tend to transliterate borrowed words into the same script (Abuhakema, 2013; Bali et al., 2014).
This still poses a significant challenge to multilingual models (Khanuja et al., 2020).
However, since there is often no standard way of transliterating words, we preserve the embedded language’s script where possible to avoid unfairly penalizing the target model.
Scalable sense disambiguation.
Due to the polysemous nature of many words, translating the right sense is crucial to preserving the word’s (and sentence’s) semantics.
Common word sense disambiguation methods (Agirre and Edmonds, 2007) use a sense tagger trained on an annotated sense inventory such as WordNet (Miller, 1995).
However, this approach requires individual taggers and sense inventories for each matrix and embedded language, making it a serious challenge to extend
P OLY G LOSS to low-resource languages.
Instead, we propose to ﬁlter candidate perturbations using the embedded language translation of the clean example.
This is easily done by checking if the candidate perturbation exists in the translation.
Since our examples tend to be single sentences, the probability of different senses of the same word occurring in a single sentence is generally low (Conneau et al., 2018; Popel et al., 2020).
This approach only requires a machine translation (MT) system and no extra linguistic information, making it highly scalable as long as a supervised (or unsupervised) machine translation system is available.
By using gold translations instead of machine translations, it is even possible to mostly guarantee semantic preservation at the word-level.
3. B UMBLEBEE : Phrase-Level Adversaries
Although using bilingual dictionaries with our ﬁltering method ensures that the semantics of a borrowed word matches the original, the dictionary’s comprehensiveness determines the presence of sufﬁcient candidate adversaries.
In addition, P OLYG LOSS swaps words at the word level, which may hurt the naturalness of the resulting sentence since
Require: Clean example-label pair ( x, y ) , Target Model M ,
Embedded languages L
Ensure: Adversarial example x (cid:48)
T ← T RANSLATE ( x, target-languages =
L ) L x ← G ET L OSS ( M , x
, y ) B ← { ( L x , x, 0) }
(cid:46) Initialize beam P ← A LIGN A ND E XTRACT P HRASES ( x, T ) while N OT E MPTY ( B ) do L x c , x c , i ← P OLL ( B ) C ← G ET C ANDIDATES ( x c , P
[ i ])
L ← G ET L OSS ( M , C, y ) (cid:46) Losses for C
i ← i +
U PDATE B EAM ( B, L , C, i ) end while x (cid:48) ← P OLL ( B ) return x (cid:48) Model XNLIXNLI- Clean
PG uf.
PG ﬁlt.
Clean PG ﬁlt. XLM-R large 81. XLM-R base 75.
mBERT base 67.
Unicoder base 74.
[Table]
it is more common for code-mixers to borrow phrases than individual words (Abuhakema, 2013).
A solution to these issues is to replace phrases in the matrix sentence with their equivalents from the reference translations instead of using a dictionary lookup (Alg. 1).
A key advantage of this approach is its ﬂexibility and scalability to more languages since it only requires parallel bitexts from the matrix and embedded languages.
With the advent of neural sequence-to-sequence models, such bitexts can be easily generated using publicly available
MT models.
However, a key challenge for this approach is extracting the matrix-embedded phrase pairs from the clean example and its translation.
We follow common phrase-based machine translation methods and accomplish this by aligning the matrix and embedded sentences (Koehn, 2010).
Implementation details can be found in Appendix B.2.
Syntactic preservation.
To improve the adversaries’ naturalness, we impose an equivalence constraint (Poplack, 1980), preventing a perturbation from being applied if it is from the same language as the previous word and will disrupt the syntax of the current phrase if applied (Winata et al., 2019).
Such disruptions usually occur when borrowing words from languages with a different word order.
[Table]
Experiments
We ﬁrst evaluate P OLY G LOSS and B UMBLEBEE on XNLI (Conneau et al., 2018), then evaluate the stronger attack on XQuAD (Artetxe et al., 2020).
XNLI is a multilingual dataset for natural language inference (NLI) with parallel translations for each example in ﬁfteen languages.
Each example comprises a premise, hypothesis, and a label with three possible classes: { contradiction, neutral, entailment } .
We construct two more datasets from XNLI:
XNLI-13 and XNLI-32.
XNLI-13 comprises all
XNLI languages except Swahili and Urdu due to the lack of suitable dictionaries for P OLY G LOSS .
We then translate the English test set into eighteen other languages with MT systems to form XNLI31, increasing the number of embedded languages
P OLY G LOSS can draw from.
XQuAD is a multilingual dataset for extractive question answering (QA) with parallel translations in eleven languages.
In the cross-lingual transfer setting, the models are trained on English data, MNLI (Williams et al., 2018) and SQuAD 1.1 (Rajpurkar et al., 2016), and tested on mulitlingual data, XNLI and XQuAD, respectively.
We perturb the premise and hypothesis for NLI and only the question for QA.
More experimental details can be found in Appendix D.
Matrix language.
Although our attacks work with any language as the matrix language, we use
English as the matrix language in our experiments due to the availability of English → T translation models and the prevalence of English as the matrix language in many code-mixing societies.
Models.
We conduct our experiments on three state-of-the-art massive multilingual encoder models: XLM-RoBERTa, mBERT, and Unicoder, each pretrained on more than 100 languages.
4. Results
From Tables 2 and 3, we observe that all the models are signiﬁcantly challenged by adversarial codemixing, though XLM-R large is the most robust to [Table] both attacks, likely due to having more parameters.
However, even after ﬁltering P OLY G LOSS ’s candidate perturbations by the gold translations in
XNLI-13, we observe an average drop in accuracy of 80.01%, relative to the models’ accuracy on the clean XNLI-13.
B UMBLEBEE induces even greater performance drops (average relative decrease of 90.96% on XNLI-13), likely due to its word aligner yielding more candidates than P OLY G LOSS ’s dictionary lookup.
Increasing the number of embedded languages P OLY G LOSS can draw upon results in greater drops in model performance (average relative decrease in accuracy of 93.66% on XNLI-31).
BERT- vs. XLM-based.
We notice that mBERT is more sensitive to intra-phrasal syntactic disruption than the XLM-based models.
mBERT is the most robust to B UMBLEBEE out of all the base models when the equivalence constraint is in place, yet is the least robust to P OLY G LOSS .
However, the latter trend is replicated for B UMBLEBEE if we remove this constraint (Table 14 in Appendix G).
A possible explanation is that XLM-R and Unicoder were trained on monolingual CommonCrawl (CC) data, while mBERT was trained on multilingual
Wikipedia, which could be considered as aligned at the article level since there are articles on the same topic in different languages.
Hence, it is possible that this helped to align the languages more accurately in the feature space but made it more sensitive to syntactic disruptions.
The higher performance of the XLM-based models on clean data can likely be attributed to the CC corpus being an order of magnitude larger (Lauscher et al., 2020).
Candidate ﬁltering.
In the unﬁltered setting, it is impossible for P OLY G LOSS to discriminate between valid and invalid senses for a given context.
Hence, a potential criticism is that the large difference in P OLY G LOSS ’s success rate between the ﬁltered and unﬁltered settings could be attributed to the inappropriate senses of polysemous words being chosen and disrupting the semantics of the sentence.
On the other hand, ﬁltering perturbations with reference translations of the sentence shrinks the space of perturbations to ∼ 1 per language.
Due to the dictionaries’ non-exhaustive nature, not ev- ery word in the matrix sentence has an entry in the dictionary to begin with, making this ﬁltering step a signiﬁcant reduction of the space of candidates.
To determine the likely cause of the accuracy difference between the ﬁltered and unﬁltered settings in XNLI-13, we increase the number of languages available to P OLY G LOSS to thirty-one.
If the difference between the ﬁltered and unﬁltered settings were not due to a lack of sufﬁcient candidates, we should observe only a minor difference between the ﬁltered settings for both XNLI-13 and -31.
However, we observe a 69% drop for XLM-R large , indicating that the former accuracy difference is likely due to the reduced number of valid candidates.
Phrase-level adversaries.
In addition to generating more ﬂuent sentences (Table 1), extracting the candidate perturbations directly from the translations does away with the need for sense disambiguation and increases the number of perturbations per example since it is not limited to a static dictionary.
The increased effectiveness of B UMBLEBEE compared to P OLY G LOSS (1.13x) is further evidence that a key factor to the success of such adversarial attacks is the availability of sufﬁcient candidates; increasing the dimensionality of the search space increases the probability that an adversarial example for the model exists (Goodfellow et al., 2015).
We also include a non-adversarial baseline (Rand.) by sampling candidates from a uniform distribution instead of searching for the worst-case perturbations.
Our results in Table 3 indicate that the worst-case performance of multilingual models on code-mixed data may be much lower than the scores reported on human-produced test sets since they were not created in a targeted, adversarial fashion.
Experiments on beam width and a proof of concept for fully unsupervised adversaries are in Appendix E.
XQuAD.
We observe that both XLM-R and mBERT are signiﬁcantly challenged by B UMBLEBEE even though only the question was modiﬁed (Table 4).
We did not experiment on Unicoder to reduce carbon costs since its performance was almost identical to XLM-R base in our XNLI experiments.
P OLY G LOSS or B UMBLEBEE ?
As expected, inspection of individual adversarial examples revealed that B UMBLEBEE generated more natural sentences than P OLY G LOSS since the languages used within phrases were more consistent (Table 1).
However, incorrect alignments due to the word aligner’s probabilistic nature could introduce occasional noise into the adversarial examples.
For example, we found “the” (en) to be often aligned with “ 的 ” (zh) even though the former is an article and the latter a possessive.
We observe that the aligner performs better when the sentences have similar word orders (e.g., English-French vs.
English-Chinese)
and we can expect the adversaries generated in these settings to be more natural.
Hence, we recommend P OLY G LOSS when greater preservation of word-level semantics is desired, and
B UMBLEBEE when phrase-level perturbations are desired or bilingual dictionaries are unavailable.
Discussion.
K et al.
(2020) noted signiﬁcant performance drops in XNLI accuracy for mBERT when the premise and hypothesis were in different languages, theorizing this to be an effect of disrupting the model’s reliance on lexical overlap.
Our experiments in §4 and §5 lend support to this hypothesis.
In Table 1, we see multiple examples where the prediction was ﬂipped from “contradiction” to “entailment” simply by perturbing a few words.
If the models did not rely on lexical overlap but performed comparisons at the semantic level, such perturbations should not have severely impacted their performance.
Our results on QA also corroborate Lee et al.
(2019)’s ﬁnding that models trained on SQuAD-style datasets exploit lexical overlap between the question and context.
Code-Mixed Adversarial Training
Finally, we propose code-mixed adversarial training (CAT), an extension of the standard adversarial training paradigm (Goodfellow et al., 2015), to improve the robustness of multilingual models to adversarial polyglots.
In standard adversarial training, adversarial attacks are run on the training set to generate adversaries for training.
However, this makes adversarial training computationally expensive.
Hence, we take inspiration from Tan et al.
(2020a)’s method of randomly sampling perturbations from an adversarial distribution and generate code-mixed perturbations using word alignment.
To generate the code-mixed adversarial training set X (cid:48) , we ﬁrst compute the adversarial distribution
P adv by enumerating the perturbations per embedded language in all successful adversaries (§4).
Formally, P adv =
{ f i } i =1 ...
| L | , where f i = [Equation] and L is the set of embedded languages.
Next, for each clean example x , we sample n languages from P adv before translating the example into the n languages and aligning the translations [Table] with x .
For sentence-pair classiﬁcation tasks like
NLI, we use a per-sentence n to further increase variation.
Intuitively, limiting n improves the example’s naturalness and the algorithm’s efﬁciency (the alignment is the most costly step).
We then extract phrases from the aligned sentences, yielding our candidate perturbations P .
Next, we sample a perturbation with probability ρ from P for each phrase in x .
Reducing ρ yields more natural sentences since they will be less perturbed.
Finally, we apply these perturbations to x , obtaining a CAT example x (cid:48) .
Doing this k times for all x in X and adding the result to X yields X (cid:48)
(Alg. 3 in Appendix G).
In contrast to running the adversarial attack on the training set, sampling perturbations from a distribution does not guarantee that the resulting example will be adversarial to the model.
This issue can be mitigated by increasing the number of CAT examples observed during training.
However, this would increase the computational cost if we were to train the model for the same number of epochs.
Hence, we set k to one less than the number of epochs XLM-R base was ﬁne-tuned for in §4 and train the model for one epoch on the adversarial training set.
This exposes the model to more variation in the same number of training steps.
Setting.
We conduct our experiments on NLI with XLM-R base with no loss of generality.
In §4, the model was trained for ten epochs.
Hence, we set k = 9 , n = 2 , ρ = 0 .
5 for CAT and train all models for a similar number of steps (60k) with the same hyperparameters as §4.
We ﬁrst test the models on the B UMBLEBEE adversaries generated in §4 before directly attacking the model.
Next, we construct more realistic settings by running B UMBLEBEE with only 1-2 embedded languages from standard XNLI, Swahili (sw), Hindi (hi), and Urdu (ur).
These languages were the lowest resourced in the pretraining data (Conneau et al., 2020a).
We also construct another non-adversarial test set from XNLI by randomly choosing hypotheses and premises from different languages (K et al., 2020).
Since the original examples are individually monolingual, this test set will reveal if a model is simply exploiting lexical overlap rather than comparing the underlying concepts.
Finally, we run B UMBLEBEE with embedded languages not seen during task-speciﬁc training and from a different family (Austronesian) from the XNLI languages, Filipino (tl) and Indonesian (id).
This zero-shot defense setting will reveal if
CAT encourages the learning of more languageinvariant representations, or is simply allowing the model to adapt to the adversarial distribution.
Baselines.
Since training on languages in the test set takes us out of the cross-lingual transfer setting, we train a translate-train- n baseline for a fair comparison.
In this setting, we train on every x and its translations in the n languages sampled in
CAT, regardless of whether they contributed words to the ﬁnal CAT examples.
We also include Ganin et al.
(2016)’s domain adversarial neural network (DANN), which has been used for cross-lingual adaptation (Joty et al., 2017; Chen et al., 2018).
5. Results
From Table 5, we observe that both training on fully translated data and on CAT examples improved accuracy on the non-adversarial test sets and robustness to code-mixed adversaries, compared to the cross-lingual transfer model that was only trained on English data.
Similar to K et al.
, we found that disrupting the models’ reliance on lexical overlap (Clean DL ) hurt performance.
The drop was particularly signiﬁcant for the cross-lingual transfer (8 points) and translate-train- n models (5.24 points).
On the other hand, our CAT model only suffered a 1.5-point drop, indicating that the former two models likely rely heavily on lexical overlap to make predictions, while our CAT model may be using “deeper”, more language-agnostic features.
Crucially, our CAT model achieves similar to better clean accuracy than the baselines, contrasting with prior work showing that adversarial training hurts clean accuracy (Tsipras et al., 2019).
Finally, our CAT model is > 1.7x more robust to [Figure] adversaries constructed from all ﬁfteen XNLI languages than the translate-train- n model.
Although
DANN-type training improved robustness to the previous B UMBLEBEE adversaries, clean performance was signiﬁcantly degraded and B UMBLEBEE was able to ﬁnd even more damaging adversaries upon attacking the model directly.
When attacked with 1-2 embedded languages that were seen during training, CAT also yields signiﬁcant improvements in robustness over the baselines: a > 7 point increase compared to translatetrain- n and a > 19 point gain over the zero-shot transfer setting.
In the zero-shot defense setting,
CAT shows a > 12-point gain over the zero-shot transfer model and a > 4.7-point gain over the translate-train- n model.
We believe these results to be due to CAT encouraging the learning of language-invariant representations by exposing the model to cross-lingual lexical variation and preventing the model from exploiting lexical overlaps.
Seeing is Believing
To further understand the effect of various ﬁnetuning methods on XLM-R base , we visualize the 
 vector from the layer before the classiﬁcation head using t-SNE (Linderman et al., 2019).
Here, all sentences from XNLI are passed through the representations individually.
If a representation were 100% language-invariant, we should expect tSNE to be unable to separate individual languages into their own clusters.
Hence, the extent to which t-SNE is able to do so would indicate the amount of language-speciﬁc information in this last layer.
From Fig.
2a, we observe that for the crosslingual transfer model (§4), t-SNE managed to organize the sentences from several languages (Chinese, Hindi, Thai, Urdu) into distinct clusters.
This indicates that a signiﬁcant amount of languagespeciﬁc information remains in the vector representations of sentences from these languages.
Visualizing the sequence-averaged embeddings makes this even clearer (Fig. 5 in Appendix G).
Hence, while
XLM-R may be multilingual, it appears to be structured as a space of individual language subspaces as opposed to a mixed, or language-invariant space.
On the other hand, t-SNE was much less successful when given the representation trained with CAT (Fig. 2b).
Mixing multiple languages in the same sentence and showing the model multiple variants of the same sentence likely encourages the model to reﬁne its representation such that all variants of the same sentence are represented similarly, resulting in a more language-invariant representation.
T-SNE plots of the other models are in Appendix G.
Limitations and Future Work
We acknowledge that our methods do not fully model real code-mixing since we do not learn the mixing patterns from real data and there are subtleties in real code-mixing we ignore for simplicity, e.g., accounting for the prestige of participating languages (Bhatia, 2011).
In addition, it is impossible to guarantee the semantic preservation of a sentence generated by B UMBLEBEE due to the word aligner’s statistical nature, though we can expect more accurate alignments to improve semantic preservation.
Finally, while CAT improves robustness, there remains a signiﬁcant gap between the robust and clean accuracies.
In line with recent work challenging the Anglocentricity of crosslingual models (Anastasopoulos and Neubig, 2020;
Liang et al., 2020), a promising direction of future work lies in investigating how the choice of matrix language affects model robustness.
Conclusion
Ensuring that multilingual models are robust to both natural and adversarial code-mixing is important in today’s increasingly multilingual world if they are to allow their target users to fully express themselves in human-machine conversations and to defend against adversarial users attempting to evade toxicity/misinformation detection systems.
To approximate a lower bound for model performance on lexical code-mixing, we propose two strong black-box multilingual adversarial attacks and demonstrate their effectiveness on state-of-theart cross-lingual NLI and QA models.
The former generates perturbations from bilingual dictionaries and disambiguates between senses using sentence translations, while the latter generates perturbations by aligning sentences from different languages.
Next, we show that training on code-mixed data synthesized via word alignment improves clean and robust accuracy when models are prevented from exploiting lexical overlap without hurting clean accuracy.
Crucially, we achieve this in the same number of steps as standard supervised training.
Finally, we use t-SNE visualizations to show that multilingual models are not necessarily languageinvariant and that our code-mixed adversarial training scheme encourages language-invariance.
9 Broader Impact / Ethical Considerations
Adversarial attacks and defenses are double-edged swords.
On one hand, adversarial examples expose the gaps in existing models and help to focus the research community’s attention on ﬂaws that need to be addressed before these models can be used reliably in noisy, real-world environments.
On the other, the same adversarial attacks can be used by malicious actors to bypass toxicity/misinformation detection systems.
Similarly, methods for improving adversarial robustness can be used to defend against malicious actors and improve robustness to natural noise or linguistic variation, yet they can also be used to strengthen automated censorship systems and limit freedom of speech.
For example, our adversarial attacks could be used both as a lower bound for model performance on naturally occurring code-mixed text and to bypass misinformation detection systems while preserving the message’s intelligibility for multilingual speakers.
Our adversarial training method could be used to both improve machine understanding of code-mixers by making multilingual representations more language-invariant and suppress the freedom of speech of polyglots who could have been using code-mixing to evade censorship.
At the same time, technology strongly shapes our behavior (Reeves et al., 2019).
Consequently, given the centrality of code-switching/mixing to many polyglots’ lived experiences (Duff, 2015) and the positive correlations between multilingualism, code-switching, and creativity (Leikin, 2013;
Kharkhurin and Wei, 2015; F¨urst and Grin, 2018), we should ensure that the natural language technologies we build do not inhibit multilingual speakers from fully expressing themselves, e.g., by discouraging code-mixing due to non-understanding.
In addition, studies have found that aphasic polyglots code-mix more frequently than neurotypical polyglots to cope with word-retrieval difﬁculties (Goral et al., 2019), making it important for natural language technologies to be robust to code-mixing if they are to be inclusive.
Therefore, we include both adversary generation and defense methods to avoid tipping the balance too far in either direction.
Acknowledgments
We would like to thank Cynthia Siew (NUS Psychology), Greg Bennett and Kathy Baxter (Salesforce), Lav Varshney (UIUC Electrical and Computer Engineering), Min-Yen Kan (NUS Computer Science), and our anonymous reviewers for their invaluable feedback.
We are also grateful to
Guangsen Wang, Mathieu Ravaut, Soujanya Lanka, and Tuyen Hoang for contributing manually codemixed sentences and Bari M Saiful for pointers on replicating the XLM-R results. Samson is supported by Salesforce and Singapore’s Economic Development Board under its Industrial Postgraduate Programme. A Examples of Real Code-Mixed Text
English + Spanish (Sridhar and Sridhar, 1980)
• The man que vino ayer (who came yesterday) wants ito buyun carro nuevo (a new car).
• El (The) old man esta enojide (is mad).
• Me lleve chile ya roasted y peeled ... para hacerlo ells.
(I picked up the chile already roasted and peeled for making it there.)
Hindi + English (Bali et al., 2014)
• Beﬁtting reply to mere papa ne maaraa (My father gave a beﬁtting reply) • ... and the party workers [will] come with me without virodh (protest/objection)
Sarnami + Sranan + Dutch (Yakpo, 2015)
• dus gewoon calat jaiye, tab ego kerki links ki rechts.
(So just keep on walking, then [there’s] achurch, left or right.)
• kaun w ´ al ´ a damr ´ u, ego haig ´ a jaun men n ´ a verﬁ bhail, ma ego wel hai.
(Which [kind of] damru drum, there’s one which is not coloured inside but one actually is.)
B Attack Implementation Details B. P OLY G LOSS
Algorithm 2 P OLY G LOSS
Require: Clean example-label pair ( x, y ) , Target Model M ,
Embedded languages L
Ensure: Adversarial example x (cid:48)
[Table]
beam while
[Table] i ) end while x (cid:48) ← P OLL ( B ) return x (cid:48)
To reduce the practical running time of our attack, we make use of cross-lingual dictionaries released by Lample et al.
(2018) for generating candidate perturbations instead of translating the words in an online fashion.
We also use the gold translations of the clean examples when they are available (such as in XNLI), and use the models released by Tiedemann and Thottingal (2020) in the transformers library (Wolf et al., 2019) to translate the examples to other languages.
We also cache them in hashtables for fast retrieval.
B. B UMBLEBEE
We use the gold translations where available and
Tiedemann and Thottingal (2020)’s translation models for the other languages, and align sentences with a neural word aligner (Sabet et al., 2020) backed by XLM-R base in our implementation of
B UMBLEBEE .
Although Sabet et al.
(2020) found the “Itermax” algorithm to yield the best performance for their experimental settings, we suggest using the high recall (“Match”)
algorithm for candidate generation.
We inspected the output of both algorithms and found that while Itermax generates more candidates, it also tends to generate noisier alignments compared to Match, which we found to be more conservative.
C P OLY G LOSS and CAT Samples
Almost random samples for P
OLY
G LOSS and CAT (we tried to include the sentences with Thai and
Hindi characters but did not manage to get them to render with pdﬂatex).
[Table] The
[Table] included the most basic things. He [Table]
But
[Table] [Table] [Table] [Table]
being
[Table] by ranching [Table] that crept into Western songs du north of us, cayuse, for example.
We should nie think of human equality when we consider social and political justice.
Not much has sich innerhalb des Jahrzehnts ver¨andert.
Ese (cid:160)A(cid:149) el most historic weather (cid:157)w§ in la historia registrada para el el clima severo en el country
Como ustedes γν ω ρίζετε , last enero hemos issued un νέο όγκο de reports, la Performance και λογοδοσίας serie de, en los que se esbozan los de gesti´on desaf´ıos που αντιμετ ω πίζουν οι nuestras mayores federal agencies and the substantial opportunities para mejorar su rendimiento.
because a lot trong số họ are are similar um-hum bueno that’s incre ´ ıble como i used to cuando i was в college sol´ıa la have el stereo en all през time or i ten´ıa en
MTV or something but ever que i’ve been out de college
D Experiment Details D. Datasets
Standard XNLI 3 comprises 7,500 parallel examples (2,490 dev.
, 5,010 test) in ﬁfteen languages:
English (en), Spanish (es), German (de), Greek (el),
Russian (ru), Turkish (tr), Arabic (ar), Vietnamese (vi), Thai (th), Chinese (zh), Hindi (hi), French (fr),
Bulgarian (bg), Swahili (sw), and Urdu (ur).
The labels are uniformly distributed between the three classes (contradiction, neutral, entailment).
The machine-translated training set 4 of standard
XNLI comprises the MNLI training examples in addition to their translations in the same fourteen non-English languages as the dev.
and test sets.
XNLI-13 comprises all standard XNLI languages except Swahili and Urdu due to the lack of suitable dictionaries for P OLY G LOSS .
XNLI31 comprises all languages in XNLI-13, in addition to another eighteen: Afrikaans (af), Albanian (sq), Catalan (ca), Czech (cs), Danish (da), Dutch (nl), Estonian (et), Filipino (tl), Finnish (ﬁ), Hebrew (he), Hungarian (hu), Indonesian (id), Italian (it), Macedonian (mk), Romanian (ro), Slovak (sk),
Swedish (sv), and Ukrainian (uk).
MNLI 5 comprises 392,702 examples in English with the following label distribution: 130,899 entailment, 130,900 neutral, 130,903 contradiction.
XQuAD 6 comprises 1,190 question-answer pairs with guaranteed answers in eleven languages:
English (en), Spanish (es), German (de), Greek (el), Russian (ru), Turkish (tr), Arabic (ar), Vietnamese (vi), Thai (th), Chinese (zh), and Hindi (hi).
XQuAD examples are drawn from the SQuAD 1. development set.
SQuAD 1.1 7 comprises 87,599 question-answer pairs with guaranteed answers in English.
D. Metrics
The metric used for XNLI is simple accuracy:
[Equation]
The metrics used for SQuAD are F 1 :
[Equation]
and exact match:
Exact Match =
# ˆ y =
y
# total (4)
D. XQuAD Preprocessing
We use Phatthiyaphaibun et al.
(2016) to tokenize
Thai text, jieba 8 for Chinese text, and split all other languages on whitespace.
D. Training Details
[Table] [Table]
Hyperparameters.
Table 8 contains the hyperparameters we used to ﬁne-tune our models on
MNLI.
We used the hyperparameters suggested by
Devlin et al.
(2019) 9 for mBERT, the hyperparameters suggested by Bari et al.
(2020) for the XLM-R models, and the hyperparameters from Liang et al.
(2020) for Unicoder.
Table 9 contains the hyperparameters we used to ﬁne-tune our models on SQuAD 1.1.
We used the default SQuAD hyperparameters from the transformers library 10 for XLM-R base and adjusted the hyperparameters for XLM-R large to ﬁt it onto the GPU.
The mBERT model from the HuggingFace model repository 11 was used instead of ﬁne-tuning our own.
The clean scores reported in
Table 4 are similar to those reported in the XQuAD
GitHub repository.
Validation Performance for CAT Exps.
[Table]
D. Infrastructure Details
Models were trained on single V100 GPUs.
Attacks were run on 8 V100 GPUs, parallelized with ray 12 to make full use of GPU memory.
On the standard XNLI test set (15 languages, 5,010 examples), B UMBLEBEE takes 60-90 minutes in total.
With 1 embedded language, B UMBLEBEE runs on the test set in under 10 minutes.
P OLY G LOSS is generally much faster (under 30 minutes on XNLI31) due to not needing a neural aligner.
E Extra B UMBLEBEE Experiments
[Table]
the number of embedded languages on the attack success rate (%) while holding the other variable constant at 1.
We use Swahili, Swahili and French, and Swahili, French, and Spanish as the embedded languages when v = 1 , 2 , 3 , respectively.
Rates are computed relative to the average clean XNLI score on the languages involved.
Beam search.
In our experiments, we found that increasing the beam width yielded a higher attack success rate.
However, this increases running time with only minor improvements (Table 11).
We found increasing the number of embedded languages (and hence candidates) to be a much more efﬁcient method of increasing the success rate with only a relatively minor increase in running time.
Although the time complexity (in number of model queries) is O ( | B || C ||
L || S | ) where | S | is the sentence length, increasing | L | had a greater impact on the success rate than increasing | B | by the same number.
[Table]
Fully unsupervised adversaries.
A potential drawback of B UMBLEBEE is that it requires translations of the clean example, which may be challenging to obtain for low-resource languages.
However, it is possible to use unsupervised MT models for this purpose.
We use Song et al.
(2019)’s unsupervised English-French model to generate translations as a proof of concept and ﬁnd that they achieve similar results (Table 12).
F Plausible Language Combinations
To explore the effect of different combinations of languages on a multilingual model’s performance, we run B UMBLEBEE with different sets of embedded languages that could be plausibly spoken by (adversarial) polyglots around the world.
English is used as the matrix language for all experiments.
We observe a general trend of decreasing model accuracy as we increase the number of mixed languages (Table 13), and XLM-R base ’s robustness to an embedded language appears to be generally more dependent on the size of its pretraining dataset than language typology.
For example, Russian ( en+ru ) and Indonesian ( en+id ), languages with notably high accuracies, were the two most resourced languages after English in the corpus used for pretraining (Conneau et al., 2020a), while Swahili and Filipino were among the lowerresourced languages.
A notable outlier is Afrikaans ( en+af ), which was the lowest resourced language in our experiments yet XLM-R base was quite robust to adversaries constructed from it and English.
A possible explanation is that Afrikaans’ language family, Indo-European, is highly represented in the pretraining corpus.
Another notable outlier is
Vietnamese, which was the fourth most resourced language in the pretraining corpus, yet the model was more vulnerable to adversaries constructed from English and Vietnamese than adversaries constructed from English and Filipino, one of the lowest resourced languages.
A possible explanation for this is the use of Latin characters combined with little vocabulary overlap between English and
Vietnamese, and differing adjective positions.
[Table]
sizes reproduced from Conneau et al.
(2020a).
We also ﬁnd XLM-R base to be twice as robust in the en+da+de+nl+sv condition as the en+id+th+tl+vi condition.
It is likely that the structural similarity of the mixed languages also plays an important role in determining the model robustness, reinforcing K et al.
(2020)’s similar ﬁndings on cross-lingual transfer.
Investigating the effects of typology, vocabulary and orthographic overlap, pretraining corpus size, and interaction effects between different sets of languages on a model’s robustness to code-mixed adversaries could lead to new insights into how different languages interact in the multilingual embedding space and we leave this to future work.
G More Tables, Figures, and Algorithms
Algorithm
Require: Original examples X , Embedded languages L ,
Num. perturbed examples k , Adversarial distribution P adv , Max.
langs.
per example n , Phrase perturbation prob.
ρ
Ensure: Adversarial training set X (cid:48)
X (cid:48)
← { ∅ } for x in X do S ← S AMPLE L ANGUAGES ( L , n, P adv )
T ← T RANSLATE ( x, target-languages =
S ) P ← A LIGN A ND E XTRACT P HRASES ( x, T ) for i = 1 to k do x (cid:48) ← P ERTURB ( x, P , ρ )
X (cid:48) ← X (cid:48) ∪ { x (cid:48) } end for end for return X ∪ X (cid:48)
Section continues on
Model
Clean
PG unﬁlt.
PG
[Table]
XLM-R large
[Table]
[Figure] [Figure]