SlowGuess/NAACL_2021_OCR huggingface dataset:
Table 10 contains the hyperparameters we used to fine-tune our models on SQuAD 1.1. We used the default SQuAD hyperparameters from the transformers library for XLM-Rbase and adjusted the hyperparameters for XLM-Rlarge to fit it onto the GPU. The mBERT model from the HuggingFace model repository was used instead of fine-tuning our own. The clean scores reported in Table 4 are similar to those reported in the XQuAD GitHub repository.
cims.nyu.edu/\~sbowman/xnli
${}^{4}$  Same link as XNLI dev/test set.
cims.nyu.edu/ sbowman/multinli/
$^{6}$ github.com/deepmind/xquad
7rajpurkar.github.io/../train-v1.1.json
$^{8}$  github.com/fxsjy/jieba
<sup>9</sup>https://github.com/google-research/../multilingual.md
github.com/huggingface/../question-answering
$^{11}$ huggingface.co/salti/bert-base-multilingual-casedfinetuned-squad
D.5 Validation Performance for CAT Exps.
Model	Clean	Adv§4-dev
Cross-lingual transfer (from §4)	73.90	4.17
Translate-train-n	76.99	29.27
DANN (Ganin et al., 2016)	51.81	34.89
Code-mixed adv. training (CAT)	77.13	52.32
Table 11: Accuracy on the XNLI dev. set and BUMBLEBEE adversaries generated from the dev. set.
D.6 Infrastructure Details
Models were trained on single V100 GPUs. Attacks were run on 8 V100 GPUs, parallelized with ray $^{12}$  to make full use of GPU memory. On the standard XNLI test set (15 languages, 5,010 examples), BUMBLEBEE takes 60-90 minutes in total. With 1 embedded language, BUMBLEBEE runs on the test set in under 10 minutes. POLYGLOSS is generally much faster (under 30 minutes on XNLI-31) due to not needing a neural aligner.
E Extra BUMBLEBEE Experiments
Variable (v)	v = 1	v = 2	v = 3
Beam width	48.52%	48.95%	49.67%
Embedded lgs.	48.52%	69.09%	75.73%
Table 12: Effect of increasing the beam width vs. the number of embedded languages on the attack success rate  $(\%)$  while holding the other variable constant at 1. We use Swahili, Swahili and French, and Swahili, French, and Spanish as the embedded languages when  $v = 1,2,3$  , respectively. Rates are computed relative to the average clean XNLI score on the languages involved.
Beam search. In our experiments, we found that increasing the beam width yielded a higher attack success rate. However, this increases running time with only minor improvements (Table 12). We found increasing the number of embedded languages (and hence candidates) to be a more efficient method of increasing the success rate with a minor increase in running time. Although the time complexity (in number of model queries) is  $O(|B||C||\mathbb{L}||S|)$  where  $|S|$  is the sentence length, increasing  $|\mathbb{L}|$  had a greater impact on the success rate than increasing  $|B|$  by the same number.
Model	Clean	Supervised	Unsupervised
XLM-Rbase	81.32	49.00	46.78
Table 13: Accuracy after running BUMBLEBEE on XNLI in supervised and unsupervised settings (matrix: English, embedded: French).
Fully unsupervised adversaries. A potential drawback of BumbleBee is that it requires translations of the clean example, which may be challenging to obtain for low-resource languages. However, it is possible to use unsupervised MT models for this purpose. We use Song et al. (2019)'s unsupervised English-French model to generate translations as a proof of concept and find that they achieve similar results (Table 13).
F Plausible Language Combinations
To explore the effect of different combinations of languages on a multilingual model's performance, we run BUMBLEBEE with different sets of embedded languages that could be plausibly spoken by (adversarial) polyglots around the world. English is used as the matrix language for all experiments.
We observe a general trend of decreasing model accuracy as we increase the number of mixed languages (Table 14), and XLM- $\mathbf{R}_{\mathrm{base}}$ 's robustness to an embedded language appears to be generally more dependent on the size of its pretraining dataset than language typology. For example, Russian  $(en + ru)$  and Indonesian  $(en + id)$ , languages with notably high accuracies, were the two most resourced languages after English in the corpus used for pretraining (Conneau et al., 2020a), while Swahili and Filipino were among the lower-resourced languages. A notable outlier is Afrikaans  $(en + af)$ , which was the lowest resourced language in our experiments yet XLM- $\mathbf{R}_{\mathrm{base}}$  was quite robust to adversaries constructed from it and English. A possible explanation is that Afrikaans' language family, Indo-European, is highly represented in the pretraining corpus. Another notable outlier is Vietnamese, which was the fourth most resourced language in the pretraining corpus, yet the model was more vulnerable to adversaries constructed from English and Vietnamese than adversaries constructed from English and Filipino, one of the lowest resourced languages. A possible explanation for this is the use of Latin characters combined with little vocabulary overlap between English and Vietnamese, and differing adjective positions.
$^{12}$ github.com/ray-project/ray
Lgs. (en+)	Exemplar Region	Acc.	Size (GB)
tr	Turkey	55.76	20.9
de	Germany	55.42	66.6
bg	Bulgaria	54.87	57.5
ru	Russia	54.33	278.0
th	Thailand	52.43	71.7
el	Greece	52.41	46.9
zh	China	52.17	46.9
ar	Middle East	51.53	28.0
es	Spain	50.07	53.3
fr	France	49.00	56.8
hi	India	48.62	20.2
ur	Pakistan	42.09	5.7
sw	Kenya	38.54	1.6
vi	Vietnam	36.32	137.3
ro	Romania	56.26	61.4
id	Indonesia	50.91	148.3
af	Namibia	49.30	1.3
sq	Albania	40.13	5.4
tl	Philippines	36.96	3.1
ru+uk	Ukraine	44.37	362.6
ar+he	Israel	37.24	59.6
af+de	Namibia	36.46	67.9
de+fr	Switzerland	35.84	123.4
id+zh	Indonesia	35.30	195.2
hu+ro	Transylvania	34.11	119.8
ar+fr	Morocco	32.15	84.8
hi+ur	Kashmir	29.12	25.9
ar+sw	Tanzania	24.37	29.6
fi+sv+ru	Finland	30.27	344.4
cs+hu+sk	Slovak	30.21	97.2
es+fr+it	S. Europe	26.58	140.3
mk+sq+tr	Albania	23.53	31.1
da+de+nl+sv	Denmark	27.74	153.6
id+th+tl+vi	S.E. Asia	12.53	360.4
Table 14: How XLM- $\mathbf{R}_{\mathrm{base}}$  might fare against real-life adversarial polyglots and where they might be found. Scores are accuracy on the XNLI test set with English as the matrix language. Corpora sizes reproduced from Conneau et al. (2020a).
We also find XLM-R<sub>base</sub> to be twice as robust in the  $en + da + de + nl + sv$  condition as the  $en + id + th + tl + vi$  condition. It is likely that the structural similarity of the mixed languages also plays an important role in determining the model robustness, reinforcing K et al. (2020)'s similar findings on cross-lingual transfer. Investigating the effects of typology, vocabulary and orthographic overlap, pretraining corpus size, and interaction effects between different sets of languages on a model's robustness to code-mixed adversaries could lead to new insights into how different languages interact in the multilingual embedding space and we leave this to future work.
G More Tables, Figures, and Algorithms
Figure 3: Domain Adversarial Neural Network Section continues on next page.
Table 15: A comparison of code-mixed examples produced by bilinguals (R) and BUMBLEBEE (BB). Since the humans were only provided with the English examples, some differences in phrasing is to be expected.
Model	Clean	PGunfit.	PGfit.	BUMBLEBEE	BUMBLEBEEunconstr
XLM-Rlarge	81.10	6.06	28.28	11.31	5.22
XLM-Rbase	75.42	2.17	12.27	5.08	1.47
mBERTbase	67.54	2.15	9.24	6.10	1.19
Unicoderbase	74.98	1.99	11.33	4.81	1.29
Table 16: POLYGLOSS (PG) and BUMBLEBEE results on the XNLI-13 test set with a beam width of 1. PG{fil.,unfiltr.} indicates whether the candidate substitutions were filtered using reference translations. BUMBLEBEEunconstr refers to the setting without the equivalence constraint (§3.2).
(a) Cross-Lingual Transfer
(b) Translate-Train- $n$
(c) Domain Adversarial Neural Network
(d) Code-Mixed Adversarial Training
Figure 4: t-SNE visualizations of XLM- $\mathbf{R}_{\mathrm{base}}$  representations fine-tuned using different methods. We use Linderman et al. (2019)'s t-SNE algorithm implemented in openTSNE (Policar et al., 2019) and tried to arrange related languages close to each other on the color spectrum (to the extent possible on one dimension) so it would be obvious if similar languages were getting clustered together, as in (c).
(b) Translate-Train-  $n$
(d) Code-Mixed Adversarial Training
Figure 5: t-SNE visualizations of XLM- $\mathbf{R}_{\mathrm{base}}$  representations fine-tuned using different methods. Here, we average all the token embeddings in the sentence instead of just using the  $
$  embedding.