Hyperpolyglot LLMs: Cross-Lingual Interpretability in Token Embeddings
Authors: Andrea W Wen-Yi
David Mimno
Venue: EMNLP 2023 Main (2023)
Keywords: Multilingual models
cross-lingual representations

Abstract: Cross-lingual transfer learning is an important property of multilingual large language models (LLMs). But how do LLMs represent relationships between languages?
Every language model has an input layer that maps tokens to vectors. This ubiquitous layer of language models is often overlooked. 
We find that similarities between these input embeddings are highly interpretable and that the geometry of these embeddings differs between model families. In one case (XLM-RoBERTa), embeddings encode language: tokens in different writing systems can be linearly separated with an average of 99.2\% accuracy. Another family (mT5) represents cross-lingual semantic similarity: the 50 nearest neighbors for any token represent an average of 7.61 writing systems, and are frequently translations. 
This result is surprising given that there is no explicit parallel cross-lingual training corpora and no explicit incentive for translations in pre-training objectives. 
Our research opens the door for investigations in 1) The effect of pre-training and model architectures on representations of languages and 2) The applications of cross-lingual representations embedded in language models.

Hyperpolyglot LLMs: Cross-Lingual Interpretabilityy in Token Embeddings
Andrea W Wen-Yi
David Mimno Cornell University mi mimoo@cornell.c edu
Cornell University
andreawwennyieinoocic.commm wwtp://einn.ccc..oooe...eeeee
Abstract
total parameters in a model. It also serves as the connection between headalre strings and the latent vector representations that initialize the remaining layers. such, the initial token em- bedding layer is both geometrically expressive and readily interpretable. We explore the initial token embeddings of two highly multilingual model fam- ilies, XLM- ALE-RaBERTa (XLM-R) (Conneau et al., 2020) and mT5 (Xue et al., 2021). We find that m'55 discovers a universal, cross-lingual semantic space that assigns words (or word fragments) with similar meanings to nearby vector positions."
Cross-lingual transfer learning is un important property of multilingual large language mod- els (LLMs). But how do LLMs represent rela- tionships between languages? Every language model has an input layer that maps tokens vectors. This ubiquitous layer of language mod- is often overlooked. We find that similari- ties between these input embeddings are highly interpretable and that the geumetry of these em- beddings differs between model families. In one case (oLMMRoBERTaa), embeddings en- code language: tokens in different writing sys- tems can be linearly separated with an average of 99.2% accuracy. Another family (mT5) rep- resents cross-lingual semantic similarity: the 50 nearest neighbors for any token represent an average of 7.61 writing systems, and are fre- quently translations. This result is surprising given that there no explicit parallel cross- lingual training corpora and no explicit incen- tive for translations in pre-training objectives. Our research opens the door for investigations in 1) The effect of mre-trining and model chitectures on representations of languages and 2) The applications of cross-lingual representa- tions embedded language models.
Previous work has shown that algorithms exist to train multilingual word embeddings without ex- plicit parallel text or glossarics (Ammar ct al., 2016; Chen and Cardie, 2018). What novel about the current work is the discovery that certain highly multilingual LLMs contain such an embedding as an emergent property without any explicit instruc- tion to do so.
Explaining the factors that cause LLMs to find cross-lingual semantic embeddings would require pre-training experiments that are beyond the scope of this short paper. Describing these behaviors is. however, a necessary first step. At the same time. there is increasing attention on producing language technology for lower-resource languages (Kumar and Albuquerque, 2021), where the data- hungry methods that have been successful for high- resource languages may not be applicable. Creat- ing predictive theories that explain the relationship between properties of pre-training data and repre- sentations learned by LLMs could lead us to build collections, architcctures, and algorithms that most efficiently improve performance. This will be an extremely valuable step to enhance language tech- nology for low-resource languages.
1 Introduction
Multilingual large language models (LLMS) have the potential to support transfer learning between languages with little to no additional training data (Lauscher et al.. 2020; Wu and Dredze, 2019; Con- neau et al., 2020; Winata el al., 2021). Bul we have limited theory for how LLMs represent meanings across languages. This work describes a mecha- nism for lingull transfer learning by measur- ing the properties of the input embedding vectors. While most of the interest in LLMs rightly focuses their ability to produce contextualized output, in this study we focus specifically on the lowest network layer: the initial token embedding layer. This layer often comprises a large percentage of the
'Code is available ut: andreawwenyl hwwpwwwnyi/hppprreeeee /
https://gåthub.com

---

XLM-RoBERTa
mTS-XL
casagory
LATIN
CJK
ARABIC
CYRILLIC
CREEK
KATAKANA
HEBREW
HANGUL
S (Symbol))
HIRAGANA
THAI
DEVANAGARI
Figure 1: 2D Projections of input token embeddings. In XLM-RoBERTa-XL. (left), writing systems widely spaced; while for mT5-XL (right), vectors are more mixed.
2
Related work
3 Multilingual vocabularies
Previous work has found interpretable patterns in feed-forward layers (Geva al., 2021, 2022), self- attention (Mrini et al., 2020; Clark et al., 2019; Serrano and Smith, 2019), and input embeddings from the perspective of adversarial perturbation (Sato et al., 2018). this work, we directly inter- pret the relative positions of the token embeddings through the lens of the vocabularies.
Sub-word tokenization and writing systems Initial embedding layer maps tokens to vectors. Most contemporary language models sub-word tokenization schemes such byte-pair encoding. These methods balance representational richness (providing distinct vectors for as many inputs as possible) with efficiency in limiting overall vocab- ulary size. While some methods produce tokens consisting of byte sequences that are valid UTF- 8 characters, in practice almost all tokens are valid, displayable sequences of Unicode characters, and large number recognizable words
Prior to the explosion of contextualized language models, there was substantial interest in cross- lingual word embeddings (CLWE) (Ruder et al., 2019; Mikolov et al., 2013; Lazaridou al., 2015). The goal often map monolingual word embed- dings from different languages the same shared space, so that words of the same meaning are close to each other. CLWE approaches involve some levels of supervised alignment (Faruqui and Dyer, 2014; 4: Zou et al., 2013), seed dictionaries (Arteixe et al., 2017; Gouws and Søgaard, 2015) or adver- sarial training (Lample et al., 2018; Artetxe et al., 2018; Zhang al., 2017; Miceli Barone, 2016). Contexualized embeddings from language mod- els have also been used in combination with static word embeddings to improve alignments of cross- lingual word vectors (Aldarmaki and Diab, 2019; Zhang et al., 2021). Contrary to our findings for the token embeddings of LLMs,, was not clear that aligning word vectors possible without some level of supervision, to more than two languages at time. Previous results also showed that CLWE approaches are sensitive to language pairs, where languages with large semantic or structural differ- ences usually failed, such as English-Japanese and Spanish-Chine Chinese (Xu et al., 2022).
As it is difficult to assign tokens to specific lan- guages (c.g. war can be an English noun Ger- man verb), we use Unicode metadata to define cat- egories of tokens. For cxample, is LATIN SMALL LETTER Most characters are letters (L), vo- cabularics also include punctuation (P), numbers (N), and symbols (S, which include emoji). Letters also marked by writing system, such LATIN, CY CILIII,,, BENGALL. We define the calegory of character as either its writing system (for let- ters) or its Unicode class for non-letters. A token can mix characters of different categories, but they typically have most frequent category: the string doesn' 't contains letters and punctuation, but is primarily LATIN letters. We define the category of token based on its majority character category. As result, doesn' is classified as LATIN.
Overlap between model vocabularies While model families have distinct sets of vocabular-
We find 39, 93% of mT5 and 47.78% of XLM--R vocah- ularies in multilingual dictionaries provided by Lample et al. (2018).

---

$13 folon
prportier
models from mT5 and XLM-R families. In ad- dition to XLL models being potentially more ex- pressive, XLM-R-XI. and mT5-XI. are also more comparable in parameter size (3.5B and 3.7B, re- spectively). Figure I shows UMAP (Mclnnes el al., 2018) projections of the embeddings from XLM- R-XL and mT5-XL for each loken in the shared vocabulary, colored by Unicode category. We find that XLM-R's representations encode languages tokens of different categories form isolated clusters. also noticeable
6770
-22
There is substantial overlap in vocabulary between models (mean 60.4%), but much of the overlap is Unicode LATIN, mTS und XLM-R have the largest non-Latin intersection (59.4%). While large, BLOOM vocabularies do not include Slavic languages, Japanese, or Korean.
Figure 2:
Clusters of Unicode categories in mT5 but they are more overlapping. The ex- ception is Symbols (S) and Numbers (N): they are scattered in XX,, but clustered in mT5-XL.
W1T54L
XL44OL
200
cos
To further show how embeddings encode lan- guages, we use logistic regression to predict Uni- code category from embeddings. For each category, we construct balanced datasct and run 10-fold cross-validation. Embeddings from XLM-R-XL and mT5-XL both encode a high level of language in information. Tokens with different Unicode cate- gory could be linearly separated with an average of 99. 24% and 93.32% accuracy, respectively. Figure 3 shows accuracy for selected categories. XLM-R- XL has significamtly higher (near perfect) accuracy across the majority of categories than mT5-XL
Figure 3: Token embeddings can predict Unicode cat- egories. Both models have high accuracy, but XLM- R-XL is significantly higher than mT5-XL across the majority of categorics.
ies, they are often comparable. Current vocab- ulary sizes of monolingual models are often s 32,000 (BERT, T5, LLaMa), 50,000 (GPT2, GPT- J. Pythia), or 65,000 (falcon). Highly multilinuual models have larger sizes around 250,000 (mT5, umT5, XLM-R, BLOOM).
Embedding neighborhoods encode semanties across languages. We next zoom in to study the neighbors of individual tokens. Neighbors from mT5-XL are often direct translations, whereas XLM-R-XX XL finds tokens in the same language that semantically or linguistically similar. Figure 5 shows that the 50 nearest neighbors for mTT5XLL tokens have an average of 7.61 different Unicode categories, whereas XL has 1.64. Figure 4 shows examples of 20 nearest neighbors of selected tokens.
To make strings comparable across tokenzation methods, we replace the space character in BPE to- kenizations with Unicode LOWER ONE EIGHTH BLOCK, which looks like long underscore, as it is more readable and compatible with T5-style SentencePiece vocabularies.
Figure 2 shows the total overlap in vocabularies between six LLMs. We select mT55 and XLM-R as the most multilingual comparison due to their large vocabulary size, significant overlap, and large overlap in non-LATIN lokens. of the models with a large vocabulary, BLOOM focuses more on African and South Asian languages and thus has a much smaller token overlap in CYRIILIC, HANGUL, HIRAGANA, and KATAKANA.
The neighboring calegories of tokens in mT5 vary by the category of a token, as shown in Figure 6. We find interesting comparisons between two Japanese writing systems, KATAKANA and HIRA- GANA. KATAKANA, a writing system often used for words of foreign origin, has more diverse neigh- bors (most often LATIN). HIRAGANA, used more for native Japanese words, has more neighbors in HIRAGANA and CJK (Kanji). We do not find evi- dence to suggest that tokens appearing more often in pre-training corpora have more diverse neigh- bors (Figure 9 in the Appendix).
4 Results
Models encode languages differently. We make comparisons between the embeddings of XL-scale

---

token
XLM-R-XL neighbors
mT5-XL neighbors
Comment
Comments, Review, Blog, Text, Group, Info, Support, Photo, Share, Work, Information, Articie, Link, Video, Chat, Search, News
Commants, komment ("zommwal" Kommentar ("smoowa", su), omeniária, Commentaree ("Comesen", 85, Koueurap I'Comment" bil. Kommentare (CCommeti" de), Komentar (CCnoner"" ms), Komant ("Dommentay", sq, kommentarer, comentario ('oammorr, es), coment
Nike
Adides, Samsung. Rose, Mini, BMW, Apple, NBA, Toyota, Blue, Mike Honda, Valkswagen, Green, Max, Puma, Alex, 55, Ferran
Adidas, 4 (Wil", Asics, Reebok, Puma, 4 ("MM", Converse, SICS, Jordan, LHO13l (PMkee Ro). sheaker
milki
mempunyei (havee; me) shiks ("hldd cm), memilki ('posseest; me) hilzji ("reed: saji ("egister, sul, Li gunakan ('use", met, shiriu ("aectoppst"" svy, Ka, ("ook") met
punya ("have", ma) milkk ('oosession, ma), HHAi ('ow",, zty, lakukan ('de', berikan flgvv", mot, liene ('has" oe), makai, sahip (oowmrr" t5. possu, og. irkich, (hhudrie ('own,, (N), enggunekan, km), R& Tpooooss" At. (6 (oows) my), ("hoo', (j) ndiakan, Seru
""pusses".
mu). bewe ('wing" am), hitadhi ("rowros" au)) ("ive" aw), shaun ("o ativior" and, bentuk ("stape" ma). kata ('oxt) on), gambar ('onnoe' mo)
('ovee
nar
when, nár ('when", do), da ('hhnn av), eftersom ("ance" ovt, Ivá r'woa. av), neer ("claser, (9), dar ("whoret, sx,, for (foor av, försia ('frrt, avi frán I'rom", här (here", ad, ocksä ("aleo" nástan fammmoet novent; avt, ("are" sv), innan ('botave", av), stora ("Targo" sv). mánniskor ('people av)
när ("umort, da), ner ("obserr 1j), when, lätte ('20;, et, cuardo ("wmeer, os), quando ('whont, at), up ('Whont, not, für ("arr) ad, imaa ("bofore". Wenn (?)) de), hur ('oow,, quand ("wmert, M.
('hhent sv)
quan
7 I"Amerta",
M& r'USA,, 1 ('Egglad" 19E9 ('oraa", 3] R PUSA"
TD0 ("Amekcar, 0 CUSA", ko), Amerika ("Amaria") 28 MUSA", jal, 90:30 rUSA", zt), K ("Einllad" jsll, AMrриss ("Amenca", mit, wltefan ("Amorias", ka). #6 ('USA' 20), 7552 ("Frnnee" 19). &+Cioffl y'Amstcant, ral, America, mpeadde) ("Amenca" mi,
Ro), 253/2 ("Francet,00 .8). (Semmnyy" (8). (917 (1tat") (8). 757
("Ass",, jot. (3 B3:000 (MAapanccc" (al, B5/7 ('Ruasict jal, B-Por
("Ewrooe", js).. NER ("baayot, jo), 869 ("abroast, ja), 90 rusa", zM) RM
("Crgtona"", 2t), (S/))) ("American (Sue ("Amrroo"" fa). M a r'epann. (o). X08 PUSA" zn), ("modenst, ab)
merrkknnk ('Amerizan", amerkanische ("Amerizan" de), meerika, suflm ("Amenca", 1h)
Figure 4: The top 20 neighbors from mT5-X XL are often direct translations; whereas those from XLM-R-XL are often semantically finguistically similar words in the same language. The last row is example of KATAKANA, the Japanese writing system used for words foreign origin. ISO 639-1 language ubbreviations are included in the Appendix. Some words exist in several languages, we record only one. Closely related languages are often neighhors, c.g. Swedish (sv) and Danish (da). Malay (ms) words translated from root terms.
XLM RoBERTa XL
KATAKANA
1500
COURT
1250
1000
HIRAGANA
750
COUTT
500
250
CJK
mT5-XL
250
200
LATIN
150
100
CYRILLIC
KETRAAMA
HERAGANA
qK
LATIN
ORRLIC
neigh bog.bbrr.cc Category
Figure 5: The 50 nearest neighbors for mT5-XL XL tokens have an average of 7.61 Unicode categories, and XLM- R-XL has 1.64. The figure is made by randomly selected 1% of tokens.
Figure 6: In the 50 nearest neighbors of mT5-XL to- kens, HIRAGANA and LATIN find the majority of neighbors in their respective writing systems; whereas KATAKANA and CYRILLIC tokens have more diverse neighbors.
Embedding geometries are similar across pa- rameter scales. Finally, we consider whether ini- tial token embeddings as whole are similar across model families and parameter scales. We use two metrics to quantify similarity in geometric struc- tures of embedding space.
include those tokens. For cach token in the shared vocabulary, we then find the 100 nearest neighbors within each matrix using cosine distance. Finally, we calculate the overlap between these neighbor sets from each pair of models. For example, average, 60 of the 100 nearest neighbors of token in XLM-R-XL will also be nearest neighbors of the same token in XXX-- Figure 7 compares the average number of overlapping terms out of 100
The first metric measures the local similarity between vectors in two matrices. We first find the set of tokens shared between model families and create subset embedding matrices that only

---

neighber count @100
bicom-1o)
50
0.99
m55 base
miSlaage
20
xim roberta-g)
-0
boom
mi
mis
mnS
stmvesbara
Figure 8: Embeddings for overlapping tokens have simi- lar geometries. as measured by canonical angles. XLM- R models are extremely similar to each other and mT5 small and base. All models are far from random (0. 14- 0.27).
Figure There is high level of overlap between the nearest neighbors of tokens, derived from embeddings. XLM-R models are the most similar, with up out of 100 shared neighbors. mT5 is less consistent, and BI BLOOM is significantly different.
5 Conclusion
While impul embeddings and sub-word tokenized vocabularies of LLMs may appear inscrutable, we find that they are fact interpretable and mean- ingful. We observe significant patterns that differ by model families, including an emergent ability of mT5 discover shared semantic space that spans languages accidentally achieving goal that defied decade of research on cross-lingual word embeddings.
nearest neighbors for pairs of models, including BLOOM for comparison. Results are averaged over 45,000 shared tokens hetween mT5, XLM-R, and BLOOM. We find that XLM-R models have the largest similarity, mT5 are more varied, still find 20-30 shared tokens on average. BLOOM is the most different: the 1.IB model shares only 5-10 tokens with other models, the 3B 15-20.
Future directions include explaining factors that cause the different token embedding patterns we observe in XLM-R and mT5. This could include investigations model architectures, pre-training procedures, and data-centric strategies such as the curation of pre-training corpora. Finding ef- ficient path to more equitable language model performance will be valuable for enhancing lan- guage technology for low-resource languages. An interesting next study could explore the utility of combining low-resource languages with closely related higher-resourve languages in pre-training corpora. Another future direction to explore the potential applications of cross-lingual represen- tations embedded in LLMs What does it say about downstream applications? Could it be used to guide practitioners to select models that are more appropriate for their intended uses?
The second metric uses canonical angles, linear algebraic measure of rotational alignment. While it extremely unlikely that input embedding trices will be comparable at the level of individual columns or entries, the matrices of two models may be identical up to rotation or permutation. We can measure the degree to which two matrices are rotations through the method of canonical angles, Given matrices and B both with n rows, we cal- culate Q. A and QnR form the SVD UTVT Q8OBB. The singular val- ues E are the cosines of the angles of rotation. The single largest value is the cosine of the smallest angle, so it forms an upper bound on our ability to rolate one matrix lo match the other. Figure 8 shows the first singular values for pairs of models restricted to rows for their shared vocabularies, in- cluding random "embedding" matrix with size 512. We find that XLM-R models have high ro- tational similarity to each other (0.99 of 1.0), while mT5 models are more differentiated (0.95- 0.97) but still highly similar. All models are signifi- cantly more similar to each other than to random matrix (0. 10.15-10 15-0.77).
B. Then we
6 Limitations
This work is descriptive rather than explanatory. We observe that there are patterns in the geometric structure of input embedding matrices in families of LLMs, but we are unable to identify why these

---

patterns emerge and differ. There are many differ- ences in model architectures. training methods, and pre-training corpora between LLM families. It is out of the scope of this work to determine what fac- tors are causal. As we have limited ability to carry out pre-training experiments, we chose to focus on descriplive observations of existing LLMs.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettle- moyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 8440- 8451, Online. Association for Computational Lin- guisties.
Acknowledgements
Manaal Faruqui and Chris Dyer. 2014. Improving vec- for space word representations using multilingual correlation. In Proceedings of the 14th Conference of the European Chapter of the Association for Com- putational Linguisties, pages 462-471, Gothenburg, Sweden. Association for Computational Linguistics.
We would like to thank the anonymous review- ers for their valuable comments. We'd also like thank Inle Bush, Hyunju Kim, Omary Mzava, Damiel Mwesigwa, Cheng Perng Phoo. Top Piriyakulkij, and Ahkln Wong for their assistance in vocabulary translation.
Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Gold- berg. 2022. Transformer beed-forward layers build predictions by promoting concepts in the vocabulary space. In Proceedings the 2022 Conference Empirical Methods Natural Language Process- ing. pages 30-45, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
References
Hanan Aldarmaki and Mona Diab. 2019. Context-aware
cross-lingual mapping. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume I (Long and Short Papers), pages 3906-3911 Minneapolis, Minnesota. Association for Computational Linguistics.
Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer food-fore forward layers are key- value memories. In Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing, puges 5484-5495, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Waleed Ammar, George Mulcaire, Yulia Tsvetkov, Guil- laume lample, Chris Dyer, and Noah A Smith, 2016. Massively multilingual word embeddings. arXiv preprint
Stephan Gouws and Anders Søgaard. 2015, Simple task- specific bilingual word emheddings. In Proceedings of the 2015 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 1386-1390, Denver, Colorado. Association for Computational Linguistics.
Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2017. Learning bilingual word embedldings with (almost) no bilingual data. In Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers). pages 451-462, Vancouver, Canada. Association for Computational Linguistics.
Akshi Kumar and Victor Hugo C. Albuquerque, 2021. Sentiment analysis using xlm-r transformer and zero- shot transfer learning on resource- -poor indian lan- guage. ACM Trans. Asian Low-Resour. Lang. Inf. Process., 20(5).
Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2018 A robust learning method for fully unsupervised cross-lingual mappings of word embeddings. In Pro- ceedings of the 56th Annual Meeting of the Associa- tion for Computational Linguistics ( (Volume I: Long Papers), pages 789-798, Melbourne, Australia. As- sociation for Computational Linguistics.
Guillaume Lample, Alexis Conneau, Marc' Aurelio Ran- zato, Ludovic Denoyer, and Hervé Jégou. 2018 Word translation without parallel data. In Interna- tional Conference on Learning Representations.
Xilun Chen and Claire Cardic. 2018 Unsupervised multilingual word embeddings. In Proceedings of the 2018 Conference on Empirical Methods in Natu- ral Language P Processing, pages 261-270. Brussels, Belgium. Association for Computational Linguistics.
Anne Lauscher, Vinit Ravishankar, Ivan Vulić, and Goran Glavaš. 2020, From zero to hero: On the limitations of zero-shot language transfer with mul- tilingual Transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Lon- guage Processing (EMNLP), pages line. Association for Computational Linguisties
On-
Kevin Clark, Urvashi Khandelwal, Oner Levy. and Christopher D. Manning. 2019. What does BERT look at? analysis of BERT's attention. In Pro- ceedings of the 2019 ACL Workshop BlackboxNLP. Analyzing and Interpreting Neural Networks for NLP, pages 276-286, Florence, Italy. Association for Com- pulational Linguistics.
Angeliki Lazaridou, Georgiana Dinu, and Marco Ba- roni. 2015. Hubness and pollution: Delving into cross-space mupping for zera-shot learning. In Pro- ceedings of the 53rd Annual Meeting of the Asso- ciation for Computational Linguistics and the 7th

---

International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 270-280, Beijing, Chima. Association for Computational Lin-
Yuemei Xu, Han Cao, Wanze Du, and Wenqing Wang- 2022. survey of cross-lingual sentiment analy- sis: Methodologies, models and evaluations. Data Science and Engineering, 7(3):279-299.
guistics.
Leland McInnes. John Healy, and James Melville. 2018. Umap: Uniform manifold approximation and pro- jection for dimension reduction. arXiv preprint
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami AI-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mTS: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Longuage Technologies, pages 483 483-498, On- line. Association for Computational Linguistics.
arXiv:1802.03426.
Antonio Valerio Miceli Barone. 2016, Towards cross- lingual distributed representations withoul parallel text trained with adversarial autocncoders. In Pro- ceedings of the 1st Workshop on Representution Learning for NLP, pages 121-126, Berlin, Germany. Association for Computational Linguistics.
Jinpeng Zhang, Baijun Ji, Nini Xiao, Xiangyu Duan, Min Zhang, Yangbin Shi, and Weihua Luo. 2021. Combining static word embeddings and contextual representations for bilingual lexicon induction. In Findings of the Association for Computational Lin- guisticx: ACL-IJCNLP 2021. pages 2943-2955, On- line. Association for Computational Linguisties.
Tomas Mikolov, Quoe Lc,, and Ilya Sutskever. 2013. Exploiting similarities among languages for machine translation. arXiv preprint XrXiv: 1309.4168,
Khafil Mrini, Franck Demoncourt, Quan Hung Tran, Trung Bui, Walter Chang, and Ndapa Nakashole. 2020. Rethinking self-attention: Towards inter- pretability in neural parsing. In Findings of As- sociation for Computational Linguistics: EMNLP 2020, pages 731-742, Online. Association for Com- putational Linguistics.
Meng Zhang, Yang Liu, Huanbo Luan, and Maosong Sun. 2017. Adversarial training for unsupervised bilingual lexicon induction. In Proceedings of the 55th Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), pages 1959-1970, Vancouver, Canada. Association for Computational Linguistics.
Schastian Ruder, Ivan Vulié, and Anders Søgaard. 2019. A survey of cross-lingual word embedding models. Journal of Artificial Intelligence Research, 65:569- 631.
Will Y. Zou, Richard Socher, Daniel Cer, and Christo- pher D. Manning, 2013. Bilingual word emheddings for phrase-based machine translation. In Proceed- ings of the 201.3 Conference Empirical Methods in Natural Language Processing, pages 1393-1398, Seattle, Washington, USA. Association for Computa- tional Linguistics.
Motoki Sato, Jun Suzuki, Hiroyuki Shindo, and Yuji Matsumoto. 2018 Interpretable adversarial perturba- tion in input embedding space for text. In Proceed- ings of the Eventy-Seventh International Joint Con- ference on Artificial Intelligence, IJCAI-18, pages 4323-4330. International Joint Conferences on Arti- ficial Intelligence Organization.
A ISO 639-1 language codes
The language codes for Figure 4: ar: Arabic, bg: Bulgarian, da: Danish, de: German, es: Spanish, et: Estonian, fa: Persian, fr: French, he: Hebrew, hu: Hungarian, is: Icelandic, ja: Japanese, km: Khmer, ko: Koream, mk: Macedonian, ml: Malayalam, ms: Indonesian/Malay, my: Burmese, pt: Portuguese, ru: Russian, sw: Swahili, sv: Swedish, sq: Alba- nian, tr: Turkish, th: Thai, zh: Chinese
Sofia Serrano and Noah A. Smith. 2019. Is attention in- terpretable? In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 2931-2951, Florence, Italy. Association for Computational Linguistics.
Genta Indra Winata, Andrea Madotto, Zhaojiang Lin, Rosanne Liu, Jason Yosinski, and Pascale Fung. 2021. Language models are few-shot multilingual learners. In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 1-15, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Frequency of tokens does not correlate with diversity of neighbors.
It could be that words with fewer occurrences in the training corpus would have more or less diversity in Unicode categories in their neighbor sets. We calculated an estimate of the frequency of mT55 SP tokens based sample from the mC4 dataset. We then took stratified sample of tokens from the mT5 vocabulary from 10 frequency bands and calculated the mean number of distinct Unicode categories for their neighbors, see Figure 9. We
Shijie Wu and Mark Dredze. 2019. Beto, bentz, becas: The surprising cross lingull effectiveness of BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language pages 833-844, Hong
Processing (EMNLP-IJC.N) Kong, China, Association for Computational Linguis- tics.

---

find correlation between the frequency of terms and the diversity of their nearest neighbor sets.
hurhhegghorCCaoooeyy @ 50
Daciliss tokan (eequancy tramiing
Figure 9: There is correlation between how often token is found in pre ve-training corpora and how diverse token's 50 nearest neighbor set is. The neighbor set calculated with mLT5XLL embeddings.


Reviews:
[{"review_id": "QBumouXa4b", "reviewer": "Reviewer_Eu6p", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the reviews! We are particularly grateful for the recognition of what can be accomplished in the scope of a short paper. We are not aware of prior work exploring the interpretability of multilingual LLMs’ initial embedding layer. As we put in the related works section, prior works on building cross-lingual word embeddings relied on supervision and are sensitive to language pairs; we believe this is what makes our findings exciting: language models construct embeddings that encode cross-lingual semantic relationships without supervision and with multiple languages concurrently.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "I think this is a very well-written and dense short paper. It answers all the questions about its specific topic that could be answered both in feasible time and in 3 pages, and includes both experiments at scale and closer detailed looks. While I'm not familiar enough with the literature to be sure that there is no prior work on this, on the assumption that there isn't, this seems to me a specific and confused contribution in the best sense. The authors acknowledge that it would be much more interesting to create experiments to determine why the embedding spaces are so different and how that might affect the model performance, but I agree that it would be out of the scope of this paper.", "reasons_to_reject": "I can't immediately think of any (beyond possibly the limited scope, which would seem unfair for a very dense short paper), but I'm not familiar enough with the literature to exclude the possibility that they exist. ", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper investigates the token emebdding layer of multilingual LMs. It automatically investigates the distribution of writing systems. It also trains a language classifier for scripts with 1-1 language-script mappings, and looks at the nearest neighbours both automatically and with manual investigation. Interesting differences are found between the embedding layers of different popular multilingual LMs.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Is something wrong with the references? Sometimes they just say \"In ACL\". I recommend to use the official ACL anthology bibkeys wherever possible."}, "extra_text": {}}, {"review_id": "D7uOUVEivg", "reviewer": "Reviewer_KmTx", "summary": "", "questions": "- Not necessarily a question, but it might also be interesting to compare the models wrt their \"type\", i.e., decoder-only vs. encoder-only models. For this, you might add multi-lingual BERT, for example.\n", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for your input! We are glad that you find our insights interesting and particularly appreciate that you checked our figures in the appendix – we would certainly love to move some of the content currently in the appendix to the main paper in a final version.\n\nRegarding how to use our insights: While our goal in this short paper is interpreting encodings of multilingual LLMs, we can further clarify directions for future work, taking inspiration from the questions raised here. Specifically, future directions could include explaining what contributes to the different geometry of embeddings we observe in XLM-R and mT5. This could include investigations in model structure (encoder/decoder) as well as data-centric strategies such as curation of pretraining data. This would be especially valuable for enhancing language technology for low-resource languages where the current data-hungry methods do not work well. Another future direction will be how to use our knowledge about the embedding geometry of multilingual LLMs – For example, what does it say about downstream applications? Could it be used to guide practitioners to select models that are more appropriate for their intended uses?", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper provides interesting insights and sheds some light on the differences between the encodings of different models.\n- Although the results are not \"super surprising\", they might help to choose models depending on their characteristics. The findings of the paper might be an interesting add-on to, e.g., the model cards on huggingface.\n", "reasons_to_reject": "- The paper is rather descriptive, as the authors say themselves, in that they only *show* the described characteristics, and do not explain them. \n- The authors should add some ways how to use their insights: What can we learn from knowing the difference of encoding styles (mT5-XL, XLM-R-XL). Can we learn when to select which model or how to leverage them better differently? Similarly, what can we learn from the embedding similarity among families?", "questions_to_authors": "- Not necessarily a question, but it might also be interesting to compare the models wrt their \"type\", i.e., decoder-only vs. encoder-only models. For this, you might add multi-lingual BERT, for example.\n", "paper_topic_and_main_contributions": "The authors investigate the questions of how multilinguality is encoded in the embedding layers of different multi-lingual LMs.\nThey compare XLM-R and mT5 in more detail (but also other models) and find that XLM-R encodes languages by writing systems, while mT5 rather aggregates similar words in the same areas, no matter their language.\nThis is interesting, since these models were not trained on parallel data.\nThey find that the XLM family\n- encodes languages into more isolated clusters (fig 1)\n- achieves a higher accuracy on unicode prediction (fig 2)\n- shows higher language unification among tokens' neighbors (fig 3)\n- neighbors tends to be synonym than translation (fig 4)\n\nComparing model families, the XLM family encodes the embedding the most alike.\nFurther, the XLM family seens to have the highest overlap between the nearest neighbors of token embeddings followed by the mT5 family. Finally the BLOOM family has the most variants.\nThe XLM family embeddings are more alike than those of mT5 family (appendix fig 8)\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Some of the plots would be nice to be in the main part of the paper, e.g., Figure 6, maybe you can shorten some of the text. \n"}, "extra_text": {}}, {"review_id": "L35ySTemQ3", "reviewer": "Reviewer_E6Zp", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We are delighted that you found our work in cross-lingual interpretability of embedding layers interesting and easy to follow!\n\nRegarding the significance of the work: the result we are most excited about is that some (but not all) models encode semantics *across languages* despite having no supervision to do so and no explicit parallel cross-lingual training data. We can clarify this in a final version. As in many cases, we will also try to emphasize that there are several possible hypotheses for how embedding vectors are organized, any one of which might have seemed \"obvious\" in retrospect.\n\nRegarding our findings being not new, we have had difficulty finding any comparable work and would be happy to include any references that can be suggested.\n\nWith respect to the vector distribution of different categories not being consistent among model families – this is exactly what we find interesting and important in our work! In both our introduction and Figure 1, we emphasize that embeddings of XLM-R encode language first and then semantics, while that of mT5 encode semantics across languages. We did not know this property in advance and cannot currently explain it. We believe this discovery will be useful in guiding future development of language models, especially for low-resource languages.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper is well-organized and easy to follow. \n2. The idea of studying cross-lingual interpretability is interesting.\n2. The analysis in this paper is appropriate and adequate.", "reasons_to_reject": "The findings of this paper are somewhat weak. Firstly, the vector distribution of different categories is not consistent among model families. Moreover, the findings that \"embedding neighborhoods encode semantics\" and \"embedding geometries are similar across scales and model families\" are not new to me.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper studies the cross-lingual interpretability of token embeddings by analyzing the distribution of vector space. The experiments demonstrate that: 1) the distribution of Unicode categories differs among model families; 2) embedding geometries are similar across scales and model families.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work examines the input (non-contextual) embedding structure of two families of multilingual LMs, finding that one (XLM-RoBERTa family) learns to separate languages (as one might guess) while the mT5 family unsupervisedly learns cross-lingual structure in the embedding space, as nearest neighbors tend to be related words in other languages. Reviewers praised it for being well-written and interesting, but had some concerns as to the lack of concrete takeaways, a somewhat small scope, and the importance of the results relative to existing knowledge of cross-linguality in language models. 

Overall, I think that it is a reasonable contribution for a short paper, but I would challenge the authors to engage with the question of what the importance of these results are for the broader goal of understanding the models–do they suggest an interesting next study? Are they related to an overall improved cross-lingual transfer behavior in mT5?
Decision: Accept-Main