Do Llamas Work in E nglish? On the Latent Language of Multilingual Transformers (2024)

Abstract: We ask whether multilingual language models trained on unbalanced, English-dominated corpora use English as an internal pivot language—-a question of key importance for understanding how language models function and the origins of linguistic bias. Focusing on the Llama-2 family of transformer models, our study is based on carefully constructed non-English prompts with a unique correct single-token continuation. From layer to layer, transformers gradually map an input embedding of the final prompt token to an output embedding from which next-token probabilities are computed. Tracking intermediate embeddings through their high-dimensional space reveals three distinct phases, whereby intermediate embeddings (1) start far away from output token embeddings; (2) already in middle layers allow for decoding a semantically correct next token, but giving higher probability to its version in English than in the input language; (3) move into an input-language-specific region of the embedding space. We cast these results into a conceptual model where the three phases operate in ”input space”, ”concept space”, and ”output space”, respectively. Crucially, our evidence suggests that the abstract ”concept space” lies closer to English than to other input languages, which may have important consequences regarding the biases embodied by multilingual language models.

## introduction most modern large language models (llms) are trained on massive corpora of mostly english text @xcite . despite this, they achieve strong performance on a broad range of downstream tasks, even in non-english languages @xcite . this raises a compelling question: how are llms able to generalize * equal contribution. figure 1 : illustration of logit lens, which applies language modeling head (here, llama-2-7b) prematurely to latent embeddings in intermediate layers, yielding one next-token distribution per position (x-axis) and layer (y-axis). we show final tokens of translation prompt (cf. sec. 3.3) ending with "français: "fleur" -中文: "" (where "中文" means "chinese"). final layer correctly ranks "花" (translation of "fleur") on top, whereas intermediate layers decode english "flower". color indicates entropy of next-token distributions from low (blue) to high (red). (plotting tool: @xcite .) so well from their mainly english training data to other languages? intuitively, one way to achieve strong performance on non-english data in a data-efficient manner is to use english as a pivot language, by first translating input to english, processing it in english, and then translating the answer back to the input language. this method has been shown to lead to high performance when implemented explicitly @xcite @xcite . our guiding inquiry in this work is whether pivoting to english also occurs implicitly when llms are prompted in non-english. in the research community as well as the popular press, many seem to assume that the answer is yes, epitomized by claims such as, "the machine, so to say, thinks in english and translates the conversation at the last moment into estonian" @xcite . in this work, we set out to move beyond such speculation and investigate the question empirically. the question is of major importance. on the one hand, implicitly using english as an internal pivot could bias llms toward anglocentric patterns that could predispose the model to certain linguistic elements (lexicon, grammar, metaphors, etc.), while also shaping more profound behaviors related to, e.g., emotional stance @xcite or temporal reasoning @xcite . on the other hand, if llms do not use english as a pivot, it raises questions of how else they manage to work so remarkably well even in low-resource languages. overall, the quest for an internal pivot language holds promise to advance our understanding of how llms function no matter if we succeed. investigating the existence of an internal llm language is complicated by the scale and notoriously inscrutable nature of the neural networks behind llms, which after the input layer do not operate on discrete tokens, but on high-dimensional floating-point vectors. how to understand if those vectors correspond to english, estonian, chinese, etc.-or to no language at all-is an open problem, and the question of whether llms use an internal pivot language has therefore, to the best of our knowledge, not been addressed empirically before. ## related work multilingual language models. multilingual language models (lms) are trained to simultaneously handle multiple input languages. examples include mbert @xcite , mbart @xcite , xlm-r @xcite , mt5 @xcite , xglm @xcite , mgpt @xcite , bloom @xcite , and polylm @xcite . current frontier models such as gpt-4, palm, and llama-2, despite performing better in english due to their anglocentric training data @xcite @xcite , still do well across languages @xcite . researchers have devised numerous methods for efficiently transferring lm capabilities across languages, e.g., by aligning contextual embeddings @xcite , relearning embedding matrices during finetuning on a new language @xcite , or repeatedly doing so during pretraining @xcite . several approaches leverage english as a pivot language. for instance, @xcite show that llama can be efficiently augmented with multilingual instruction-following capabilities thanks to its english representations. likewise, @xcite demonstrate the feasibility of leveraging language models' proficiency in english for non-english contexts by fine-tuning them on translation data and english-only instructional data. they successfully employ this approach to enhance the multilingual reasoning capabilities of llama-2. regarding non-latin low-resource languages, @xcite illustrate that leveraging both romanized and english data proves to be an effective strategy for efficiently improving multilingual task performance. prompting strategies, too, can improve multilingual performance by leveraging english as a pivot language, e.g., by simply first translating prompts to english @xcite @xcite or by instructing lms to perform chain-of-thought reasoning @xcite in english @xcite . although employing high-resource languages can enhance performance on low-resource languages, it might also bias output generation in low-resource languages, e.g., in terms of grammar @xcite . researchers have also investigated how latent representations differ across languages within multilingual models. in the case of encoder-only models such as mbert, converging evidence suggests the existence of a language-agnostic space in later layers following language-specific early layers @xcite @xcite . ## results when presenting results, we first (sec. 4.1) take a probabilistic view via the logit lens (sec. 3.2), for all tasks and all model sizes. (since the results are consistent across languages, we focus on chinese here and refer to appendix b for french, german, and russian.) then (sec. 4.2) we drill deeper by taking a geometric view of how token embeddings drift as the transformer computes layer by layer. ## conceptual model next, we formulate a conceptual model that is consistent with the above observations. in order to predict the next token, the transformer's job essentially consists in mapping the input embedding of the current token to the output embedding of the next token. phase 1 is focused on building up a better feature representation for the current token from its input embedding, by dealing with tokenization issues (e.g., integrating preceding tokens belonging to the same word), integrating words into larger semantic units, etc. this phase is not yet directly concerned with predicting the next token, with latents remaining largely orthogonal to output token space (low token energy), leading to small dot products between latents and output token embeddings, and thus to high entropy. in phase 2, latents live in an abstract "concept space", which, unlike in phase 1, is no more orthogonal to the output token space. rather, latent "concept embeddings" are closer to those output token embeddings that can express the respective concept (across languages, synonyms, etc.), leading to low entropy. among the concept-relevant tokens, english variants lie closer to the concept embedding than non-english variants (due to the model's overwhelming exposure to english during training), leading to higher probabilities for english than chinese tokens. despite the correlation between concept and token embeddings, concept embeddings also carry much information that goes beyond output tokens (including input-specific contextual information and information about the target language), leading to a still-low token energy. in phase 3, the model maps abstract concepts to concrete words/tokens in the target language. information that is irrelevant for next-token prediction is discarded, leading to a spike in token energy. sketch. this model is illustrated-with a strongly simplified toy-like sketch-in fig. 4(d) . in this picture, the model operates in 3d (rather than the actual 8192d) space. all embeddings (output tokens and latents) lie on a sphere around the origin. token embeddings lie on the equator and are mostly spread out along the x-axis (left/right), which captures language (english left, chinese right). the y-axis (front/back) captures concepts, in this toy picture along a 1d "sweetness" scale. the z-axis (bottom/top) provides an extra degree of freedom that can be used to store information about context, language, etc. a transformer forward pass moves along the surface of the sphere. in phase 1, the latent starts out at the north pole, orthogonal to both output token and concept embeddings. phase 2 rotates the latent into concept space; english tokens are more likely because their embeddings have a stronger concept component y. finally, phase 3 rotates the latent along the equator into the target language's hemisphere, onto the output token that best captures the active concept in that language. ## discussion in our attempt to answer whether llama-2 models internally use english as a pivot language, we found that latent embeddings indeed lie further from the correct next token in the input language than from its english analog, leading to overwhelmingly english internal representations as seen through the logit lens. it might thus be tempting to conclude that, yes, llama-2 uses english as an implicit pivot, similar to researchers' prior use of english as an explicit pivot @xcite @xcite . but our answer must be more nuanced, as much of the latents' "energy" points in directions that are largely orthogonal to output token embeddings and thus do not matter for next-token prediction. the model can use these directions as extra degrees of freedom for building rich feature representations from its raw inputs @xcite @xcite , which could be seen as forming an abstract "concept space". in this interpretation, the model's internal lingua franca is not english but conceptsconcepts that are biased toward english. hence, english could still be seen as a pivot language, but in a semantic, rather than a purely lexical, sense. our experiments involve three text completion tasks. the translation and cloze tasks operate at a semantic level, whereas the word repetition task is purely syntactic. yet, in most languages (fig. 7 ) the pattern is similar to that for the two other tasks, with tokens first going through an "english phase"possibly because recognizing that the task is to simply copy a token requires semantic understanding, which is achieved only in concept space, which in turn is closer to english token embeddings. this said, note that the english-first pattern is less pronounced on the repetition task (fig. 7 ), where the input language rises earlier than on the other tasks or, for chinese (fig. 7(e )) even simultaneously with, or faster than, english. this might be due to tokenization: for chinese we explicitly chose 100% single-token words, as opposed to only 13% for russian, 43% for german, and 55% for french ( future work should investigate in what ways an english bias in latent space could be problematic, e.g., by biasing downstream model behavior. we see promise in designing experiments building on work from psycholinguistics, which has shown that concepts may carry different emotional values in different languages @xcite and that using one word for two concepts (colexification) may affect cognition @xcite . future work should also study how english bias changes when decreasing the dominance of english during training, e.g., by applying our method to llama-2 derivatives with a different language mix @xcite @xcite , or by using less anglocentric tokenizers. such work will give important clues for decreasing english bias and enabling more equitable ai.