How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoders (2025)

Abstract: This study explores how bilingual language models develop complex internal representations.We employ sparse autoencoders to analyze internal representations of bilingual language models with a focus on the effects of training steps, layers, and model sizes.Our analysis shows that language models first learn languages separately, and then gradually form bilingual alignments, particularly in the mid layers. We also found that this bilingual tendency is stronger in larger models.Building on these findings, we demonstrate the critical role of bilingual representations in model performance by employing a novel method that integrates decomposed representations from a fully trained model into a mid-training model.Our results provide insights into how language models acquire bilingual capabilities.

## introduction large language models (llms) have demonstrated remarkable multilingual capabilities @xcite @xcite . however, it is not yet clear how such capabilities emerge during pre-training. specifically, do llms initially learn each language separately before aligning them? is cross-lingual alignment distributed across layers or concentrated in specific components? how does model size affect this alignment process? these are not just theoretical questions; they directly impact our understanding of model scalability and the emergence of generalization abilities @xcite . to address these questions, in this study, we explore the internal mechanisms through which llms develop their internal representations; namely, we trace when, where, and how bilingual alignment (english-japanese) emerges during pretraining. for this purpose, we use sparse autoencoders (saes; @xcite as a tool for our analysis, which enable us to extract interpretable latent features from hidden representations. unlike previous approaches @xcite @xcite , our method captures fine-grained distinctions between languagespecific and bilingual features, as well as semantic features, and allows analysis of their emergence across training stages and model layers. we conduct experiments on decoder-only models with a variety of sizes, pretrained on an english-japanese bilingual corpus. our observations highlight three key findings, as illustrated in figure 1 . beyond these observations, we introduce an sae-based method to identify which types of representations are most essential to the model. we first decompose the representations of a fully trained model into three distinct types: english-specific, japanese-specific, and bilingual. these components are then selectively injected into the model at a mid-training stage, allowing us to evaluate their importance by analyzing the resulting changes in the model's behavior. our results demonstrate that bilingual representations from a fully trained model enhance the performance of a mid-training model (section 5). beyond simply using saes to interpret language models, we harnessed them to directly manipulate internal representations, demonstrating their versatility as tools for both analysis and intervention. we believe that our approach can be further extended to investigate beyond bilinguality in language models, providing valuable insights to the broader research community. ## sparse autoencoders a sparse autoencoder (sae) is an autoencoder that enforces a sparsity constraint on its hidden layer. in this study, we adopt a variant called topk-sae @xcite , where the topk activation function is applied at the hidden layer. compared to a relu-based sae @xcite , topk-sae has been shown to be easier to train while maintaining sparsity and achieving higher reconstruction performance @xcite . let @xmath0. the encoder e and decoder d are defined as follows: the training objective is the following mean squared error (mse) loss: in this study, we control topk-sae by two hyperparameters: n, the dimension of the hidden layer, and k, the number of hidden dimensions to keep active. interpreting w dec as n distinct vectors in r d , topk-sae can be seen as selecting k vectors from n and using their weighted sum to reconstruct the input. in this study, we denote each dimension of the encoder output e(x) ∈ r n as a feature. we say the feature is activated when it is selected during the topk operation (i.e., utilized in reconstruction). ## experiments in this section, we describe our experimental setup for analyzing the internal representations of bilingual language models using saes. we detail the language models, datasets, and sae training procedure (section 3.1); the procedure to find activation patterns of individual features (section 3.2), and the evaluation of language and concept selectivity of individual features (sections 3.3 and 3.4). ## observations we first examine the internal representations of the model by analyzing the distribution of each sae's features trained on various checkpoints (section 4.1), layers (section 4.2), and model sizes (section 4.3). ## intervention we hypothesize that bilingual representations, which correspond to bilingual features, play a crucial role in the performance of a fully trained model. if this is true, integrating these representations into a mid-training model should significantly enhance its performance. to test this, here, we extract bilingual representations from a fully trained model using a topk-sae and inject them into the intermediate representations of a mid-training model. this process is illustrated in figure 10 . ## related work understanding the internal mechanisms of llms has become a major focus of the research community. recent studies show neural networks can represent more features than their dimensions @xcite . to disentangle these representations, saes have emerged as a key tool for decomposing them into interpretable components @xcite . while early work primarily focused on a single sae, recent studies have shifted toward comparing sae features across layers @xcite , model architectures @xcite , or fine-tuning stages @xcite . @xcite concurrently tracks feature formation during training, but lacks quantitative evaluation. another line of research has explored the multilingual capability of language models. @xcite explored the formation of multilingual capabilities through neuron-level analysis and showed that as models become larger and training progresses, they exhibit an increasing degree of multilingual understanding. this result aligns with our sae-based analysis results. @xcite identified neurons shared across languages and tasks, while @xcite and @xcite highlighted language-specific neurons, demonstrating their impact on model performance and language output. our research builds on these foundations and contributes to them in three key ways: (1) we investigate the formation process of bilingual capabilities within a bilingual language model, (2) we conduct a comparative analysis across training stages, model sizes, and layers, and (3) we exmploy saes to perform direct interventions on bilingual representations, offering novel insights on the dynamics of bilingual representation in language models. ## conclusion in this study, we investigated the evolution of internal representations in language models using saes. our analysis revealed that bilingual language models initially learn languages independently and later develop bilingual alignments, particularly in the mid-layers of larger models. we further demonstrated the importance of bilingual representations by conducting targeted interventions with saes. beyond using saes solely for interpreting language models, we leveraged them to manipulate internal representations, showcasing their potential as a tool for both analysis and intervention. we believe that our approach can be extended to explore beyond analyzing the bilinguality of language models and offer valuable insights for the broader research community.