The Hidden Space of Transformer Language Adapters (2024)

Abstract: We analyze the operation of transformer language adapters, which are small modules trained on top of a frozen language model to adapt its predictions to new target languages. We show that adapted predictions mostly evolve in the source language the model was trained on, while the target language becomes pronounced only in the very last layers of the model. Moreover, the adaptation process is gradual and distributed across layers, where it is possible to skip small groups of adapters without decreasing adaptation performance. Last, we show that adapters operate on top of the model’s frozen representation space while largely preserving its structure, rather than on an isolated subspace. Our findings provide a deeper view into the adaptation process of language models to new languages, showcasing the constraints imposed on it by the underlying model and introduces practical implications to enhance its efficiency.

## introduction the adaptation of pre-trained transformer-based language models (lms) to new languages has become a widely adopted practice in natural language processing (nlp). a prominent approach for achieving this is to train small feed-forward network neural modules, called adapters, on top of the lm layers, while freezing the lm parameters @xcite . to successfully adapt an lm to a new language, adapters should introduce changes that steer the prediction but still can be processed by subsequent layers. adapters have demonstrated impressive capabilities in zero-shot cross-lingual settings @xcite @xcite , multi-task learning @xcite , and integrating knowledge into pre-trained lms @xcite . however, despite this vast success, how lm predictions are being adapted internally is largely unknown. nonetheless, a better understanding of the internal operation of lm adapters would be valuable to inform language selection for multilingual pre-training and for designing more effective adaptation approaches. in this work, we tackle the question of how language adapters work, while focusing on lm predictions adapted from one or more languages (source) to another language (target). for our study, we train an auto-regressive decoder-only lm from scratch on english and then adapt it separately to four different target languages: german, french, hebrew and arabic ( §2). to demonstrate the generality of our findings, we additionally experiment with bilingual models as well as mbert @xcite , a massively multilingual lm pre-trained on 104 languages. first, we consider the sequence of hidden representations across the layers as an information stream being evolved through additive updates @xcite , and analyze the adapters' updates to the prediction ( §3). we find that throughout most of the inference pass, the prediction is evolved in the source language and only in the very last layers the target language abruptly becomes pronounced. then, we explore the contributions of individual adapters ( §4) and observe that adapters introduce gradual updates, which often can be canceled without any decrease in performance, while the last few adapter layers are critical for adaptation success. based on these findings, we explore two alternative hypotheses for how adapters interact with the underlying frozen lm ( §5). either adapters operate in an "isolated" subspace of the representation space that is unused for predictions in the source language, or they operate on top of the model's representation space, while preserving its structure. we test the first hypothesis by training sparse probing classifiers to detect features that change the most during adaptation and then intervene on these features. we observe that while the identified features indeed deteriorate adaptation performance upon intervention, intervening on random features also leads to substantial drops in adaptation performance, and therefore refute the first hypothesis. to test the second hypothesis we analyze principal components in the representation space of the monolingual base model as well as the adapted models. for different properties such as part-ofspeech, number, and tense, we observe a strong alignment between the original representation space and its adapted counterpart. in summary, we provide novel insights into the prevalent language adaptation process of pretraining lms, showing that adapted predictions are gradually evolved on top of the representation space of the source language, while being shifted towards the target language only at the end of the computation and that adapters operate on top of the existing structure of the pre-trained model's representation space. our work not only provides a first step towards a better understanding of language model adaptation but also opens interesting avenues for future work on making language adaptation more efficient. ## experimental setup we employ a controlled setting of pre-training our own lms from scratch, and then adapting them to a target language by training language adapters @xcite . this is important because some of our experiments require language identification based on sub-tokens. achieving this with existing multilingual lms is challenging due to the diverse languages on which the models and their tokenizers have been trained. where possible, we extend our experiments to mbert @xcite , a multiligual lm trained on 104 languages. models as our base model, we pre-trained an auto-regressive decoder-only lm on english (en) texts. our model follows the gpt-neox architecture @xcite with @xmath0, 16 attention heads per layer, and a hidden dimensionality of 1024. we used a vocabulary size of 250k 2 , which is large enough to support multiple languages, and trained a bpe tokenizer @xcite from scratch 2 resulting in a total of 814m trainable parameters. on a combination of sentences sampled from these languages. for our experiments in §3, we additionally train a bilingual model on english and a small fraction of german data. ## adapted predictions evolve in the source language adapters operate on top of the computation of the frozen pre-trained lm, introducing changes that the subsequent layer does not "expect" since its parameters were not updated during the adapter training. this raises the question of whether the adapted model "thinks" in the distribution of the source language(s) it predominantly saw during pre-training and translates its predictions during adaptation or whether it operates exclusively in the target language it was adapted to. experiment to address this question, we inspect the adapted lm predictions across layers via projection to the vocabulary space @xcite . we feed examples in each target language into the model and extract the hidden representations created during inference at the last position, from which the next-token prediction is obtained. then, we project each representation to the vocabulary by multiplying it by the lm head, and apply the softmax function to obtain an intermediate output distribution for every hidden representation. for each example we analyze in which language -the source language the lm was trained on or the target languagethe lm constructs its adapted prediction. this is done by taking the top-k tokens with the highest probabilities in each layer, and categorizing them to their respective languages. we perform this classification based on the script for ar and he and based on token statistic for en, de, and fr (see appendix c for details). results figure 1a shows the portion of predicted tokens from the target language with @xmath1, across the layers of a monolingual english model adapted to each of the target languages indepen-dently. 5 for all target languages, we observe that starting from the second layer and up until the last two layers, the presence of target language tokens remains notably limited, suggesting that the vast majority of the layers promotes tokens in the pretraining language rather in the target language. in contrast, for the last two layers we observe a dramatic increase of target language tokens presence in the top of the output distribution. figure 1b shows a similar trend when adapting with lora instead. we additionally pre-trained a model on 1% of the german data together with all of the english data. we adapt this model to all target languages and repeat our previous analysis. figure 1c shows the result. as expected, the fraction of german tokens is now higher, even for the middle layers. overall, however, we still see the same pattern as before, with most of the predictions being in the source language except for the last two layers. ## two adaptation hypotheses our experiments so far have analyzed how lm adapters influence the predictions of the underlying model, making several key observations: 1) adapted lm predictions are evolved in the source language distribution; decoding the output distribution from most hidden layers shows that source language tokens are substantially more pronounced than target language tokens, until reaching the very last layers where the target language abruptly becomes pronounced; 2) the adaptation process is gradual across most layers, as it is possible to skip multiple adapters without decrease in performance, while the last few adapters are crucial for adaptation success; 3) the adaptation of our pretrained model is notably better for french and german than for hebrew and arabic. this is reflected in the larger norm of the adapter updates for these two languages and the bigger impact on perplexity from these findings, we conclude that the adaptation process during the forward pass is distributed across all layers, with each individual layer-except the last few layers-making gradual updates to the overall adaptation. in what follows, we set out to investigate two alternative hypotheses that could explain the interplay between the adapter updates and the underlying prediction space. hypothesis 1: adapters operate in an isolated subspace we hypothesize that adapters operate in a specific subspace of the model's residual stream which is not utilized by the underlying model, leaving most of the representation space untouched. this isolated representation space then becomes pronounced in the last few layers, leading to the prediction of tokens in the target language. hypothesis 2: adapters operate on top of the model's representation space an alternative hypothesis is that adapters operate on top of the structure in the underlying model's representation space, preserving its overall structure while gradually pushing representations closer to the embedding space of the target language. ## related work language adaptation various efforts have been aiming to extend the capabilities of pre-trained lms to previously unseen languages @xcite @xcite @xcite . while prior approaches have largely relied on continued or adaptive pre-training which updates all the model's parameters @xcite @xcite , more recent approaches rely on parameter efficient methods @xcite @xcite and have been shown to be effective across many languages. our work seeks to provide a better understanding of the adaptation process via adapters, which could potentially lead to concrete ways to improve it. ## conclusion and discussion we experiment with language adaptation applied to mono-, bi-, and multilingual pre-trained lms and study how language adapters interact with the underlying model. we show that adapted lm predictions are mostly evolved in the distribution of the source language(s) the model saw during pre-training and that the adaptation is gradually happening on top of the existing representation structure of the underlying models. we not only provide a unique perspective on the inner-working of language adaptation but our findings also open up several interesting avenues for future work on language adaptation. future research on designing more efficient adaptation approaches could build on our findings on the relationship between the "ease of adaptation" and source-target language similarity, to automatically reduce the number of adapters during inference and to study the transfer of adapters across languages. our insights on the alignment between adapters and the underlying representation space could be extended to investigate the extend to which this alignment restricts the amount of adaptation possible informing studies on alternative ways of adaptation which are less constrained by the existing structure in the pre-trained model, which might lead to better performance on languages that are more distant from the source language.