Unveiling Multilinguality in Transformer Models: Exploring Language Specificity in Feed-Forward Networks (2023)

Abstract: Recent research suggests that the feed-forward module within Transformers can be viewed as a collection of key-value memories, where the keys learn to capture specific patterns from the input based on the training examples. The values then combine the output from the ‘memories’ of the keys to generate predictions about the next token. This leads to an incremental process of prediction that gradually converges towards the final token choice near the output layers. This interesting perspective raises questions about how multilingual models might leverage this mechanism. Specifically, for autoregressive models trained on two or more languages, do all neurons (across layers) respond equally to all languages? No! Our hypothesis centers around the notion that during pre-training, certain model parameters learn strong language-specific features, while others learn more language-agnostic (shared across languages) features. To validate this, we conduct experiments utilizing parallel corpora of two languages that the model was initially pre-trained on. Our findings reveal that the layers closest to the network’s input or output tend to exhibit more language-specific behaviour compared to the layers in the middle.

## introduction one of the least studied aspects of the transformer @xcite models in general and large language models (llms) in particular is the feed-forward layers (ffns). although they contain almost two-thirds of the parameters, it is only recently foot_0 that their role in the working of the models is being seriously studied. @xcite have earlier demonstrated that ffns could be seen as "key-value memories" where figure 1 : transformer block and the structure of ffn each neuron (key) foot_2 in the lower sub-layer of the ffn gets triggered by specific patterns in the input data and the higher sub-layer (values) produces a distribution over the output vocabulary. this leads us to a perspective (figure 1 ) where the ffn first captures certain patterns or concepts foot_3 in the input (conceptualization), selects the important aspects (using the activation function i.e. selection) and then combines them to emit an output which can be interpreted as a prediction of the possible next-word token for that layer, i.e. synthesis. to highlight this view throughout the rest of the paper, we will use the term 'detectors' instead of the rather generic 'keys' to refer to the neurons in the earlier layer and 'combinators' instead of 'values' to refer to the later layer. repeating this across layers leads to a process of incremental prediction of the next token, with the prediction from previous layers being refined in the next layers @xcite . this perspective however raises an important question. for models trained with a causal-language modeling objective in multilingual settings, what sort of patterns do the detectors encode across layers? more precisely, are some detectors triggered by input only from specific languages? in this paper, we investigate this phenomenon of language specificity of the detectors in a multilingual model, pretrained on 30 languages from 16 language families. earlier work has shown that transformer models encode more shallow features in the earlier layers 4 while encoding more semantic features in the later layers 5 @xcite . we hypothesise that the shallow processing would require more language-specific detectors than the semantic aspects of the input. and hence, we posit that during pretraining of the multilingual models, two kinds of neurons would emerge: language-specific and language-agnostic. thorough investigations into the role of the ffn layers in transformer is an interesting research direction, and to our best knowledge, this is the first work that tries to look at the ffn 6 from the perspective of multilinguality. the rest of the paper is structured as follows: a brief discussion of the related works (section 2) is followed by the description of the models and data (section 3) and models (section 4). this is followed by the presentation (section 5) and simultaneous discussion of the results (sections 6 and 7). ## related work exploring the role and capabilities of the ffn sub-layer in transformer models is a still nascent field of research with only a few papers exploring their working. as mentioned earlier, @xcite have proposed an interesting perspective of looking at how the ffn layer of the transformer contributes during language generation. recent work @xcite exploring the capabilities of the ffn has also looked into how the activations of ffns could be used for understanding how autoregressive models deal with facts. other works @xcite have analysed activation patterns in ffns to study sparsity in transformers. in other words, they show that only a few neurons in the ffns are activated corresponding to inputs to transformers. on the front of studying multilingual models, @xcite demonstrated that representations in encoder-only models can be split into language specific and language-neutral components. but to our best knowledge, no equivalent study has been done for autoregressive language models. additionally, deshpande et al. (2022); blevins et al. (2022); lauscher et al. (2020); choudhury and deshpande (2021 from the perspective of studying multilinguality in 4 close to the input 5 near the output 6 in a decoder-only transformer model the human brain, neuroimaging studies @xcite @xcite have shown that although neural circuits for different languages are highly overlapping, there are distinct brain areas for language-specific processing and areas that are language-agnostic. ## model and testing data we use a pretrained xglm model @xcite with 1.7 billion parameters, available on the hugging face @xcite ) repository foot_4 for our experiments. we use sentences from the training data of the czeng 2.0 corpus foot_5 @xcite for our experiments. the model description of the xglm model states that the model was trained on commoncrawl data of various languages. czeng heavily relies on various freely accessible web sources and a part of the data included in czeng is also drawn from commoncrawl among other sources. thus, we expect that the sentences used for the experiments are of the same domain/style as the model was originally trained on, and they can even overlap. we do not consider such a possible overlap a serious problem for our analysis, because we are not measuring any processing performance or generalization capability. ## experiment we first extract a sample of sentences from the czeng corpus, giving us a set of czech and english parallel sentences. we only select sentences with lengths between 20 and 50. we then feed the model with all 'prefixes' of the sampled sentences from both languages. in other words, for each sentence, we incrementally feed the model one subword at a time and record our observations. for instance, for a czech sentence like "tenhle úkol je obtížný" (this task is difficult), the prefixes fed to the model would be "tenhle", "tenhle úkol", "tenhle úkol je" and "tenhle úkol je obtížný". the parallel sentences ensure that the semantic contents of the sentences for the two languages are similar. we go on to collect the data about the model state corresponding to each prefix. ## observations as an example, in the following sections, we present the results from our observations of the selection coefficients of detectors across the layers of the model. ## discussion we started with the hypothesis that language-specific detectors would be more common in the layers closer to the input and output. we analysed the detectors across the layers using sentences from a czech-english parallel corpus. we note that in the underlying xglm model, english (with 803,527 million training tokens) was much more dominant than czech (with 8,616 million training tokens) @xcite . we thus consider the model to be a primarily english model that saw some czech sentences during pretraining. from the results, we observe that the layers closer to the input and output indeed perform more language specific processing than others. we also see that considerably lower number of detectors are triggered by the czech prefixes than english prefixes, probably reflecting the data imbalance during training. while looking at the behaviour of czech-specific detectors, we find that their numbers drop near the middle layers (8-15). we know that the model is primarily english centric. and since it is well known that higher-layers of transformers are involved in more semantic processing, it is likely that the model uses more language-agnostic detectors and only a few czech-specific detectors for processing semantic aspects of the input. studies with humans have previously shown that semantic processing in humans is often language-agnostic. we thus see a possible way to connect these observations in the future. from a different perspective, the analysis of the selection coefficients also agrees with the recent theories and observations about the sparse nature of ffn modules. we hypothesise that the sparsity (lesser numbers of unique detectors) might be an indicator of shallow processing and density might be an indicator of semantic processing. the sparsity argument might also be extended to claim that only a subset of detectors are required for language specific processing while greater numbers of detectors are required for more languageagnostic (i.e. semantic) however, such claims warrant extensive experimentation that we wish to conduct as a followup to this work. ## conclusion in this study, we focused on the analysis of the feed forward layers (ffns) of a pretrained multilingual transformer model. we look at the ffns as a system that first identifies patterns in the input representations (detector), selects the relevant information (selector), and then combines it to make a guess of the next token (combiner). we assess the degree of language specificity of the detectors in this multilingual model with two experiments. we observe that there are greater number of language specific detectors near the input and output of the model. additionally, we observe how data imbalance during training is reflected in the behaviour of the multilingual detectors. we also try to link our observations with recent studies on the sparse activations in ffns. overall, our findings shed light on the language specificity of ffns in multilingual models.