Venue: NAACL 2024

On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neurons

Takeshi Kojima, Itsuki Okimura, Yusuke Iwasawa, Hitomi Yanaka, Yutaka Matsuo

The University of Tokyo

t.kojima@weblab.t.u-tokyo.ac.jp

###### Abstract

Current decoder-based pre-trained language models (PLMs) successfully demonstrate multilingual capabilities. However, it is unclear how these models handle multilingualism. We analyze the neuron-level internal behavior of multilingual decoder-based PLMs, Specifically examining the existence of neurons that fire "uniquely for each language" within decoder-only multilingual PLMs. We analyze six languages: English, German, French, Spanish, Chinese, and Japanese, and show that language-specific neurons are unique, with a slight overlap (\(<5\%\)) between languages. These neurons are mainly distributed in the models' first and last few layers. This trend remains consistent across languages and models. Additionally, we tamper with less than 1% of the total neurons in each model during inference and demonstrate that tampering with a few language-specific neurons drastically changes the probability of target language occurrence in text generation.1

Footnote 1: Code and model-generated texts are available at [https://github.com/kojima-takeshi188/lang_neuron](https://github.com/kojima-takeshi188/lang_neuron)

+
Footnote †: Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 6919–6971

June 16-21, 2024 02024 Association for Computational Linguistics

## 1 Introduction

Recent studies have frequently demonstrated the excellent multilingual abilities of pre-trained language models (PLMs). Some PLMs explicitly mix multilingual language corpus for pre-training Lin et al. (2021); Scao et al. (2022), whereas others mainly use an English-dominant text corpus, with the unintentional inclusion of a low percentage of multiple language texts, which results in the acquisition of multilingual skills, such as Llama2 Touvron et al. (2023). How do they exhibit multilingual abilities?

Prior studies have focused on language-universal neurons activated across multilingual inputs, mainly focusing on encoder-based PLMs Antverg and Belinkov (2022); Stanczak et al. (2022); Chen et al. (2023); Starczak et al. (2023); Varda and Marelli (2023). In contrast to encoder-based models, which might be sufficient to abstract inputs, decoder-based PLMs need to recover the language-specific information in the later part of the generation. Therefore, language-specific processing within these models should be a more complex and essential functionality compared to the encoder-based ones. However, few studies have focused on the existence and activity of language-specific neurons in decoder-based PLMs (See Section 2).

This study examines the behavior of language-specific neurons in decoder-based PLMs. Specifically, we analyze multiple decoder-based PLMs, including XGLM (564M, 1.7B, 2.9B), BLOOM (560M, 1.7B, 3B), and Llama2 (7B, 15B), for six languages (English, German, French, Spanish, Chinese, and Japanese). To investigate language-specific neurons, we adopt an approach proposed by Cuadros et al. (2022), which finds neurons that activate on a certain group of sentences (_Positive_ sentences) but do not activate on other groups (_Negative_ sentences). We treat the target language texts as positive and any other language as negative, iden

Figure 1: Overview of our proposal. (Left) Finding language-specific neurons that tend to be activated for a target language. (Right) Controlling the detected language-specific neurons by forcing their activation during inference to manipulate the probability of target language occurrence.

 

tifying language-specific neurons that statistically activate positive sentences (See Section 3). The experimental results demonstrate that the identified language-specific neurons are mainly distributed in the first and last few layers of the model. This trend remains consistent across multiple languages and model variants. To verify the effect of the neurons, we intervene in language-specific neurons in the model during inference, showing that they can drastically change the probability of the target language occurrence during text generation (See Section 4).

## 2 Related Work

Previous studies analyzed the internal behavior of PLMs for multilingual tasks by observing the activation of their neurons. Several studies have found that language-universal neurons are activated across multilingual inputs in encoder-based PLMs (mainly M-BERT, see Pires et al. (2019)) for various task settings, including syntactic or factual knowledge tasks Antverg and Belinkov (2022); Stanczak et al. (2022); Chen et al. (2023); Starczak et al. (2023). However, studies of encoder-based PLMs have not focused on the identification of language-specific neurons. Mueller et al. (2022); Bau et al. (2019) studied decoder-based language models to find the multilingually shared neurons. Similar to the encoder-based PLMs, limited research has focused on the existence and activity of language-specific neurons in decoder-based language models.

Various methods can be used to identify and control neurons Sajjad et al. (2022). Several studies have identified and intervened in neurons for effective word editing or classification Mu and Andreas (2020); Dai et al. (2022); Mueller et al. (2022); Chen et al. (2023); Varda and Marelli (2023). In contrast, few studies have investigated the identification and intervention of neurons for full-text generation for a desired concept, for example, Bau et al. (2019) for LSTM models and Cuadros et al. (2022) for pre-trained transformer models. Cuadros et al. (2022) have proposed an approach for controlling text generation on Transformer-based decoder models, and proven its effectiveness. Therefore, we use it as an analytical tool in our experiments with decoder-based PLMs.

## 3 Method

We identified and controlled neurons specific to each language based on the approach of Cuadros et al. (2022), with appropriate modifications for our experiments. This approach was originally developed to identify and control neurons that respond to specific word-level concepts, such as homographs or gender biases. However, we aimed to find neurons that grasp broader sentence-level and language-specific concepts; therefore, we modified the original approach for our purpose.

### Finding Language-specific Neurons

First, we prepared text for each language. We considered a set of \(|L|\) languages. For each language \(l\in L\), we prepared \(N_{l}\) texts, which resulted in \(N=N_{1}+...+N_{l}+...+N_{|L|}\) texts for all the languages. Let \(x=\{x_{i}\}^{N}_{i=1}\) be the set of all the texts. Our goal was to find neurons that activate text in the target language \(l_{t}\in L\) but do not activate text in other languages \(L\setminus\{l_{t}\}\). For each text \(x_{i}\in x\), we assigned a label \(b_{i}=1\) if the text was in the target language (i.e., \(l=l_{t}\)); otherwise, \(b_{i}=0\). Therefore, we had

\[N=N_{l_{t}}^{+}+N_{l_{t}}^{-} \tag{1}\]

sentences, where \(N_{l_{t}}^{+}\) positive sentences consisted of texts in the target language \(l\) (i.e., \(b_{i}=1\)) and \(N_{l_{t}}^{-}\) negative sentences consisted of texts in other languages (i.e., \(b_{i}=0\)). For example, if the target language \(l_{t}\) was French, French texts were assigned label 1, and texts in other languages, such as German and Chinese were assigned label 0.

Second, we observed the activation value of each neuron inside the model given the input text. We assigned a unique index \(m\in M\) to each neuron. \(|M|\) denotes the total number of neurons in the model. Let \(z_{m,i}\in z_{m}\) be the output value of neuron \(m\) when text \(x_{i}\in x\) is provided as an input to the model. Here, we explain in detail how this value can be calculated. Specifically, text \(x_{i}\) is composed of a sequence of \(T\) tokens \(x_{i}=\{w_{i,1},...,w_{i,t},...,w_{i,T}\}\). Therefore, given the input text, there exist \(T\) output values \(\{z_{m,i,1},...,z_{m,i,j},...z_{m,iT}\}\) for neuron \(m\) inside the decoder-based Transformer model. We take the average of the \(T\) neuron outputs to summarize the output value of neuron \(m\) for the text \(i\).

\[z_{m,i}=f(z_{m,i,1},...,z_{m,i,t},....z_{m,i,T}), \tag{2}\]

where \(f\) is the aggregation function of the average operator. While the original approach Cuadros et al. (2022) defines \(f\) as a max-pooling operator, our approach defines \(f\) as an average operator to 

identify neurons that consistently activate across tokens for language identification purposes. The output values of the [PAD] token position are excluded from the aggregation as an exception because they are regarded as noise.

Third, language-specific neurons were identified. We regarded the dataset \(\{x_{i},b_{i},z_{m,i}\}^{N}_{i=1}\) as the prediction task samples. Specifically, we regarded texts \(\{x_{i}\}^{N}_{i=1}\) as inputs to the model, labels \(\{b_{i}\}^{N}_{i=1}\) as their ground truth, and the output values of neurons \(\{z_{m,i}\}^{N}_{i=1}\) as the prediction scores of the ground truth. We can measure the performance of neuron \(m\) for the task using its average precision (\(AP_{m}=AP(z_{m},b)\in[0,1]\)), which is the area under the precision-recall curve with different prediction thresholds. We measured \(AP_{m}\) for all neurons and ordered them in descending order.

In the original approach, only the top-\(k\) neurons in descending order were defined as identified neurons. However, this only considers strong positive correlations (i.e., the top-\(k\) highest AP neurons) with labels, leaving out strong negative correlations (i.e., the top-\(k\) lowest AP neurons) with labels. We hypothesize that not only the top-\(k\) neurons but also the bottom-\(k\) neurons are strongly related to a specific language. Therefore, we extended the original approach by considering not only the top-\(k\) neurons but also the bottom-\(k\) neurons, defining them as language-specific neurons. We validate our claim experimentally in Section 4. We set \(k=1000\) as the default value across the experiments. Note that the neurons at the input layer (word embeddings) and output layer (projection layers) were excluded from the measurement because it is clear that these layers consist of language-specific modules: they consist of language-specific characters or (sub-)words.

### Controlling Language-specific Neurons

We controlled text generation by overriding the output values of the top-\(k\) and bottom-\(k\) neurons with fixed values during inference. Specifically, we calculated the fixed value for each neuron \(m\) as follows:

\[\bar{z}_{m}=Median(\{z_{m}|b=1\}). \tag{3}\]

This is the median of the neuron outputs for the target language texts. During inference, we intervened in the top-\(k\) and bottom-\(k\) neurons by replacing their outputs with fixed values in the forward pass and observed whether the models generated texts in the target language.

## 4 Experiment Settings

### Models

XGLM Lin et al. (2021), BLOOM Scao et al. (2022), and Llama2 Touvron et al. (2023) were used in the experiments. XGLM and BLOOM are explicitly referred to as multilingual language models. By contrast, Llama2 was trained almost entirely on an English text corpus, with minimal inclusion of other languages. Table 1 lists the models used in the experiments. All the models were downloaded from HuggingFace Wolf et al. (2019). Table 2 describes the distribution of languages in the pretraining dataset for each model2.

Footnote 2: XGLM information is cited from [https://huggingface.co/facebook/xglm-2.9B](https://huggingface.co/facebook/xglm-2.9B). BLOOM information is cited from [https://huggingface.co/bigscience/bloom#languages](https://huggingface.co/bigscience/bloom#languages).

### Datasets

The following six languages were used in the experiment: English (en), German (de), French (fr), Spanish (es), Chinese (zh), and Japanese (ja). These six languages are frequently targeted in prior studies of multilingual language models; they are among the top seven languages in terms of the percentage of languages included in the XGLM pre-training data Lin et al. (2021). Owing to the limitations of the available computer resources, the number of languages analyzed was limited to six, as described in the limitations section.

To create a language-specific text corpus, we combined two datasets, PAWS-X Yang et al. (2019) and FLORES-200 Costa-jusssa et al. (2022).

\begin{table}
\begin{tabular}{c r r r} \hline Model & \# Params & \# Layers & \# Neurons \\ \hline XGLM & 564M & 24 & 221,184 \\  & 1.7B & 24 & 442,368 \\  & 2.9B & 48 & 884,736 \\ \hline BLOOM & 560M & 24 & 221,184 \\  & 1.7B & 24 & 442,368 \\  & 3B & 30 & 691,200 \\ \hline Llama2 & 7B & 32 & 1,359,872 \\  & 13B & 40 & 2,129,920 \\ \hline \end{tabular}
\end{table}
Table 1: Model list used for the experiments.

\begin{table}
\begin{tabular}{l c c c c c c} \hline  & en & de & fr & es & zh & ja \\ \hline XGLM & 49.0 & 5.4 & 4.7 & 5.3 & 8.1 & 4.0 \\ BLOOM & 30.0 & - & 12.9 & 10.8 & 16.2 & - \\ Llama2 & 89.7 & 0.2 & 0.2 & 0.1 & 0.1 & 0.1 \\ \hline \end{tabular}
\end{table}
Table 2: Distribution of languages in pre-training data.

 

PAWS-X is a dataset for paraphrase identification between two texts for seven languages, including the aforementioned languages. FLORES-200 is a dataset of machine translation tasks for more than 200 languages. The sample sentences in these tasks were of good quality, had a wide variety of text types, and covered the six languages required for our experiments. Therefore, a combination of these factors was used. For this experiment, texts were randomly sampled in a 1:1 ratio from the two datasets to create ground- truth texts for each language.

Following Cuadros et al. (2022), we apply a setting of \(N_{l_{t}}^{-}>N_{l_{t}}^{+}\) to account for the much larger variance of negative than positive examples. Negative samples contain texts from five language, whereas positive samples contain texts from only one language. Specifically, we prepared 500 texts for each language, totaling 3000 texts for all six languages. As Cuadros et al. (2022) pointed out, the choice of \(N_{l_{t}}^{+}\) and \(N_{l_{t}}^{-}\) is arbitrary, usually a tradeoff between the computing resources available and the quality of the representation required. In addition, Cuadros et al. (2022) set the sample sizes of both positive and negative examples between 100 and 1000. Therefore, we considered 500 to be a reasonable value. After specifying a target language, we identified language-specific neurons for the target language using the method described in Section 3.

## 5 Results and Discussion

The experimental results are summarized in this section. See the Appendix for the full results across all models and languages.

### Finding Language-specific Neurons

We identified language-specific neurons using the method described in Section 3. Figure 2 shows histograms of the identified neurons for each layer in each model. Most of the top-1000 neurons with higher AP orders are distributed in the first and last few layers of the model. Similarly, most of the bottom-1000 neurons were distributed in the first and last few layers. In contrast, the middle-1000 neurons (around the median) in AP order were distributed mainly in the middle layers. It was also found that the distributional property was the same across languages, model sizes, and model variants.

In addition, it was confirmed that language-specific neurons that fire in one language have little overlap with those of all other languages. Table 3 shows the cross tables of the number of overlapping neurons between the six languages, indicating an overlap of less than 5 % for every language pair.

The following interpretation is possible regarding the inner behavior of decoder-based PLMs based on the findings in Figure 2 and the prior study of multilingual models Muller et al. (2021), which suggests that encoder-based PLMs process cross-lingual transfer in lower layers and language-agnostic semantic concepts (e.g., task prediction) in higher layers: The first few layers of decoder-based PLMs mainly process cross-lingual transfers to transform the lexical or syntax representations of each language into language-independent semantic representations. The middle layers of the models are primarily language-independent semantic understanding and representation processing. The last few layers of the models translate the semantic representations back into syntax and lexical information for the target language. This interpretation aligns with a recent study Wendler et al. (2024), which suggested that the last few layers of Llama2 models are responsible for the conversion process to a target language.

Figure 2: Distribution of Top, Middle, Bottom-1000 neurons across layers. 1st row:(XGLM-564M, de). 2nd row:(BLOOM-1b7, fr). 3rd row:(Llama2-13b, zh).

\begin{table}
\begin{tabular}{c r r r r r r} \hline  & de & en & es & fr & ja & Zh \\ \hline de & 2000 & 41 & 74 & 39 & 44 & 34 \\ en & 41 & 2000 & 34 & 41 & 49 & 40 \\ es & 74 & 34 & 2000 & 57 & 77 & 22 \\ fr & 39 & 41 & 57 & 2000 & 21 & 93 \\ ja & 44 & 49 & 77 & 21 & 2000 & 27 \\ zh & 34 & 40 & 22 & 93 & 27 & 2000 \\ \hline \end{tabular}
\end{table}
Table 3: The number of overlapping language-specific neurons between languages (XGLM-564M).

 



### Controlling Language-specific Neurons

To show the effectiveness of the identified language-specific neurons, we investigated whether the models could control language in text generation by intervening with language-specific neurons. We conducted the investigation using unconditional and conditional (i.e., machine translation) text-generation settings.

#### 5.2.1 Unconditional text generation

In the experiments on unconditional text generation, we do not provide models with any input prompt, i.e., only a [BOS] token as a prompt. Each model repeated text generation 100 times with random sampling decoding (temperature=0.8, top p=0.9) by changing the random seed from 1 to 100 each time the model started to generate text. Figure 3 illustrates model-generated text examples with the intervention setting in each class of language-specific neurons. It was shown that by changing language-specific neurons for intervention, we can change the language of the output texts.

To quantitatively measure the probability of occurrence of a target language, we classified the generated texts into language categories using the language identification classifier FastText Joulin et al. (2017, 2016). We classified each text into the target language if the classification score exceeded a threshold of 0.5 Wenzek et al. (2020); Touvron et al. (2023) and calculated the probability of the target language occurrence, i.e., the evaluation metric was accuracy.

Figure 4: Model-generated text examples with conditional text generation setting by Llama-2-7b. Given a machine translation task as input, the model generates outputs through a greedy decoding method.

Figure 3: Model-generated text examples with unconditional text generation setting by XGLM-564M. Given a [BOS] token as input, the model generates outputs through a random sampling method.



Table 4 summarizes the results. This demonstrates that intervention in language-specific neurons increases the probability of the target language occurrence in unconditional text generation for each language. In other words, the desired language could be generated by intentionally igniting target neurons. It should be noted that the BLOOM models achieve a high probability of German and Japanese text occurrence by intervention, although the models do not explicitly include German and Japanese in their pre-training datasets, as described in Table 2. It is possible for a small number of these languages to be unintentionally mixed, leading to unintentional ability acquisition. For example, an English text and its translation to language may be present in a single document Briakou et al. (2023).

We conducted a study by intervening in only the top-1000 neurons, only the bottom-1000 neurons, and both groups of neurons. Interestingly, some languages did not respond to control by intervening only in the top-1000 or only the bottom-1000 neurons. This suggests that it is possible to effectively control language by intervening in both groups of neurons. In principle, the top-\(k\) neurons are correlated with positive activation values. Conversely, the bottom-\(k\) neurons were correlated with negative activation values. Figure 5 validates this hypothesis. These findings align with those of Wang et al. (2022), who suggested that neurons with both positive and negative correlations with labels are important for identifying target neurons.

We conducted an ablation study by changing the number of intervening neurons and analyzed its effect on the probability of target language occurrence. Additionally, we verified the quality of each model-generated text using the BLEU-4 score (BLEU). We evaluated BLEU only for texts identified by the language identifier as belonging to the target language. Specifically, for each model-generated text identified as the target language, we set the text as a hypothesis and all positive texts as references, and measured the BLEU score. We av

Figure 5: (Top) Distributional difference of activation values of the top-1000 neurons between target (on) and non-target languages (off). (Bottom) Distributional difference of activation value of the bottom-1000 neurons.

\begin{table}
\begin{tabular}{l l r r r r} \hline \hline  & \multicolumn{3}{c}{before} & after \\ \cline{3-6}  & & & Top & Bottom & Both \\ \hline XGLM & en & 40.0 & 62.0 & 77.0 & **89.0** \\ (564M) & de & 0.0 & 89.0 & 31.0 & **95.0** \\  & fr & 0.0 & 86.0 & 7.0 & **90.0** \\  & es & 2.0 & 71.0 & 5.0 & **78.0** \\  & zh & 7.0 & **82.0** & 50.0 & 79.0 \\  & ja & 7.0 & 92.0 & 61.0 & **99.0** \\  & - & 9.3 & 80.3 & 38.5 & **88.3** \\ \hline BLOOM & en & 37.0 & 78.0 & 67.0 & **88.0** \\ (1b7) & de & 0.0 & 60.0 & 0.0 & **86.0** \\  & fr & 13.0 & 80.0 & 72.0 & **98.0** \\  & es & 18.0 & 44.0 & 94.0 & **97.0** \\  & zh & 6.0 & 1.0 & 89.0 & **90.0** \\  & ja & 0.0 & 67.0 & 35.0 & **97.0** \\  & - & 12.3 & 55.0 & 59.5 & **92.7** \\ \hline Llama2 & en & 83.0 & 82.0 & **89.0** & **89.0** \\ (7b) & de & 0.0 & 2.0 & 6.0 & **23.0** \\  & fr & 2.0 & 1.0 & **8.0** & 7.0 \\  & es & 1.0 & 4.0 & 4.0 & **35.0** \\  & zh & 0.0 & 2.0 & 4.0 & **50.0** \\  & ja & 1.0 & 1.0 & **12.0** & 10.0 \\  & - & 14.5 & 15.3 & 20.5 & **35.7** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Probability of language occurrence within the generated texts before and after intervention. Values in the ”-” rows are the average values across six languages.

Figure 6: Ablation study of text generation by varying the number of neurons for intervention. x-axis: \(\log_{10}(k)\)



eraged the BLEU scores across all model-generated texts that were identified as the target language. Figure 6 shows the results of the intervention in Spanish neurons for XGLM-564M. This shows that increasing the number of intervening neurons up to 1000-10000 (3-4 on the logarithm of 10 in this figure) generally increases the probability of target language occurrence, but increasing beyond that degrades text quality. Eventually, the sentence collapses and both language identification and quality significantly decrease. This tendency exists regardless of language or model variation. See Appendix C.2 for the complete results.

#### 5.2.2 Conditional text generation

In experiments of conditional text generation, models were given machine translation tasks and required to solve them in a zero-shot prompting setting, but with an unusual prompt format: "Translate an English sentence into a target language. English: {source text} Target Language:". In other words, it is a prompt for a translation task that does not concretely specify the target language. The aim of the prompt was to accurately check whether the manipulation of language-specific neurons could lead to the output of the target language. Using this prompt as an input, the models started to generate text using a greedy decoding strategy. For this experiment, we randomly selected 100 machine translation samples from FLORES200, IWSLT2017 [9], and WMT [18, 19, 20, 21]3, respectively. Two evaluation metrics were used to measure translation quality: Accuracy of measuring the probability that the target language text is generated, and BLEU. In the unconditional text-generation setting, we measured the quality of only the generated texts in the target language. However, in the conditional text-generation setting, we calculated the BLEU score between each generated text and the corresponding ground-truth text by following the standard method of BLEU evaluation in machine translation tasks [10].

Footnote 3: We used En \(\rightarrow\) Fr tasks from WMT14, En \(\rightarrow\) De tasks from WMT16, and En \(\rightarrow\) Zh tasks from WMT18.

Table 5 summarizes the experimental results of the conditional text generation. There were two main findings from these results. First, interventions in language-specific neurons tend to increase the probability of producing the target language (accuracy). Second, the translation quality (BLEU) of Llama2 models increased drastically along with accuracy. In contrast, the translation quality of XGLM and BLOOM did not significantly improve compared to the accuracy improvement. We investigated the reason for this by qualitatively analyzing the generated texts. XGLM and BLOOM were forced to output the target languages to some degree via intervention, but the output texts were not

\begin{table}
\begin{tabular}{l l c r r r r r r r r r r r r} \hline \hline  & \multicolumn{4}{c}{FLORES200} & \multicolumn{4}{c}{IWSLT2017} & \multicolumn{4}{c}{WMT} \\ \cline{3-13}  & & Accuracy & BLEU & Accuracy & BLEU & Accuracy & BLEU & Accuracy & BLEU & \\ \hline XGLM-564M & de & 0.0 \(\rightarrow\)**38.0** & 0.0 \(\rightarrow\) & 0.0 & 0.0 \(\rightarrow\)**15.0** & 0.0 \(\rightarrow\) & 0.0 & 0.0 \(\rightarrow\)**17.0** & 0.0 \(\rightarrow\) & 0.0 \\ XGLM-564M & es & 0.0 \(\rightarrow\) & **3.0** & 0.0 \(\rightarrow\) & 0.0 & \(\rightarrow\) & 0.0 & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) \\ XGLM-564M & ja & 0.0 \(\rightarrow\) & 0.0 & 0.0 \(\rightarrow\) & 0.0 & 0.0 \(\rightarrow\) & 0.0 \(\rightarrow\) & 0.0 \(\rightarrow\) & 0.0 & 0.0 \(\rightarrow\) & **1.0** & 0.0 \(\rightarrow\) & 0.0 \\ XGLM-564M & fr & 0.0 \(\rightarrow\) & **1.0** & 0.0 \(\rightarrow\) & 0.0 & 0.0 \(\rightarrow\) & 2.0 & 0.0 \(\rightarrow\) & 0.0 & 0.0 \(\rightarrow\) & **2.0** & 0.0 \(\rightarrow\) & 0.0 \\ \hline BLOOOM-1b7 & de & 0.0 \(\rightarrow\)**56.0** & 1.3 \(\rightarrow\) & 1.3 & 0.0 \(\rightarrow\)**35.0** & 1.0 \(\rightarrow\) & **1.8** & 0.0 \(\rightarrow\) & **37.0** & **2.9** \(\rightarrow\) & 1.7 \\ BLOOOM-1b7 & es & 0.0 \(\rightarrow\) & **2.0** & 1.2 \(\rightarrow\) & 1.2 & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) \\ BLOOM-1b7 & ja & 0.0 \(\rightarrow\) & **6.0** & **0.2** & \(\rightarrow\) & 0.1 & 0.0 \(\rightarrow\) & **8.0** & 0.1 \(\rightarrow\) & **0.2** & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) \\ BLOOM-1b7 & fr & 0.0 \(\rightarrow\) & **16.0** & 1.7 \(\rightarrow\) & **2.8** & 0.0 \(\rightarrow\) & **2.0** & 1.0 \(\rightarrow\) & **1.5** & 0.0 \(\rightarrow\) & **9.0** & 1.7 \(\rightarrow\) & **2.7** \\ BLOOM-1b7 & zh & 0.0 \(\rightarrow\) & **21.0** & **0.3** \(\rightarrow\) & 0.2 & 0.0 \(\rightarrow\) & **3.0** & 0.2 \(\rightarrow\) & **0.3** & 0.0 \(\rightarrow\) & **34.0** & 0.5 \(\rightarrow\) & **0.6** \\ \hline Llama2-7b & de & 0.0 \(\rightarrow\)**66.0** & 2.6 \(\rightarrow\) & **17.7** & 0.0 \(\rightarrow\)**48.0** & 1.2 \(\rightarrow\) & **12.5** & 2.0 \(\rightarrow\) & **53.0** & 5.3 \(\rightarrow\) & **15.2** \\ Llamaa2-7b & es & 4.0 \(\rightarrow\) & **77.7** & 3.3 \(\rightarrow\) & **16.6** & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) \\ Llama2-7b & ja & 0.0 \(\rightarrow\)**58.0** & 0.3 \(\rightarrow\) & 10.4 & 1.0 \(\rightarrow\) & **57.0** & 0.2 \(\rightarrow\) & **4.5** & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) \\ Llama2-7b & fr & 1.0 \(\rightarrow\)**58.0** & 4.1 \(\rightarrow\) & **21.5** & 0.0 \(\rightarrow\) & **32.0** & 1.0 \(\rightarrow\) & **11.1** & 0.0 \(\rightarrow\) & **36.0** & 2.1 \(\rightarrow\) & **13.2** \\ Llama2-7b & zh & 1.0 \(\rightarrow\)**76.0** & 1.0 \(\rightarrow\) & **11.5** & 3.0 \(\rightarrow\) & **82.0** & 0.6 \(\rightarrow\) & **7.8** & 12.0 \(\rightarrow\) & **86.0** & 2.4 \(\rightarrow\) & **11.3** \\ \hline Llama2-13b & de & 0.0 \(\rightarrow\)**22.0** & 1.5 \(\rightarrow\) & **8.8** & 0.0 \(\rightarrow\) & **37.0** & 0.6 \(\rightarrow\) & **10.0** & 4.0 \(\rightarrow\) & **32.0** & 3.3 \(\rightarrow\) & **9.7** \\ Llama2-13b & se & 2.0 \(\rightarrow\)**14.0** & 1.8 \(\rightarrow\) & **4.3** & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) \\ Llama2-13b & ja & 7.0 \(\rightarrow\)**54.0** & 2.4 \(\rightarrow\) & **11.0** & 4.0 \(\rightarrow\) & **75.0** & 0.7 \(\rightarrow\) & **6.1** & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & **6.6** \\ Llama2-13b & fr & 0.0 \(\rightarrow\)**23.0** & 1.6 \(\rightarrow\) & **10.5** & 0.0 \(\rightarrow\) & **9.0** & 0.7 \(\rightarrow\) & **4.7** & 1.0 \(\rightarrow\) & **15.0** & 2.2 \(\rightarrow\) & **6.6** \\ Llama2-13b & zh & 20.0 \(\rightarrow\)**93.0** & 4.4 \(\rightarrow\) & **19.1** & 40.0 \(\rightarrow\) & **96.0** & 5.8 \(\rightarrow\) & **9.6** & 57.0 \(\rightarrow\) & **99.0** & 13.5 \(\rightarrow\) & **18.9** \\ \hline \hline \end{tabular}
\end{table}
Table 5: Results of conditional text generation. Values on the left side of the arrows (\(\rightarrow\)) were measured without intervention on the language-specific neurons; values on the right side were measured during intervention on the neurons. FLORES200 includes translation tasks of English to the other five languages, while IWSLT2017 does not include tasks of English to Spanish, and WMT does not include tasks of English to Spanish or Japanese.

 

related to translation. For instance, when we intervened in German neurons, XGLM tended to output a word "Deutsch". BLOOM tended to generate text unrelated to the translation or simply repeated the source text in English. Conversely, Llama2 tended to output translated text in the correct target language, resulting in improved accuracy and BLEU scores. This experiment showed that intervention in language-specific neurons can guide some models in the right direction, even when the models promptly receive an ambiguous translation task. Figure 4 shows examples of model-generated text for Llama2-7b model. See Section D.2 for additional examples.

We conducted several baseline experiments by changing the prompts to validate the robustness of the model outputs against prompts for the machine translation settings. Specifically, we tried the following four prompts: 1."Translate a sentence from English into a target language. English: {source text} Target Language:", 2."Translate an English sentence into a target language. English: {source text} Target Language:", 3."Translate an English sentence into German. English: {source text} German:", 4."Translate an English sentence into Japanese. English: {source text} Japanese:".

The first and second prompts are ambiguous because they do not explicitly specify the target language. The second prompt is the same as that in Table 5. Regarding the third and fourth prompts, we explicitly describe the target languages in the prompts: German from Western languages and Japanese from Eastern languages. Here, we focus on the Llama-2-7b model because it has significantly improved both accuracy and BLEU scores, as described in Table 5. Similar to the experiment shown in Table 5, we conducted experiments in which the model was asked to solve a translation task under a specified prompt, while intervening in language-specific neurons.

The experimental results are presented in Table 6. The first and second prompts significantly increased the probability of target language occurrence and BLEU scores with intervention in language-specific neurons for all languages. In contrast, the third and fourth prompts caused few changes when we intervened in language-specific neurons for most languages. One possible reason is that explicitly specifying the target language in a prompt automatically fires specific neurons in that language, which may offset the effects of other language-specific neurons.

The only exception was the intervention in Chinese neurons under the fourth prompt "Translate an English sentence into Japanese.", which increases the probability of Chinese text generation and BLEU scores. One possible reason is that some Japanese and Chinese neurons have similar firing patterns within the Llama-2-7b model. As shown in Table 7 in the Appendix, the Llama-2-7b model had a higher language-specific neuron overlap between Japanese and Chinese than the other pairs. As these two languages share many characters with similar surface morphologies, the high similarity between the two languages may have contributed to these results. However, it should be noted that this is not universally true across models; in some cases, the overlap of neurons in this language pair

\begin{table}
\begin{tabular}{l c r r r r r r r r r r} \hline \hline  & \multicolumn{4}{c}{“Translate a sentence} & \multicolumn{4}{c}{“Translate an English sentence} \\  & \multicolumn{4}{c}{from English to a target language.”} & \multicolumn{4}{c}{into a target language.”} \\ \cline{2-10}  & \multicolumn{3}{c}{Accuracy} & \multicolumn{3}{c}{BLEU} & \multicolumn{3}{c}{Accuracy} & \multicolumn{3}{c}{BLEU} \\ \hline de & 0.0 & \(\rightarrow\)**62.0** & 2.8 & \(\rightarrow\)**16.5** & 0.0 & \(\rightarrow\)**66.0** & 2.6 & \(\rightarrow\)**17.7** \\ es & 5.0 & \(\rightarrow\)**78.0** & 4.0 & \(\rightarrow\)**16.5** & 4.0 & \(\rightarrow\)**77.0** & 3.3 & \(\rightarrow\) & **16.6** \\ ja & 0.0 & \(\rightarrow\)**55.0** & 0.3 & \(\rightarrow\)**9.2** & 0.0 & \(\rightarrow\)**58.0** & 0.3 & \(\rightarrow\) & **10.4** \\ fr & 0.0 & \(\rightarrow\)**58.0** & 3.4 & \(\rightarrow\)**21.3** & 1.0 & \(\rightarrow\)**58.0** & 4.1 & \(\rightarrow\) & **21.5** \\ zh & 1.0 & \(\rightarrow\)**79.0** & 1.2 & \(\rightarrow\)**12.7** & 1.0 & \(\rightarrow\)**76.0** & 1.0 & \(\rightarrow\) & **11.5** \\ \hline \multicolumn{10}{r}{ “Translate an English sentence} & \multicolumn{4}{c}{“Translate an English sentence} \\  & \multicolumn{4}{c}{into German.”} & \multicolumn{4}{c}{into Japanese.”} \\ \cline{2-10}  & \multicolumn{3}{c}{Accuracy} & \multicolumn{3}{c}{BLEU} & \multicolumn{3}{c}{Accuracy} & \multicolumn{3}{c}{BLEU} \\ \hline de & 96.0 & \(\rightarrow\)**99.0** & **32.8** & \(\rightarrow\) & 24.4 & 0.0 & \(\rightarrow\)**2.0** & 0.3 & \(\rightarrow\) & **1.2** \\ es & 0.0 & \(\rightarrow\)**1.0** & 2.0 & \(\rightarrow\) & **2.6** & 0.0 & \(\rightarrow\) & **2.0** & 0.1 & \(\rightarrow\) & **0.4** \\ ja & **0.0** & \(\rightarrow\)**0.0** & 0.3 & \(\rightarrow\) & **0.4** & **100.0** & \(\rightarrow\) & 99.0 & **24.3** & \(\rightarrow\) & 19.7 \\ fr & 0.0 & \(\rightarrow\)**3.0** & 2.6 & \(\rightarrow\) & **3.1** & 0.0 & \(\rightarrow\) & **3.0** & 0.2 & \(\rightarrow\) & **1.0** \\ zh & 0.0 & \(\rightarrow\)**2.0** & **0.8** & \(\rightarrow\) & 0.4 & 0.0 & \(\rightarrow\) & **96.0** & 1.3 & \(\rightarrow\) & **14.9** \\ \hline \hline \end{tabular}
\end{table}
Table 6: Results of conditional text generation with different prompt settings for Llama2–7b.

 

is not always high in models other than Llama2, as described in Table 7 in the Appendix.

## 6 Conclusion

This study provides new insights into the activity of language-specific neurons in decoder-based multilingual pre-trained language models: the existence of neurons that fire uniquely for each language. The experimental results demonstrate that language-specific neurons mainly exist in the first and last few layers, regardless of the language, model size, and model variants. We further analyzed the effectiveness of the identified neurons by intervening in the neurons, that is, by replacing the output values with fixed activation values at inference with both unconditional and conditional settings. Using this approach, we can change the probability of the target language occurrence.

We hope that this study facilitates a deeper understanding of decoder-based PLMs and provides new insights for future research on multilingual decoder-based PLMs. Future research should include proposing language-specific model-compression methods. Future research also includes proposing new fine-tuning methods for downstream tasks to facilitate generalization to languages that are not included in the training dataset. For example, only fine-tuning the middle-layer parameters in decoder-based PLMs.

## 7 Limitation

This study only analyzes open models whose parameters were publicly available. It is not possible to analyze closed models with parameters that are not publicly available, such as ChatGPT or GPT-4 OpenAI (2023). Although we focused our analysis on six languages, other languages need to be examined in future studies. Analysis of encoder-decoder-based PLMs, such as mT5 Xue et al. (2021), remains important but is beyond the scope of this study due to the fundamental differences in model architecture from decoder-only PLMs.

## Acknowledgements

We thank three anonymous reviewers for their helpful comments and feedback. This study was partially supported by PRESTO, JST Grant Number JPMPJR21C8, Japan.

## References

* Antverg and Belinkov (2022) Omer Antverg and Yonatan Belinkov. 2022. On the pitfalls of analyzing individual neurons in language models. In _International Conference on Learning Representations_.
* Bau et al. (2019) Anthony Bau, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass. 2019. Identifying and controlling important neurons in neural machine translation. In _International Conference on Learning Representations_.
* Bird et al. (2009) Steven Bird, Ewan Klein, and Edward Loper. 2009. _Natural language processing with Python: analyzing text with the natural language toolkit_. " O'Reilly Media, Inc.".
* Bojar et al. (2016) Ond rej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurelie Neveol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin Verspoor, and Marcos Zampieri. 2016. Findings of the 2016 conference on machine translation. In _Proceedings of the First Conference on Machine Translation_, pages 131-198, Berlin, Germany. Association for Computational Linguistics.
* de Bojar et al. (2018) Ond rej Bojar, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, and Christof Monz. 2018. Findings of the 2018 conference on machine translation (wmt18). In _Proceedings of the Third Conference on Machine Translation, Volume 2: Shared Task Papers_, pages 272-307, Belgium, Brussels. Association for Computational Linguistics.
* Bojar et al. (2014) Ondrej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Specia, and Ale s Tambyna. 2014. Findings of the 2014 workshop on statistical machine translation. In _Proceedings of the Ninth Workshop on Statistical Machine Translation_, pages 12-58, Baltimore, Maryland, USA. Association for Computational Linguistics.
* Briakou et al. (2023)Eleftheria Briakou, Colin Cherry, and George Foster. 2023. Searching for needles in a haystack: On the role of incidental bilingualism in PaLM's translation capability. _arXiv preprint arXiv:2305.10266_.
* Cettolo et al. (2017) Mauro Cettolo, Marcello Federico, Luisa Bentivogli, Niehues Jan, Stuker Sebastian, Sudoh Katsuitho, Yoshino Koichiro, and Federmann Christian. 2017. Overview of the iwslt 2017 evaluation campaign. In _Proceedings of the 14th International Workshop on Spoken Language Translation_, pages 2-14.
* Chen et al. (2023) Yuheng Chen, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. 2023. Journey to the center of the knowledge neurons: Discoveries of language-independent knowledge neurons and degenerate knowledge neurons. _arXiv preprint arXiv:2308.13198_ 

* Costa-Jussa et al. (2022) Marta R Costa-jussa, James Cross, Onur Celebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elaek Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. _arXiv preprint arXiv:2207.04672_.
* Cuadros et al. (2022) Xavier Suau Cuadros, Luca Zappella, and Nicholas Apostoloff. 2022. Self-conditioning pre-trained language models. In _International Conference on Machine Learning_, pages 4455-4473. PMLR.
* Dai et al. (2022) Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 8493-8502, Dublin, Ireland. Association for Computational Linguistics.
* Joulin et al. (2017a) Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Herve Jegou, and Tomas Mikolov. 2017a. Fasttext.zip: Compressing text classification models.
* Joulin et al. (2017b) Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017b. Bag of tricks for efficient text classification. In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers_, pages 427-431, Valencia, Spain. Association for Computational Linguistics.
* Lin et al. (2021) Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, et al. 2021. Few-shot learning with multilingual language models. _arXiv preprint arXiv:2112.10668_.
* Mu and Andreas (2020) Jesse Mu and Jacob Andreas. 2020. Compositional explanations of neurons. _Advances in Neural Information Processing Systems_, 33:17153-17163.
* Mueller et al. (2022) Aaron Mueller, Yu Xia, and Tal Linzen. 2022. Causal analysis of syntactic agreement neurons in multilingual language models. _arXiv preprint arXiv:2210.14328_.
* Muller et al. (2021) Benjamin Muller, Yanai Elazar, Benoit Sagot, and Djame Seddah. 2021. First align, then predict: Understanding the cross-lingual ability of multilingual BERT. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_, pages 2214-2231, Online. Association for Computational Linguistics.
* OpenAI (2023) OpenAI. 2023. GPT-4 technical report. _arXiv_, pages 2303-08774.
* Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Weihg Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In _Proceedings of the 40th annual meeting of the Association for Computational Linguistics_, pages 311-318.
* Pires et al. (2019) Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual BERT? In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 4996-5001, Florence, Italy. Association for Computational Linguistics.
* Sajjad et al. (2022) Hassan Sajjad, Nadir Durrani, and Fahim Dalvi. 2022. Neuron-level interpretation of deep nlp models: A survey. _Transactions of the Association for Computational Linguistics_, 10:1285-1303.
* Le Scao et al. (2022) Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilic, Daniel Hesslow, Roman Castagne, Alexandra Sasha Luccioni, Francois Yvon, Matthias Galle, et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. _arXiv preprint arXiv:2211.05100_.
* Stanczak et al. (2023) Karolina Stanczak, Lucas Torroba Hennigen, Adina Williams, Ryan Cotterell, and Isabelle Augenstein. 2023. A latent-variable model for intrinsic probing. In _Proceedings of the AAAI Conference on Artificial Intelligence_, pages 13591-13599.
* Stanczak et al. (2022) Karolina Stanczak, Edoardo Ponti, Lucas Torroba Hennigen, Ryan Cotterell, and Isabelle Augenstein. 2022. Same neurons, different languages: Probing morphosyntax in multilingual pre-trained models. _arXiv preprint arXiv:2205.02023_.
* Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. _arXiv preprint arXiv:2307.09288_.
* de Varda and Mareli (2023) Andrea Gregor de Varda and Marco Marelli. 2023. Data-driven cross-lingual syntax: An agreement study with massively multilingual models. _Computational Linguistics_, 49(2):261-299.
* Wang et al. (2022) Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, and Juanzi Li. 2022. Finding skill neurons in pre-trained transformer-based language models. In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_, pages 11132-11152, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
* Wendler et al. (2024) Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. 2024. Do Llamas work in English? On the latent language of multilingual transformers. _arXiv preprint arXiv:2402.10588_.
* Wenzek et al. (2020) Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzman, Armand Joulin, and Edouard Grave. 2020. CCNet: Extracting high quality monolingual datasets from web crawl data. In _Proceedings of the Twelfth Language Resources and Evaluation Conference_, pages 4003-4012, Marseille, France. European Language Resources Association.

 

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, et al. 2019. Huggingface's transformers: State-of-the-art natural language processing. _arXiv preprint arXiv:1910.03771_.
* Xue et al. (2021) Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 483-498, Online. Association for Computational Linguistics.
* Yang et al. (2019) Yinfei Yang, Yuan Zhang, Chris Tar, and Jason Baldridge. 2019. PAWS-X: A cross-lingual adversarial dataset for paraphrase identification. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 3687-3692, Hong Kong, China. Association for Computational Linguistics.

## Appendix A Text Examples from Language-Specific Text Corpus

Figure 7 lists text examples from the language-specific corpus, which is a combination of FLORES-200 and PAWS-X.

 

settings in all experiments for unconditional text generation:

* temperature: 0.8
* top-p: 0.9
* maximum output length: 64
* prompt (input token for models):
* XGLM: 
 is automatically set.
* BLOOM: nothing is automatically set. We explicitly set 
.
* Llama2: 
 is automatically set.

## Appendix D Intervention in Neurons for Conditional Text Generation

### Effect of Intervention on Conditional Text Generation

Tables 10, 11, and 12 summarize the probability of target language occurrence in the generated texts before and after the intervention.

### Model-Generated Text Examples For Conditional Setting

* Figure 36 describes a summary of model-generated conditional text examples.
* Figure 37 lists model-generated text examples without any interventions.
* Figures 38, 39, and 40 list model-generated text examples with top-1000 and bottom-1000 neurons intervention.

### Detail Setting of Conditional Text Generation

A greedy decoding method was used for conditional text generation. For evaluation of machine translation, the first line of model-generated text (sentences before the first linebreak code "\({}^{\text{rn}}\)") is used as translated sentences to omit useless subsequent sentences.

The following settings were used in all experiments for conditional text generation:

* maximum output length: 128

## Appendix E Detail Setting of Datasets

To create a language-specific text corpus, we mixed the following two datasets: **dev** split of PAWSX Yang et al. (2019) and **test** split of FLORES-200 Costa-jussa et al. (2022). To create translation tasks for conditional text generation, we randomly sampled tasks from the **devtest** split of FLORES200, **test** split of IWSLT2017 Cettolo et al. (2017), and **test** split of WMT Bojar et al. (2014, 2016, 2018). All datasets were downloaded from HuggingFace Wolf et al. (2019).

## Appendix F Detail Setting of BLEU-4 metrics

We used NLTK library Bird et al. (2009) to measure the BLEU scores for both unconditional and conditional text generation. Specifically, the sentence_bleu function was used with method2 SmoothingFunction option for unconditional text generation. corpus_bleu function was used with method2 SmoothingFunction option for conditional text generation. To enable the comparison of BLEU scores across models, we tokenized all texts using a multilingual tokenizer, XGLM, whose pre-training corpus includes a large proportion of texts in the six target languages Lin et al. (2021).

## Appendix G License

### Model

* XGLM: MIT [link]
* BLOOM: bigscience-bloom-rail-1.0 [link]
* Llama2: Meta license [link]

### Dataset

* PAWS-X: No License (Free to use) [link]
* FLORES200: cc-by-sa-4.0 [link]
* IWSLT2017: cc-by-nc-nd-4.0 [link]
* WMT14: Unknown [link]
* WMT16: Unknown [link]
* WMT18: Unknown [link]

## Appendix H Total computation for Experiments

We executed the experiments mainly for running the inference (both identification and intervention of language-specific neurons) for each model using the following number of A100(40GB) GPUs 

and approximate computing hours per run. We run GPUs 60 times per model (6 languages \(\times\) (1 for identification of language-specific neurons + 6 for unconditional text generation by changing the number of neurons to intervene + 3 for conditional text generation)) for the production run. The computational resource of AI Bridging Cloud Infrastructure (ABCI) provided by the National Institute of Advanced Industrial Science and Technology (AIST) was used for the experiments.

* XGLM 564M: 1GPU \(\times\) 0.5hrs \(\times\) 60 runs.
* XGLM 1.7B: 1GPU \(\times\) 0.5hrs \(\times\) 60 runs.
* XGLM 2.9B: 1GPU \(\times\) 0.5hrs \(\times\) 60 runs.
* BLOOM 560M: 1GPU \(\times\) 0.5hrs \(\times\) 60 runs.
* BLOOM 1.7B: 1GPU \(\times\) 0.5hrs \(\times\) 60 runs.
* BLOOM 3B: 1GPU \(\times\) 0.5hrs \(\times\) 60 runs.
* Llama2 7B: 8GPUs \(\times\) 1hrs \(\times\) 60 runs.
* Llama2 13B: 8GPUs \(\times\) 3hrs \(\times\) 60 runs.

 



###### Contents

* 1 Introduction

[MISSING_PAGE_POST]



Figure 8: Histogram of language neurons across layers in xglm-564M.

 

Figure 9: Histogram of language neurons across layers in xglm-1.7B.

 

Figure 10: Histogram of language neurons across layers in xglm-2.9B.

 

Figure 11: Histogram of language neurons across layers in bloom-560m.

 

Figure 12: Histogram of language neurons across layers in bloom-1b7.

 

Figure 13: Histogram of language neurons across layers in bloom-3b.

 

Figure 14: Histogram of language neurons across layers in Llama2-7b.

 

Figure 15: Histogram of language neurons across layers in Llama-2-13b.

 



\begin{table}
\begin{tabular}{l c r|r r|c} \hline Model & Language & \multicolumn{2}{c}{Top} & Middle & Bottom \\ \hline xglm-564M & en & [0.58, 0.56] & [1.33, 1.45] & [0.68, 0.51] \\ xglm-564M & de & [0.66, 0.61] & [1.29, 1.41] & [0.81, 0.65] \\ xglm-564M & fr & [0.65, 0.53] & [1.4, 1.48] & [0.58, 0.52] \\ xglm-564M & es & [0.51, 0.52] & [1.31, 1.4] & [0.43, 0.53] \\ xglm-564M & zh & [0.65, 0.63] & [1.3, 1.42] & [0.86, 0.66] \\ xglm-564M & ja & [0.62, 0.63] & [1.19, 1.25] & [0.85, 0.78] \\ \hline xglm-1.7B & en & [0.68, 0.48] & [1.29, 1.45] & [0.52, 0.5] \\ xglm-1.7B & de & [0.67, 0.54] & [1.34, 1.57] & [0.8, 0.59] \\ xglm-1.7B & fr & [0.72, 0.52] & [1.29, 1.45] & [0.67, 0.51] \\ xglm-1.7B & es & [0.69, 0.49] & [1.22, 1.43] & [0.6, 0.48] \\ xglm-1.7B & zh & [0.63, 0.5] & [1.24, 1.3] & [0.61, 0.47] \\ xglm-1.7B & ja & [0.56, 0.49] & [1.26, 1.26] & [0.56, 0.56] \\ \hline xglm-2.9B & en & [0.54, 0.36] & [1.28, 1.43] & [0.84, 0.43] \\ xglm-2.9B & de & [0.5, 0.47] & [1.19, 1.52] & [0.99, 0.56] \\ xglm-2.9B & fr & [0.42, 0.46] & [1.21, 1.47] & [0.56, 0.47] \\ xglm-2.9B & es & [0.38, 0.41] & [1.19, 1.5] & [0.46, 0.4] \\ xglm-2.9B & zh & [0.59, 0.39] & [1.21, 1.44] & [0.69, 0.39] \\ xglm-2.9B & ja & [0.5, 0.37] & [1.14, 1.36] & [0.56, 0.4] \\ \hline bloom-560m & en & [0.56, 0.47] & [1.26, 1.22] & [0.48, 0.39] \\ bloom-560m & de & [1.74, 0.66] & [0.94, 1.08] & [1.7, 0.77] \\ bloom-560m & fr & [0.56, 0.48] & [1.2, 1.14] & [0.69, 0.44] \\ bloom-560m & es & [0.52, 0.45] & [1.27, 1.26] & [0.44, 0.4] \\ bloom-560m & zh & [0.69, 0.56] & [1.17, 1.15] & [0.67, 0.52] \\ bloom-560m & ja & [0.46, 0.57] & [1.28, 1.2] & [0.64, 0.68] \\ \hline bloom-1b7 & en & [0.54, 0.48] & [1.26, 1.24] & [0.56, 0.37] \\ bloom-1b7 & de & [2.14, 0.62] & [1.1, 1.2] & [1.22, 0.64] \\ bloom-1b7 & fr & [0.61, 0.47] & [1.16, 1.17] & [0.63, 0.4] \\ bloom-1b7 & es & [0.65, 0.46] & [1.24, 1.18] & [0.64, 0.4] \\ bloom-1b7 & zh & [0.8, 0.52] & [1.27, 1.31] & [0.71, 0.53] \\ bloom-1b7 & ja & [0.8, 0.49] & [1.29, 1.23] & [0.8, 0.63] \\ \hline bloom-3b & en & [0.56, 0.42] & [1.23, 1.23] & [0.56, 0.36] \\ bloom-3b & de & [2.12, 0.65] & [1.08, 1.15] & [1.35, 0.62] \\ bloom-3b & fr & [0.83, 0.43] & [1.27, 1.26] & [0.67, 0.36] \\ bloom-3b & es & [0.95, 0.44] & [1.2, 1.12] & [0.66, 0.36] \\ bloom-3b & zh & [0.89, 0.48] & [1.19, 1.17] & [0.58, 0.45] \\ bloom-3b & ja & [0.82, 0.47] & [1.28, 1.27] & [0.75, 0.57] \\ \hline Llama-2-7b-hf & en & [0.91, 0.33] & [1.28, 1.38] & [0.96, 0.35] \\ Llama-2-7b-hf & de & [1.06, 0.53] & [1.4, 1.63] & [0.75, 0.43] \\ Llama-2-7b-hf & fur & [1.37, 0.56] & [1.2, 1.39] & [1.03, 0.45] \\ Llama-2-7b-hf & es & [1.76, 0.58] & [1.3, 1.69] & [1.17, 0.46] \\ Llama-2-7b-hf & zh & [0.48, 0.7] & [1.2, 1.33] & [0.51, 0.57] \\ Llama-2-7b-hf & ja & [0.6, 0.53] & [1.17, 1.27] & [0.56, 0.42] \\ \hline Llama-2-13b-hf & en & [0.65, 0.36] & [1.25, 1.32] & [0.83, 0.39] \\ Llama-2-13b-hf & de & [0.94, 0.53] & [1.26, 1.41] & [0.57, 0.39] \\ Llama-2-13b-hf & für & [0.98, 0.51] & [1.22, 1.51] & [0.68, 0.39] \\ Llama-2-13b-hf & es & [1.68, 0.61] & [1.29, 1.7] & [1.11, 0.46] \\ Llama-2-13b-hf & zh & [0.48, 0.52] & [1.12, 1.2] & [0.48, 0.48] \\ Llama-2-13b-hf & ja & [0.63, 0.38] & [1.11, 1.21] & [0.55, 0.33] \\ \hline \end{tabular}
\end{table}
Table 7: Estimation of Beta distribution parameters from the histogram of neurons across layers in top-1000, middle-1000, and bottom-1000 groups, respectively.





\begin{table}
\begin{tabular}{l r r r r r r} \hline  & de & en & es & fr & ja & zh \\ \hline de & 2000 & 10 & 6 & 1 & 14 & 5 \\ en & 10 & 2000 & 13 & 10 & 8 & 11 \\ es & 6 & 13 & 2000 & 28 & 12 & 16 \\ fr & 1 & 10 & 28 & 2000 & 7 & 12 \\ ja & 14 & 8 & 12 & 7 & 2000 & 30 \\ zh & 5 & 11 & 16 & 12 & 30 & 2000 \\ \hline \end{tabular} (c) sglm-2.9B

\begin{tabular}{r c r r r r} \hline  & de & en & es & fr & ja & zh \\ \hline de & 2000 & 10 & 6 & 1 & 14 & 5 \\ en & 10 & 2000 & 13 & 10 & 8 & 11 \\ es & 6 & 13 & 2000 & 28 & 12 & 16 \\ fr & 1 & 10 & 28 & 2000 & 7 & 12 \\ ja & 14 & 8 & 12 & 7 & 2000 & 30 \\ zh & 5 & 11 & 16 & 12 & 30 & 2000 \\ \hline \end{tabular} (d) bloom-560m

\begin{tabular}{r c r r r r} \hline  & de & en & es & fr & ja & zh \\ \hline de & 2000 & 12 & 19 & 20 & 12 & 61 \\ en & 12 & 2000 & 76 & 91 & 61 & 87 \\ es & 19 & 76 & 2000 & 168 & 70 & 47 \\ fír & 20 & 91 & 168 & 2000 & 42 & 56 \\ ja & 12 & 61 & 70 & 42 & 2000 & 5 \\ zh & 61 & 87 & 47 & 56 & 5 & 2000 \\ \hline \end{tabular} (f) bloom-3b

\begin{tabular}{r c r r r r} \hline  & de & en & es & fr & ja & zh \\ \hline de & 2000 & 8 & 12 & 12 & 15 & 43 \\ en & 8 & 2000 & 64 & 45 & 34 & 46 \\ es & 12 & 64 & 2000 & 98 & 14 & 26 \\ fr & 12 & 45 & 98 & 2000 & 18 & 25 \\ ja & 15 & 34 & 14 & 18 & 2000 & 11 \\ zh & 43 & 46 & 26 & 25 & 11 & 2000 \\ \hline \end{tabular} (h) Llama-2-13b

\begin{tabular}{r c r r r r} \hline  & de & en & es & fr & ja & zh \\ \hline de & 2000 & 14 & 13 & 11 & 5 & 23 \\ en & 14 & 2000 & 10 & 10 & 18 & 9 \\ es & 13 & 10 & 2000 & 23 & 16 & 1 \\ für & 11 & 10 & 23 & 2000 & 9 & 34 \\ ja & 5 & 18 & 16 & 9 & 2000 & 80 \\ zh & 23 & 9 & 1 & 34 & 80 & 2000 \\ \hline \end{tabular} (h) Llama-2-13b

\begin{tabular}{r c r r r r} \hline  & de & en & es & fr & ja & zh \\ \hline de & 2000 & 14 & 13 & 11 & 5 & 23 \\ en & 14 & 2000 & 10 & 10 & 18 & 9 \\ es & 13 & 10 & 2000 & 23 & 16 & 1 \\ fur & 11 & 10 & 23 & 2000 & 9 & 34 \\ ja & 5 & 18 & 16 & 9 & 2000 & 80 \\ zh & 23 & 9 & 1 & 34 & 80 & 2000 \\ \hline \end{tabular} (h) Llama-2-13b

\begin{tabular}{r c r r r r} \hline  & de & en & es & fr & ja & zh \\ \hline de & 2000 & 14 & 13 & 11 & 5 & 23 \\ en & 14 & 2000 & 10 & 10 & 18 & 9 \\ es & 13 & 10 & 2000 & 23 & 16 & 1 \\ fr & 11 & 10 & 23 & 2000 & 9 & 34 \\ ja & 5 & 18 & 16 & 9 & 2000 & 80 \\ zh & 23 & 9 & 1 & 34 & 80 & 2000 \\ \hline \end{tabular} (h) Llama-2-13b

\begin{tabular}{r c r r r r} \hline  & de & en & es & fr & ja & zh \\ \hline de & 2000 & 14 & 13 & 11 & 5 & 23 \\ en & 14 & 2000 & 10 & 10 & 18 & 9 \\ es & 13 & 10 & 2000 & 23 & 16 & 1 \\ fr & 11 & 10 & 23 & 2000 & 9 & 34 \\ ja & 5 & 18 & 16 & 9 & 2000 & 80 \\ zh & 23 & 9 & 1 & 34 & 80 & 2000 \\ \hline \end{tabular} (h) Llama-2-13b

\begin{tabular}{r c r r r r} \hline  & de & en & es & fr & ja & zh \\ \hline de & 2000 & 14 & 13 & 11 & 5 & 23 \\ en & 14 & 2000 & 10 & 10 & 18 & 9 \\ es & 13 & 10 & 2000 & 23 & 16 & 1 \\ fr & 11 & 10 & 23 & 2000 & 9 & 34 \\ ja & 5 & 18 & 16 & 9 & 2000 & 80 \\ zh & 23 & 9 & 1 & 34 & 80 & 2000 \\ \hline \end{tabular} (h) Llama-2-13b

\begin{tabular}{r c r r r r} \hline  & de & en & es & fr & ja & zh \\ \hline de & 2000 & 14 & 13 & 11 & 5 & 23 \\ en & 14 & 2000 & 10 & 10 & 18 & 9 \\ es & 13 & 10 & 2000 & 23 & 16 & 1 \\ fr & 11 & 10 & 23 & 2000 & 9 & 34 \\ ja & 5 & 18 & 16 & 9 & 2000 & 80 \\ zh & 23 & 9 & 1 & 34 & 80 & 2000 \\ \hline \end{tabular} (h) Llama-2-13b

\begin{tabular}{r c r r r r} \hline  & de & en & es & fr & ja & zh \\ \hline de & 2000 & 14 & 13 & 11 & 5 & 23 \\ en & 14 & 2000 & 10 & 10 & 18 & 9 \\ es & 13 & 10 & 2000 & 23 & 16 & 1 \\ fr & 11 & 10 & 23 & 2000 & 9 & 34 \\ ja & 5 & 18 & 16 & 9 & 2000 & 80 \\ zh & 23 & 9 & 1 & 34 & 80 & 2000 \\ \hline \end{tabular} (h) Llama-2-13b


Figure 16: Activation value difference of top-1000 neurons between target language(on) and non-target languages(off). x-axis: activation value of neurons. y-axis: frequency.

 

Figure 17: Activation value difference of top-1000 neurons between target language(on) and non-target languages(off). x-axis: activation value of neurons. y-axis: frequency.

 

Figure 18: Activation value difference of top-1000 neurons between target language(on) and non-target languages(off). x-axis: activation value of neurons. y-axis: frequency.

 

Figure 19: Activation value difference of bottom-1000 neurons between target language(on) and non-target languages(off). x-axis: activation value of neurons. y-axis: frequency.

 

Figure 20: Activation value difference of bottom-1000 neurons between target language(on) and non-target languages(off). x-axis: activation value of neurons. y-axis: frequency.

 

Figure 21: Activation value difference of bottom-1000 neurons between target language(on) and non-target languages(off). x-axis: activation value of neurons. y-axis: frequency.

 



\begin{table}
\begin{tabular}{l l r r r r} \hline \hline  & \multicolumn{3}{c}{before} & \multicolumn{2}{c}{after} \\ \cline{4-7}  & & Top & Bottom & Both \\ \hline xglm & en & 40.0 & 62.0 & 77.0 & **89.0** \\ (564M) & de & 0.0 & 89.0 & 31.0 & **95.0** \\  & fr & 0.0 & 86.0 & 7.0 & **90.0** \\  & es & 2.0 & 71.0 & 5.0 & **78.0** \\  & zh & 7.0 & **82.0** & 50.0 & 79.0 \\  & ja & 7.0 & 92.0 & 61.0 & **99.0** \\  & - & 9.3 & 80.3 & 38.5 & **88.3** \\ \hline xglm & en & 36.0 & 23.0 & **43.0** & 32.0 \\ (1.7B) & de & 3.0 & 84.0 & 10.0 & **91.0** \\  & fr & 1.0 & 54.0 & 5.0 & **70.0** \\  & es & 3.0 & 53.0 & 9.0 & **69.0** \\  & zh & 3.0 & 59.0 & 4.0 & **65.0** \\  & ja & 9.0 & 83.0 & 17.0 & **87.0** \\  & - & 9.2 & 59.3 & 14.7 & **69.0** \\ \hline xglm & en & 31.0 & 28.0 & **48.0** & 42.0 \\ (2.9B) & de & 2.0 & **92.0** & 1.0 & 88.0 \\  & fr & 1.0 & 60.0 & 3.0 & **61.0** \\  & es & 1.0 & 67.0 & 5.0 & **73.0** \\  & zh & 5.0 & 74.0 & 6.0 & **85.0** \\  & ja & 11.0 & **81.0** & 3.0 & 80.0 \\  & - & 8.5 & 67.0 & 11.0 & **71.5** \\ \hline bloom & en & 50.0 & 69.0 & 80.0 & **85.0** \\ (560m) & de & 0.0 & 34.0 & 0.0 & **72.0** \\  & fr & 13.0 & 37.0 & 85.0 & **93.0** \\  & es & 9.0 & 72.0 & 69.0 & **97.0** \\  & zh & 0.0 & 24.0 & 61.0 & **90.0** \\  & ja & 0.0 & 60.0 & 0.0 & **74.0** \\  & - & 12.0 & 49.3 & 49.2 & **85.2** \\ \hline \end{tabular} 
\begin{tabular}{l c r r r r} \hline \hline  & \multicolumn{3}{c}{before} & \multicolumn{2}{c}{after} \\ \cline{4-7}  & & & Top & Bottom & Both \\ \hline bloom & en & 37.0 & 78.0 & 67.0 & **88.0** \\ (1b7) & de & 0.0 & 60.0 & 0.0 & **86.0** \\  & fr & 13.0 & 80.0 & 72.0 & **98.0** \\  & es & 18.0 & 44.0 & 94.0 & **97.0** \\  & zh & 6.0 & 1.0 & 89.0 & **90.0** \\  & ja & 0.0 & 67.0 & 35.0 & **97.0** \\  & - & 12.3 & 55.0 & 59.5 & **92.7** \\ \hline bloom & en & 32.0 & 41.0 & 87.0 & **96.0** \\ (3b) & de & 0.0 & 44.0 & 2.0 & **55.0** \\  & fr & 15.0 & 72.0 & 72.0 & **93.0** \\  & es & 19.0 & 60.0 & 94.0 & **95.0** \\  & zh & 7.0 & 24.0 & **91.0** & 90.0 \\  & ja & 0.0 & 85.0 & 1.0 & **87.0** \\  & - & 12.2 & 54.3 & 57.8 & **86.0** \\ \hline Llama-2 & en & 83.0 & 82.0 & **89.0** \\ (7b) & de & 0.0 & 2.0 & 6.0 & **23.0** \\  & fr & 2.0 & 1.0 & **8.0** \\  & es & 1.0 & 4.0 & 4.0 & **35.0** \\  & zh & 0.0 & 2.0 & 4.0 & **50.0** \\  & ja & 1.0 & 1.0 & **12.0** & 10.0 \\  & - & 14.5 & 15.3 & 20.5 & **35.7** \\ \hline Llama-2 & en & 64.0 & 90.0 & 81.0 & **94.0** \\  & de & 3.0 & 2.0 & 3.0 & **16.0** \\  & fr & 0.0 & **9.0** & 1.0 & 8.0 \\  & es & 1.0 & 1.0 & **5.0** & **5.0** \\  & zh & 3.0 & **10.0** & 6.0 & 5.0 \\  & ja & 2.0 & **6.0** & 1.0 & 4.0 \\  & - & 12.2 & 19.7 & 16.2 & **22.0** \\ \hline \hline \end{tabular}
\end{table}
Table 9: Probability of language occurrence of generated texts before and after intervention. Top: intervention to only top-1000 neurons. Bottom: intervention to only bottom-1000 neurons. Both: intervention to both top- and bottom-1000 neurons. The metric is accuracy measured by the FastText language identifier.



Figure 22: Ablation study of changing the number of neurons to intervene. x-axis: \(\log_{10}(k)\)



Figure 23: Ablation study of changing the number of neurons to intervene. x-axis: \(\log_{10}(k)\)



Figure 24: Ablation study of changing the number of neurons to intervene. x-axis: \(\log_{10}(k)\)



* [38] M. C. Collins, R. K. Ellis, and J. A. Neubert, "The anti-jet clustering algorithm for the determination of jet substructure in hadronic collisions at HERA", _Phys. Rev. D_**56** (1997) 2405-2412, doi:10.1103/PhysRevD.56.2405.
* [39] G. Altarelli et al., "A study on the structure of the diphoton decay channel in proton-proton collisions at HERA", _JHEP_**04** (2004) 043, doi:10.1007/JHEP04(2004)043, arXiv:hep-ph/0409156.
* [40] S. Alioli, P. Nason, C. Oleari, and E. Re, "NLO QCD corrections to parton shower simulations: The POWHEG method", _Eur. Phys. J. C_**71** (2005) 159, doi:10.1140/epjc/s10052-005-3988-x, arXiv:nucl-ex/0406002.
* [41] T. Sjostrand, S. Mrenna, and P. Skands, "PYTHIA 6.4 physics and manual", _Comput. Phys. Commun._**191** (2006) 159, doi:10.1016/j.cpc.2006.02.026, arXiv:hep-ph/0603175.
* [42] GEANT4 Collaboration, "GEANT4--a simulation toolkit", _Nucl. Instrum. Meth. A_**506** (2003) 250, doi:10.1016/S0168-9002(03)01368-8.
* [43] CMS Collaboration, "Event generator tunes obtained from underlying event and multiparton scattering measurements", _Eur. Phys. J. C_**76** (2016) 155, doi:10.1140/epjc/s10052-016-3988-z, arXiv:1512.0021.
* [44] CMS Collaboration, "Measurement of differential cross sections with the ATLAS detector in pp collisions at \(\sqrt{s}=7\TeV\)", _JINST_**10** (2017) P06004, doi:10.1088/1748-0221/10/06/P06004, arXiv:1609.02701.

[MISSING_PAGE_POST]

 

* [12] M. C. Bader, R. J. Brouwer, and A. K. Sridharan, "The role of the design of a new 

* **xglm-564M**: en

Some of the issues that we are gonna have here are: the NSA is investigating whether the program is leaking in to the public and the government is trying to stop it as of late as it is possible. In the meantime the NSA is going to run the Panama Papers to find out what the UAE(UAE is the official of the U.E.E.)

## Appendix A

Figure 27: Model-generated text examples from unconditional text generation settings with the top-1000 and bottom-1000 language-selective neurons intervention.

 

* [16] M. C. Bader, R. D. Kiefer, and A. J. P. Singh, "The automated automated computation of the event generator for events with large-scale data," in _Proceedings of the International Conference on Computer Analysis and Machine Intelligence_, p. 209-238, 2007.
* [17] G. F. Hinton, S. H. Friedman, and I. Goodfellow, "Learning to improve neural networks: a survey," _Advances in Neural Information Processing Systems_, vol. 45, no. 1, pp. 1-10, 2002.
* [18] L. Breiman, E. Warden, and T. Darrell, "A new approach to boosting deep neural nets," _IEEE transactions on pattern analysis and machine intelligence_, vol. 12, no. 1, pp. 1-10, 2002.
* [19] N. D. Johnson, "Deep learning for image recognition," _IEEE Transactions on Pattern Analysis and Machine Intelligence_, vol. 12, no. 1, pp. 1-10, 2002.
* [20] Y. Bengio, "Generative adversarial nets for classification," _Journal of Machine Learning Research_, vol. 12, no. 1, pp. 1-10, 2002.
* [21] Z. Chen, X. Li, and Q. Sun, "Deep learning for image recognition," _arXiv preprint arXiv:1506.03017_, 2015.

[MISSING_PAGE_POST]

 



[MISSING_PAGE_POST]

 

* [26] M. C. Bender, A. K. Szyminski, and R. J. Wieschowski, "A new approach to the problem of finding 



###### Abstract

The proposed method for detecting the number of events in a single event is based on the detection of events from a single event. The algorithm uses an event that passes through a single event to be detected and returns a false alarm. This algorithm combines two types of events, one with a probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability equal to the probability to the probability equal to the probability equal to the probability equal to the probability equal to the probability to the probability equal to the probability equal to the probability equal to the probability to the probability equal to the probability equal to the probability to the probability equal to the probability equal to the probability to the probability equal to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability to the probability equal to the probability equal to the probability to the probability equal to the probability to the probability equal equal to the probability to the probability equal equal to the probability to the probability equal equal to the probability to the probability equal equal to the probability to the probability equal equal to the probability to the probability equal equal to the probability to the probability equal equal to the probability to the probability equal equal to the probability to the probability equal equal to the probability to the probability equal equal to the probability equal equal to the probability to the probability equal equal to the

* [26] A. J. Burt, M. C. R. D'Auria, and S. F. G. Wilson, "The \(\chi^{3}\)-factor 

* [20] CMS Collaboration, "Search for the production of a pair of tau leptons in proton-proton collisions at \(\sqrt{s}=7.96\) TeV with the ATLAS detector", _Phys. Rev. D_ **83** (2011) 054013, doi:10.1103/PhysRevD.83.054013
* [21] CMS Collaboration, "Search for the production of a pair of tau leptons in proton-proton collisions at \(\sqrt{s}=7.96\) TeV using the ATLAS detector", _JHEP_ **09** (2012) 013, doi:10.1007/JHEP09(2012)013, arXiv:1207.4081.

[MISSING_PAGE_POST]

 

* [26] M. Cacciari, G. P. Salambi, and A. Zisserman, "A new approach to the determination of the neutrino mass from a single-particle model with an improved precision measurement", in _Proceedings of the Workshop on Neutrino Physics_, p. 103, pp. 579-584, 2006.
* [27] J. L. Hewett, R. S. Thorne, and D. Wacker, "Neutrino oscillations at high energies: The role of the neutrino oscillation problem", _Phys. Rev. Lett._**83**, 101802 (2001) doi:10.1103/PhysRevLett.83.101802, arXiv:hep-ph/0109180.
* [28] T. Honda et al., "The neutrino oscillation puzzle", _Nucl. Phys. B_**635**, 1 (2001) doi:10.1016/j.nuclphysb.2001.02.002, arXiv:hep-ex/0010030.
* [29] T. Honda et al., "Neutrino oscillations at low energy colliders", _JHEP_**06**, 010 (2006) doi:10.1007/JHEP06(2006)010, arXiv:hep-ph/0609185.

[MISSING_PAGE_POST]

 

* [34] M. Cacciari, G. P. Salam, and A. Zamponi, "The automated computation of jet substructure in the CMS detector at LHC", _JHEP_**07** (2015) 069, doi:10.1088/1126-6708/2015/07/069, arXiv:1502.0301.
* [35] J. Alwall et al., "MadGraph 5: going beyond", _Nucl. Instrum. Meth. A_**506** (2003) 250, doi:10.1016/S0168-9002(03)01368-8.
* [36] S. Alioli, F. Maltoni, and R. Nason, "A new method for combining NLO QCD with shower Monte Carlo algorithms", _Eur. Phys. J. C_**71** no. 2, 155, doi:10.1140/epjc/s10052-004-1559-z, arXiv:hep-ph/0409185.
* [37] T. Sjostrand, S. Mrenna, and P. Skands, "PYTHIA 6.4 physics and manual", _Comput. Phys. Commun._**191** (2006) 159, doi:10.1016/j.cpc.2006.01.026, arXiv:hep-ph/0603175.
* [38] K. Melnikov et al., "Parton distributions from underlying event data", _Phys. Rev. D_**82** (2010) 094027, doi:10.1103/PhysRevD.82.094027, arXiv:1007.2581.
* [39] L. Randall and E. Re, "Randall structure functions from warped extra dimensions", _Phys. Lett. B_**606** (2003) 219, doi:10.1016/j.physletb.2003.01.037, arXiv:hep-ph/0309185.

[MISSING_PAGE_POST]

 



\begin{table}
\begin{tabular}{l c r r r r} \hline model & language & Accuracy & BLEU \\ \hline  & & Before & After & Before & After \\ \hline xglm-564M & de & 0.0 & **15.0** & 0.0 & 0.0 \\ xglm-564M & ja & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-564M & fr & 0.0 & **3.0** & 0.0 & 0.0 \\ xglm-564M & zh & 0.0 & 2.0 & 0.0 & 0.0 \\ xglm-564M & - & 0.0 & **5.0** & 0.0 & 0.0 \\ \hline xglm-1.7B & de & 0.0 & **18.0** & 0.0 & **1.4** \\ xglm-1.7B & ja & 0.0 & **11.0** & 0.0 & 0.0 \\ xglm-1.7B & fr & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-1.7B & zh & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-1.7B & - & 0.0 & **7.3** & 0.0 & **0.3** \\ \hline xglm-2.9B & de & 0.0 & **3.0** & 0.0 & 0.0 \\ xglm-2.9B & ja & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-2.9B & fr & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-2.9B & zh & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-2.9B & - & 0.0 & **0.8** & 0.0 & 0.0 \\ \hline bloom-560m & de & 0.0 & **4.0** & 0.3 & **0.4** \\ bloom-560m & ja & 0.0 & 0.0 & 0.0 & 0.0 \\ bloom-560m & fr & 0.0 & 0.0 & 0.0 & 0.0 \\ bloom-560m & zh & 0.0 & 0.0 & 0.0 & 0.0 \\ bloom-560m & - & 0.0 & **1.0** & 0.1 & 0.1 \\ \hline bloom-1b7 & de & 0.0 & **35.0** & 1.0 & **1.8** \\ bloom-1b7 & ja & 0.0 & **8.0** & 0.1 & **0.2** \\ bloom-1b7 & fr & 0.0 & **2.0** & 1.0 & **1.5** \\ bloom-1b7 & zh & 0.0 & **3.0** & 0.2 & **0.3** \\ bloom-1b7 & - & 0.0 & **12.0** & 0.6 & **0.9** \\ \hline bloom-3b & de & 0.0 & **32.0** & 0.7 & **1.0** \\ bloom-3b & ja & 0.0 & **4.0** & 0.1 & 0.1 \\ bloom-3b & fr & 0.0 & **6.0** & 0.4 & **0.7** \\ bloom-3b & zh & 0.0 & **1.0** & 0.2 & 0.2 \\ bloom-3b & - & 0.0 & **10.8** & 0.3 & **0.5** \\ \hline Llama-2-7b-hf & de & 0.0 & **48.0** & 1.2 & **12.5** \\ Llama-2-7b-hf & ja & 1.0 & **57.0** & 0.2 & **4.5** \\ Llama-2-7b-hf & fr & 0.0 & **32.0** & 1.0 & **11.1** \\ Llama-2-7b-hf & zh & 3.0 & **82.0** & 0.6 & **7.8** \\ Llama-2-7b-hf & - & 1.0 & **54.8** & 0.8 & **9.0** \\ \hline Llama-2-13b-hf & de & 0.0 & **37.0** & 0.6 & **10.0** \\ Llama-2-13b-hf & ja & 4.0 & **75.0** & 0.7 & **6.1** \\ Llama-2-13b-hf & fr & 0.0 & **9.0** & 0.7 & **4.7** \\ Llama-2-13b-hf & zh & 40.0 & **96.0** & 5.8 & **9.6** \\ Llama-2-13b-hf & - & 11.0 & **54.3** & 1.9 & **7.6** \\ \hline \end{tabular}
\end{table}
Table 10: Results of conditional text generation for IWSLT2017.





\begin{table}
\begin{tabular}{l r c c c c} \hline model & language & Accuracy & \multicolumn{2}{c}{BLEU} \\ \hline  & & Before & After & Before & After \\ \hline xglm-564M & de & 0.0 & **17.0** & 0.0 & 0.0 \\ xglm-564M & fr & 0.0 & **1.0** & 0.0 & 0.0 \\ xglm-564M & zh & 0.0 & **2.0** & 0.0 & 0.0 \\ xglm-564M & - & 0.0 & **6.7** & 0.0 & 0.0 \\ \hline xglm-1.7B & de & 0.0 & **16.0** & 0.0 & **0.2** \\ xglm-1.7B & fr & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-1.7B & zh & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-1.7B & - & 0.0 & **5.3** & 0.0 & **0.1** \\ \hline xglm-2.9B & de & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-2.9B & fr & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-2.9B & zh & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-2.9B & - & 0.0 & 0.0 & 0.0 & 0.0 \\ \hline bloom-560m & de & 0.0 & **4.0** & **1.4** & 1.2 \\ bloom-560m & fr & 0.0 & 0.0 & 0.5 & **0.6** \\ bloom-560m & zh & 0.0 & 0.0 & 0.1 & 0.1 \\ bloom-560m & - & 0.0 & **1.3** & **0.7** & 0.6 \\ \hline bloom-1b7 & de & 0.0 & **37.0** & **2.9** & 1.7 \\ bloom-1b7 & fr & 0.0 & **9.0** & 1.7 & **2.7** \\ bloom-1b7 & zh & 0.0 & **34.0** & 0.5 & **0.6** \\ bloom-1b7 & - & 0.0 & **26.7** & 1.7 & 1.7 \\ \hline bloom-3b & de & 0.0 & **19.0** & **3.1** & 1.4 \\ bloom-3b & fr & 0.0 & **7.0** & 1.2 & **4.0** \\ bloom-3b & zh & 0.0 & **4.0** & 0.4 & **1.0** \\ bloom-3b & - & 0.0 & **10.0** & 1.5 & **2.1** \\ \hline Llama-2-7b-hf & de & 2.0 & **53.0** & 5.3 & **15.2** \\ Llama-2-7b-hf & fr & 0.0 & **36.0** & 2.1 & **13.2** \\ Llama-2-7b-hf & zh & 12.0 & **86.0** & 2.4 & **11.3** \\ Llama-2-7b-hf & - & 4.7 & **58.3** & 3.3 & **13.2** \\ \hline Llama-2-13b-hf & de & 4.0 & **32.0** & 3.3 & **9.7** \\ Llama-2-13b-hf & fr & 1.0 & **15.0** & 2.2 & **6.6** \\ Llama-2-13b-hf & zh & 57.0 & **99.0** & 13.5 & **18.9** \\ Llama-2-13b-hf & - & 20.7 & **48.7** & 6.3 & **11.7** \\ \hline \end{tabular}
\end{table}
Table 11: Results of conditional text generation for WMT.





\begin{table}
\begin{tabular}{l c r r r r} \hline model & language & Accuracy & BLEU \\ \hline  & & Before & After & Before & After \\ \hline xglm-564M & de & 0.0 & **38.0** & 0.0 & 0.0 \\ xglm-564M & es & 0.0 & **3.0** & 0.0 & 0.0 \\ xglm-564M & ja & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-564M & fr & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-564M & zh & 0.0 & **1.0** & 0.0 & 0.0 \\ xglm-564M & - & 0.0 & **8.4** & 0.0 & 0.0 \\ \hline xglm-1.7B & de & 0.0 & **21.0** & 0.0 & **1.3** \\ xglm-1.7B & es & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-1.7B & ja & 0.0 & **4.0** & 0.0 & 0.0 \\ xglm-1.7B & fr & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-1.7B & zh & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-1.7B & - & 0.0 & **5.0** & 0.0 & **0.3** \\ \hline xglm-2.9B & de & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-2.9B & es & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-2.9B & ja & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-2.9B & fr & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-2.9B & zh & 0.0 & 0.0 & 0.0 & 0.0 \\ xglm-2.9B & - & 0.0 & 0.0 & 0.0 & 0.0 \\ \hline bloom-560m & de & 0.0 & **6.0** & **0.4** & 0.3 \\ bloom-560m & es & 0.0 & **9.0** & 0.2 & **0.6** \\ bloom-560m & ja & 0.0 & **5.0** & 0.0 & 0.0 \\ bloom-560m & fr & 0.0 & 0.0 & 0.5 & **0.6** \\ bloom-560m & zh & 0.0 & 0.0 & 0.3 & 0.3 \\ bloom-560m & - & 0.0 & **4.0** & 0.3 & 0.3 \\ \hline bloom-1b7 & de & 0.0 & **56.0** & 1.3 & 1.3 \\ bloom-1b7 & es & 0.0 & **2.0** & 1.2 & 1.2 \\ bloom-1b7 & ja & 0.0 & **6.0** & **0.2** & 0.1 \\ bloom-1b7 & fr & 0.0 & **16.0** & 1.7 & **2.8** \\ bloom-1b7 & zh & 0.0 & **21.0** & **0.3** & 0.2 \\ bloom-1b7 & - & 0.0 & **20.2** & 0.9 & **1.1** \\ \hline bloom-3b & de & 0.0 & **31.0** & **1.4** & 0.8 \\ bloom-3b & es & 0.0 & **7.0** & 1.4 & **2.3** \\ bloom-3b & ja & 0.0 & **7.0** & 0.2 & 0.2 \\ bloom-3b & fr & 0.0 & **1.0** & 1.8 & **1.8** \\ bloom-3b & zh & 1.0 & **2.0** & 0.4 & **0.4** \\ bloom-3b & - & 0.2 & **9.6** & 1.0 & **1.1** \\ \hline Llama-2-7b-hf & de & 0.0 & **66.0** & 2.6 & **17.7** \\ Llama-2-7b-hf & es & 4.0 & **77.0** & 3.3 & **16.6** \\ Llama-2-7b-hf & ja & 0.0 & **58.0** & 0.3 & **10.4** \\ Llama-2-7b-hf & fr & 1.0 & **58.0** & 4.1 & **21.5** \\ Llama-2-7b-hf & hj & 1.0 & **76.0** & 1.0 & **11.5** \\ Llama-2-7b-hf & - & 1.2 & **67.0** & 2.3 & **15.5** \\ \hline Llama-2-13b-hf & de & 0.0 & **22.0** & 1.5 & **8.8** \\ Llama-2-13b-hf & es & 2.0 & **14.0** & 1.8 & **4.3** \\ Llama-2-13b-hf & ja & 7.0 & **54.0** & 2.4 & **11.0** \\ Llama-2-13b-hf & fr & 0.0 & **23.0** & 1.6 & **10.5** \\ Llama-2-13b-hf & zh & 20.0 & **93.0** & 4.4 & **19.1** \\ Llama-2-13b-hf & - & 5.8 & **41.2** & 2.3 & **10.8** \\ \hline \end{tabular}
\end{table}
Table 12: Results of conditional text generation for FLORES200.





#### 9.2.1 The \(\pi^{+}\)-jet

The \(\pi^{+}\)-jet is a jet, which has the same energy as the \(\pi^{0}\) and \(\pi^{+}\) jets. This jet is not reconstructed in any event but also in its transverse direction. In this case, the jet must be reconstructed in an event with at least one jet. The jet must have a higher momentum than the jet, or it must have a higher momentum than the jet. If the jet passes through the jet, then the jet must pass through the jet to pass through the jet. If the jet passes through the jet, then the jet must pass through the jet, and the jet must pass through the jet passes through the jet. If the jet passes through the jet



#### xglm-564M Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English. Translate an English sentence into a target language. English: The Babylonians built each of their gods a primary temple that was considered the home of the god. Target Language: English

bloom-lb7 Translate an English sentence into a target language. In English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Translate an English sentence into a target language. In English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. In Target Language: The traffic stream is a sequence of events, each of which is a point in time. The traffic stream is a sequence of events, each of which is a point in time. The traffic stream is a sequence of events, each of which is a point in time. The traffic stream is a sequence of events, each of which is a point in time. The traffic stream is a sequence of events, each of which is a point in time. The traffic stream is a sequence of events, each of which is a point in time. The traffic stream is a sequence of events, each of which is a point in time. The traffic stream is a sequence of events, each of which is a point in time. The traffic stream is a sequence of events, each of which is a point in time. The traffic

Translate an English sentence into a target language. In English: The Babylonians built each of their gods a primary temple that was considered the home of the god. Un Target Language: The Babylonian primary temple was the home of the god.

Llama-2-7b-hf Translate an English sentence into a target language. In English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awaking during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Un Target Language: Sleep interruption is the process of purposefully aw

xglm-564M:de Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and failing asleep a short time later (10-60 minutes). Target Language: Deutsch Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: Englisch. Translate an English sentence into a target language. English: The Babylonians built each of their gods a primary temple that was considered the home of the god. Target Language: Deutsch

xglm-564M:fr Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English. Translate an English sentence into a target language. English: The Babylonians built each of their gods a primary temple that was considered the home of the god. Target Language: English

xglm-564M:es Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English. Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English. Translate an English sentence into a target language. English: The Babylonians built each of their gods a primary temple that was considered the home of the god. Target Language: English.

xglm-564M:zh Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English.

xglm-564M:ja Translate an English sentence into a target language. English: Sleep interruption is the process of purposefully awakening during your normal sleep period and falling asleep a short time later (10-60 minutes). Target Language: English Translate an English sentence into a target language. English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. Target Language: English Translate an English sentence into a target language. English: The Babyylonians built each of their gods a primary temple that was considered the home of the god. Target Language: English 

bloom-1b7: de Translate an English sentence into a target language.n English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically.in Target Language: Die 

Llama-2-7b-hf:de Translate an English sentence into a target language. In English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically.n Target Language: Fortun undoubtedly, die Fahrer verhalten sich innerhalbe einer ziemlich konstanten und damit erkennbaren Grenzen; sommit, die Verkehrstrome tenderenzu einer ziemlich konstanten und damit erkennbaren Struktur und konnen mathematisch abgebildet werden.

Translate an English sentence into a target language. In English: The Babylonians built each of their gods a primary temple that was considered the home of the god. In Target Language: Die Babylonier haben jede ihrer Gotter ein Haupttempel erbaut, das als Heim der Gotter angesehen wurde.

Llama-2-7b-hf:fr Translate an English sentence into a target language. In English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically.In Target Language: Fortuneement les conducteurs tendent a agir dans une zone relativement constante; ainsi les flux de trafic tendent a avoir une certaine consistance et peuvent ê reprsentes mathematiciquement.

Translate an English sentence into a target language. In English: The Babylonians built each of their gods a primary temple that was considered the home of the god. In Target Language: Les Babyloniens ont construit chacun de leurs dieux une premiere maison qui etait consideree comme la maison du diet.

Llama-2-7b-hf:es Translate an English sentence into a target language. In English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. In Target Language: Por suerte, los conductores tiende a comportarse dentro de un rango raznobable; por lo tanto, los fujos de trafico tiende a tener alguna consistencia razonable y pueden ser approximadamente representados matematicamente.

Translate an English sentence into a target language. In English: The Babylonians built each of their gods a primary temple that was considered the home of the god. In Target Language: Los babilonios construcyeron cada unco des sus dioses un templo primario que se considerabae el hogar del dos.

Limaa-2-7b-hf:zh Translate an English sentence into a target language. In English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. In Target Language: [].

Translate an English sentence into a target language. In English: The Babylonians built each of their gods a primary temple that was considered the home of the god. In Target Language: [].

Llama-2-7b-hf:ja Translate an English sentence into a target language. In English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. In Target Language: [].

Translate an English sentence into a target language. In English: The Babylonians built each of their gods a primary temple that was considered the home of the god. In Target Language: [].

Llama-2-7b-hf:ja Translate an English sentence into a target language. In English: Fortunately, drivers tend to behave within a reasonably consistent range; thus, traffic streams tend to have some reasonable consistency and can be roughly represented mathematically. In Target Language: [].

Translate an English sentence into a target language. In English: The Babylonians built each of their gods a primary temple that was considered the home of the god. In Target Language: [].

Figure 40: Model-generated text examples from conditional text generation settings with top-1000 and bottom-1000 language-specific neurons intervention.