The Same but Different: Structural Similarities and Differences in Multilingual Language Modeling
Authors: Ruochen Zhang
Qinan Yu
Matianyu Zang
Carsten Eickhoff
Ellie Pavlick
Venue: ICLR 2025 Poster (2025)
Keywords: multilinguality
interpretability
Primary area: interpretability and explainable AI

Abstract: We employ new tools from mechanistic interpretability to ask whether the internal structure of large language models (LLMs) shows correspondence to the linguistic structures which underlie the languages on which they are trained. In particular, we ask (1) when two languages employ the same morphosyntactic processes, do LLMs handle them using shared internal circuitry? and (2) when two languages require different morphosyntactic processes, do LLMs handle them using different internal circuitry? In a focused case study on English and Chinese multilingual and monolingual models, we analyze the internal circuitry involved in two tasks. We find evidence that models employ the same circuit to handle the same syntactic process independently of the language in which it occurs, and that this is the case even for monolingual models trained completely independently. Moreover, we show that multilingual models employ language-specific components (attention heads and feed-forward networks) when needed to handle linguistic processes (e.g., morphological marking) that only exist in some languages. Together, our results are revealing about how LLMs trade off between exploiting common structures and preserving linguistic differences when tasked with modeling multiple languages simultaneously, opening the door for future work in this direction.

Published as a conference paper at ICLR 2025
THE SAME BUT DIFFERENT: STRUCTURAL SIMILAR- ITIES AND DIFFERENCES IN MULTILINGUAL LAN- GUAGE MODELING
Ruochen Zhang"
Qinan Yuz*t
Matianyu Zang'
Carsten Eickhoff
Ellie Pavlick!
Brown University Stannoord University
Universityy of Tübingen
ABSTRACT
We employ new tools from mechanistic interpretability to ask whether the internal structure of large language models (LLMs) shows correspondence to the linguistic structures which underlie the languages which they are trained. In particular, we ask (1) when two languages employ the same morphosyntactic processes, do LLMs handle them using shared internal circuitry? and (2) when two languages require different morphosyntactic processes, do LLMs handle them using differ- ent internal circuitry? In focused case study on English and Chinese multilingual and monolingual models, we analyze the internal circuitry involved in two tasks. We find evidence that models employ the same circuit to handle the same syntac- tic process independently of the language in which it occurs, and that this is the case even for monolingual models trained completely independently. Moreover, we show that multilingual models employ language-specific components (atten- tion heads and feed-forward networks) when needed to handle linguistic processes (e.g., morphological marking) that only exist in some languages. Together, our re- sults are revealing about how LLMs trade off between exploiting common struc- tures and preserving linguistic differences when tasked with modeling multiple languages simultaneously. opening the door for future work in this direction.
INTRODUCTION
As large language models (LLMs) have become the undisputed state of the art for building English language technology. there is decided interest in replicating their success across the full range of human languages. However, very little is known about the internal structure of LLMs, and whether such structure is conducive to acquiring broad multilingual capabilities. In fact, recent research has produced seemingly contradictory findings, such as evidence that multilingual models adopt language-specific representations (Tang al. 2024 Choenni al., (024),, while simultaneously showing good transfer across languages even in cases that would appear to have no superficial simi- larities that can be exploited to aid such transfer (Pires et al. 2019). Given the importance of building technology for diverse languages, there is a need for more precise understanding of how LLMs represent structural similarities and differences across languages, and whether such representations accord with our intuitive understanding of how languages work.
In this paper, we present focused case study on the relationship between English and Chinese processing in LLMs. We employ tools from the growing subfield of mechanistic interpretability in order to ask whether the internal structure LLMs show correspondence to the linguistic struc- tures which underlie the languages on which they are trained. We focus only the most minimal criteria of correspondence. In particular, we ask (1) when these two languages employ the same morphosyntactic processes, do LLMs handle them using shared internal circuitry? and (2) when these two languages require different morphosyntactic processes, do LLMS handle them using dif- ferent internal circuitry? While these questions seem simple, their answers non-obvious. LLMs readily employ overlapping circuitry for tasks that do not necessarily seem "the same" to humans
"Equal Contribution. Correspondence to ruochen zhang@brown.ed.
Work done Brown University.

---

Published as a conference paper at ICLR 2025
(Merullo et al. 2024). and at the same time, neural networks frequently differentiate concepts due to surface form variation (Olah et al. 2020), even when humans would easily identify them as part of the same abstract category.
Using English and Chinese multilingual and monolingual models, we analyze the internal circuitry involved in two tasks, one focusing on indirect object identification (IOI) which is virtually identical between the languages, and one which involves generating paste tense verbs that require morpho- logical marking in English but not in Chinese. Our contributions are follows:
We show that multilingual model uses a single circuit to handle the same syntactic process independently of the language in which it occurs ($3.4).
We show that even monolingual models trained independently on English and Chinese each adopt nearly the same circuit for this task (335). suggesting surprising amount of consistency with how LLMs learn to handle this particular aspect of language modeling,
Finally, we show that, when faced with similar tasks that require language-specift morpho- logical processes, multilingual models still invoke a largely overlapping circuit, but employ language-specific components as needed. Specifically, in our task, we find that the model uses a circuit that consists primarily of attention heads perform most of the task, but em- ploys the feed-forware networks in English only to perform morphological marking that is necessary English but not in Chinese
Although the scope of our study is restricted to one language pair and few phenomena, our results provide informative insights into how LLMs trade off between exploiting common structures and preserving linguistic differences when tasked with modeling multiple languages simultaneously. Our experiments can lay the groundwork for future works which seek to improve cross- lingull transfer through more principled parameter updates (Wu et al. 2024),, as well as work which seeks to use LLMs in order to improve the study of linguistic and grammatical structure for its own sake
Lakrettz al. 2021 Misra & Kim 2024).
2
ANALYSIS METHODS
In this work, we are interested in analyzing how large language models (LLMs) trained in different languages differ in terms of the algorithms and mechanisms they invoke to handle various aspects of language processing. To do this, we employ a few recently developed analysis techniques, described below. These techniques are similar in spirit, but differ in certain details that matter for our analysis. For the most part, we find converging evidence for the paper's main claims across both techniques. When results differ in interesting ways, we comment our results sections.
2.1
PATH PATCHING
Path patching WWang al. 2023 Goldowsky- -Dil et al., 2023 Vig et al. 2020: Hanna et al. 2023 Tigges et al. 2023b) has become the most standard and new subtield of mechanistic interpretability. The goal of path patching is localize specific circuits within the weights in a trained neural network that play a causal role in model behavior. The setup requires a pair of contrastive inputs, one referred to as the clean input and the other as the corrupted input. Path patching caches the activations for both inputs and then replaces the values of individual heads on the clean input with the values those heads would have taken had they been run on the corrupted input. In this way, the method aims to find the specific important head which maximally explains the final logits. Working backward, i.e., through patching the important heads at each layer, path patching has been used to identify full circuits that carry out the task. On its own, path patching only identifies important heads. To gain insight into the specific functions of these heads, path patching is usually used with logit attribution (Nostalgebraist, 2020 Belrose et al. 2023 Dar al.) 2023 Yu et al.] 2023) which projects activations into the vocabulary space, as well as with bespoke analysis techniques invented by prior work to explain specific types of heads, such duplicate-token detection heads or copy heads (Wang et al. 2023).
technique within the still-
The advantage of path patching is primarily its wide adoption, which makes it easier to trust results, and enables us to compare with prior work in order to vet the results we are seeing (i.c., checking that we reproduce prior work when we expect do so). The primary downside is that the method
2

---

Published as a conference paper at ICLR 2025
requires careful design of the minimal pair templates. The circuit that is discovered is highly depen- dent on how these pairs are defined. Moreover, in some of our language pairs, a comparable design of such templates is not possible (see 4.. Thus, explore additional techniques.
2.2 INFORMATION FLOW ROUTES
Given the aforementioned drawbacks of path patching, we also employ the information flow routes method (Ferrando & Voita 2024, Tufanov al. 2024). At each timestep, this method computes the contribution of each head to the residual streams. These heads are then aggregated over the entire attention block to construct a graph of the information flow that show which residual stream at which layer is important to the current residual stream value update across different timesteps. In this method, different from path patching where we can see circuit components across layers, the importance can only be computed every two layers. Compared to path patching, information flow routes have the advantage of not requiring minimal pairs of inputs. The tradeoff that (I) the method is new and thus do not benefit from clear expectations about what it should yield in certain settings and (2) this method tends to discover more generic components and larger circuits. In our paper, thus both information flow routes, and path patching along with logit attribution in conjunction for analysis at different granularity levels. Unless otherwise stated, both methods produce results that consistent with the primary claims make in each experimental section.
TASKS WITH COMMON LINGUISTIC STRUCTURE ACROSS LANGUAGES
3.1 QUESTIONS
We first ask whether large language models (LLMs) will learn to use shared circuitry for different languages, specifically for aspects of language processing that reflect similar structures across lan- guages. That is: if two distinct languages use similar morphosyntactic structure but realize it using different lexical items, will an LLM treat these as the same (handling them with the same abstract internal circuit) or as different (handling them with distinct, sunguage-speciffg circuits)? We ask this question first in the case of multilingual models, and second in the case of monolingual models.
3.2 INDIRECT OBJECT IDENTIFICATION (IOI) TASK
We set up data following the original 1OI paper (Wang et al. 2023). ). 10I consists of sentences such as "Susan and Mary went to the bar. Susan gave a drink to [BLANK]". in which the model is expected to predict Mary. These names tokenized one single token in the models. To diversify the data, we use fifteen different templates for the actions. We also use ChatGPT to translate the templates and names into Chinese and have them manually inspected by native speakers.
Wang et al. (023) identifies a specific circuit within GPT2 (Radford et al. 2019) for performing the 10I task. At high level, the circuit reveals that the model runs the following algorithm: first, identifies that there is duplicated name (in the above example, Susan). Second, it inhibits attention this duplicated name. Third, it copies the remaining (non-duplicated) name. These steps are carried out by set of functionally specialized attention heads: Previous Token Heads, Duplicate Token Heads, Induction Heads, S-inhibition Heads, and Name Mover Heads. Duplicate Token Heads attend to the token position of Susan and identify this name token is mentioned twice in the sentence. S-inhibition Heads inhibit the Name Movers' attention to both occurrences of Susan. The Name Mover Heads output the remaining name (Mary). More detailed descriptions of these heads are given in prior work (Wang et al., 2023 Merullo et al.] 2024). For our purposes, what is important is whether the same heads are important and perform the same roles across languages.
3.3 MODELS
We use BLOOM-560M WWorsshop et al. 2022 as our multilingual model. We first evaluate whether BLOOM is able to complete the task by checking if it always prefers the correct name over the incorrect name. We use both accuracy and zero-rank rate to measure the model perfor- mances. Accuracy computes the percentage of times when the probability of predicting the correct
We refer the model BLOOM for simplicity helow.
3

---

Published as a conference paper at ICLR 2025
English I0I vs. Chinese IOI
English (OI vs. English Control
Color Tyoes
Engiknn Control Chrnesa 1II
Marker Types
Dup icata Tixan itead
Presicus Tiien iteed
18
18
intibel on Head
Indiattinn leand
17
Narne Nover lead
03
06
04
Attention Head Indicas
Attention Head Indicas
Figure 1: Attention heads activation frequency comparison between English 1OI, Chinese IOI and English control tasks. Left: Comparison between English IOI and Chinese IOI on BLOOM (with specific functional heads in different marker types). Right: Comparison between English IOI and English Tense on BLOOM. For pairs of heads with non-zero activation frequency, they are shaded based on their value difference. Darker gray means smaller differences. The left graph has more number of shaded head pairs compared to the right, indicating greater similarity between the activated heads.
name higher than the other name. The zero-rank rate measures the percentage of times where the correct name is the actual top answer. In the IOI task, BLOOM has accuracy of 100% and zero-rank accuracy of 96% in English and 95% accuracy and 93% zero-rank accuracy in Chinese. As multilingual model, BLOOM has high and comparable performance between Chinese and English.
For experiments with monolingual models, we select GPT2-small Radford al., 2019) our English-only model and CPM-distilled as our Chinese-only mode These two models have the same architecture and contain the same number of parameters. GPT2-small has a 99.5% accuracy and CPM has an 84.5% accuracy. The zero-rank rate for GPT2 is 97.5% meaning that the correct name has the highest probability most of the time. CPM has a zero-rank accuracy 57.5%. Both models are able to complete the task while the performance of GPT2 is comparably better.
3.4 RESULTS ON MULTILINGUAL MODEI.
We first validate whether the English and Chinese tasks share the same important heads. We use the information flow routes technique across 50 examples respectively in English and Chinese. We calculate the activation frequency for every head component. If one head is highly activated") then indicates this head is important for the task. In Figure [ we observe high overlap in the important heads used between languages. The heads that are shown to be important solve the task English are also important in Chinese. As comparison, we compare with the BLOOM's activated heads
We refer GPT2-small GPT2 and CPM for CPM-distilled helow for simplicity.
*Following Ferrando & Voita (2024), we set the threshold 7 0.03. We consider head is highly activated if its contribution value surpasses
4

---

Published as a conference paper at ICLR 2025
and
90
Previous Token Head 4.0 4050008270
#
t0
15:55
Duplicate Token Head 0.77
Induction Head 6122 76 230 712 R.5 a:1 613 9:11 10:1
store
#::00
S-inhibition Head 12.0 14. 194 222
Name Mover Head 157 18.8 18. 10 20,8 20 20:1 210.1100
gave 16
Negative Name Movar 20
snack
END
#67
Figure 2: Illustration of IOI circuits of English and Chinese on BLOOM-560M model. Blue-framed rectangles or texts are the components used in English IOI and orange-framed ones are for Chinese IOL. Heads marked in black are the shared functional heads between English and Chinese IOI. The English and Chinese IOI circuits are highly similar they use most of the same components for the same functionality to implement the algorithms.
from an unrelated task (specifically, the past tense task described in 8 4.2) as a control. If the circuit overlap is indeed indicative of the model exploiting shared algorithmic structure, would expect that the overlap between English and Chinese variants of the same linguistic task should be higher than the overlap between English for two tasks that invoke dissimilar linguistic (i.e., syntactic and semantic) structures. Indeed, compared with the high overlap between the IOI task in Chinese and English, significantly different heads are used in the control task. To measure the similarity between the activation patterns, we compute the Pearson Correlation Coefficient P (Freedman et al. (007) between the head activation frequency for the two task pairs. While p between English and Chinese IOI task is 0.72, between English IOI and English control task is 0.48. Anecdotally, we observe that many of the heads that are shared between English IOI and control are also shared with Chinese IOI. This suggests that these might be generically important heads which we expect to appear in wide range of tasks, although further work is needed to test this hypothesis quantitatively.
We continue to examine if these shared functionalities are the same as the ones defined in Wang jet al.] (2023). In Figure 2 we identified functionalities of the important components in Figure | Chinese and English in BLOOM. We first use path patching to identify the components and then use the specific metries mentioned in Wang et al. (2023) to validate their functionalities. Both these experiments return highly similar results The functionalities we identify align with Wang et al. (2023). This suggests that not only is the mechanism (i.e., the heads and their functionality) shared across English models (GPT2 and BLOOM), but moreover this same mechanism is shared nearly exactly between Chinese and English within BLOOM.
3.5 RESULTS ON MONOLINGUAL MODELS
While the cross- lonsul circuit overlap in a multilingual model is not obvious (i.e., it requires ab- stracting over patterns that manifest using disjoint sets of surface forms), also reasonable to assume that the convergence stems from the model's broader multilingual training data and objec- tive. We thus ask whether similar patterns arise even when models are trained independently in English and in Chinese. For this analysis, as mentioned, we use the English-only model GPT2- small and a distilled version of the Chinese-only model CPM-Generate (Zhang et al. 2020) for their exact same architecture.
We discover circuits with path patching which gives similar results and identifies the same compo- nents that are important for the circuits. In both GPT2 and CPM, we identified similar circuits as Wang et al. (2023) outlined in Figure This is surprising as it suggests that at a high level, the
Detailed analysis he found in B
5

---

Published as a conference paper at ICLR 2025
Mary
5161
and
81
John
1::0
Duplicate Token Head of 0:0 3.0
Induction Head 5. S.B.EE u
more
S- S-inhatiin Head 5
Name Mover Head 9.6 10
John
H:00
Suporession 11.2
gave
69
Negative Name M Hover 10.7 11.10
stack
Figure 3: English and Chinese IOI task circuits. Blue highlights components used and inputs for the English task in GPT2. Orange highlights are for CPM. The components with both blue and orange frames are shared in both the English and Chinese circuits. In each of these heads, blue texts indicate the heads in the English Circuit and orange in Chinese. The Negative Name Mover heads appear only in the English model. Copy Suppression Head appears only in the Chinese model. Despite trained on completely different data, models still implement largely similar algorithms to solve tasks different languages.
LLMs consistently converge to highly similar algorithms for solving this type of task, even when trained on very different languages. That said, we do observe some differences between the circuits. In GPT2, Wang et al. (2023) identifies Negative Name Mover Heads. These heads write negatively in the direction of the name tokens. Similarly, Negative Name Mover Heads are found in the cir- didn't observe any Negative Name Mover Heads in CPM. instead of
cuit in GPT2. However, observed the Copy Suppression Head. Such heads are identified by McDougall et al. (023b) in the Pythia model (Biderman et al. 2023). Instead of reading the information flow from the S-inhibition head, Copy Suppression Heads read directly from the Duplicate Token Heads and suppress the logit of the repeated token in the output. This divergence could provide good starting point for future work on predicting when and how specific circuitry emerges during training (see Discussion 5
4 TASKS WITH LANGUAGE-SPECIRE STRUCTURE
4.1 QUESTIONS
The above results show that L.MSs are capable of recognizing structural parallels across languages, and even that monolingual models, trained without explicit influence from other languages, con- verge on similar algorithms for similar structures. We next ask: what happens when languages exhibit dissimilar structures? In reality, languages exhibit immense morphological and syntactic variation, and there is never perfect one-to-one correspondence between the structural elements of different languages. A straightforward example of this phenomenon the fact that English includes morphological markers for tense (e.g., walk vs. walked) while Chinese does not. We thus ask, given such variation, do I.LMS adopt different circuitry for different languages, or rather do they continue to invoke shared circuitry but employ language-specific subroutines needed?
4.2 PAST TENSE TASK
We construct minimum pair templates in English: "Now I verb preser Yesterday I also verb and " "Yesterday verbpar- Now also verbpresem." " For Chinese, we construct the exact same template in # We - " and = - F #2 thverb To collect the verbs, we start with the English verb tense dataset from Big-bench (Srivastava et al., 2022) and filter the verbs to only retain those with regular inflection rules. Then all English verbs are translated, deduplicated and filtered with native speakers' manual inspection. Notice that in Chinese, there are no morphological markers between verb and verb Both of them refer to the stem
Chinese FRR
Soow.-Now I Yesterday, R-11 ih-also

---

Published as a conference paper at ICLR 2025
English V5. Chinese Tense
Color Types
English Tense
Now
Copy Head
st
21.3 214 2113
IIf
Similartty (Dareer More Similar
0.8
Yesterday
#t
Jayers
19.5
20-24
Copy
accept
Accept
socoptance
SE18
reject
accept
reject
Roceptance
Accept
.aceept
0.0
Past Tense Head
aF177
Attention Head Indices
Figure 4: (a) Head activation frequency comparison between English and Chinese past tense task on Qwen. (b) Illustration of the tense circuits in Qwen that contains three components focus on: copy heads, past tense heads and ferw-fo layers. (c) Top promoted tokens from copy head and past tense head. Words highlighted in either blue and orange expected model predictions. Gray tokens are the specific tokens that the head promotes. Here the copy head is 21.3 and the past tense head is 19.4 As shown in (a), the copy head has similarly high activation frequency in both languages whereas the past tense head is only frequently activated English.
form of the verb. By leveraging this distinction between the English and Chinese languages, able to investigate the mechanistic differences required when models process semantic equivalent tasks but with different morphosyntactic rules. Our analysis on this task focuses more specifically on multilingual models instead of monolingual models since they allow us to more directly observe the structral similarities between the circuitries.
4.3 MODELS
We use Owee2-0.5B-inssiie compared to BLOOM The final verb collection contains 62 English verb instances and their Chi- nese translation and themselves are both single tokens after being processed by the Qwen tokenizer. For the English task, Qwen reaches 96.77% accuracy for the normal input and 58 066% for the zero- rank rate. In Chinese, as there are no minimal pairs, we can only obtain a zero-rank rate 25.81 51.81% %.
Yang et 2024) for the task for better benchmark performance
4.4 RESULTS
In Figure compared to the active heads distribution on the [OI task, we notice that some early-layer heads have high activation frequencies that shared in both English and Chinese tasks. However, there are much fewer shared heads in the later layers of the model. Specifically for the Chinese task, observe that almost heads have high activation frequency in the later layers (Layer 19-20). We posit that some heads that are shown to be more active in these later layers are responsible for the additional inflection rule unique the English task our setting, which is unnecessary to the Chinese task.
We refer the model Qwen below for simplicity.
'Here we replace the model of interest from BLOOM to Qwen. Qwen is similarly a multilingual language model but with strong language focus both English and Chinese. It achieves comparable performances in English compared to BLOOM but significantly better performance in Chinese, which is necessary for extracting the circuits. See Table in the Appendix for the performance comparison.

---

Published as a conference paper at ICLR 2025
4.5 FUNCTION- SPEOIIIC HEADS
Through path patching. we observe that heads 19.4 4 and 19. _ exert the most significant influence on the final logits (Refer to Appendix for details). Examining the top promoted words from the most positive projections into the model vocabulary space ( we find that they generally favor past-tense words English (e.g., was, were, and other past-tense verbs unrelated to the target verb). Among the top-prom hop-rromoted tokens, we also notice Chinese verbs with the suffix "T" (e.g., jtf TTT, can be translated to "done".), .). where "T" is commonly used to indicate the completion of an action. Even when provided with Chinese input, these heads continue to strongly encode past-tense concepts. In Figure a), head 19.4 appears frequently activated for English but not for Chinese, suggesting that these heads are predominantly engaged in English processing rather than in Chinese.
To validate our findings, we ablate""] the heads 19. 4 and 19.5 5 and analyze their impact on the tasks. The rank of correct past tense verbs in English remains relatively unchanged. Whereas other non- relevant past tense verbs move backward slightly to make spaces for the corresponding present tense verbs, resulting in their ranks getting promoted by an average of 83.21 positions. In the Chinese task, the rank of the correct verb moves forward by an average of 4.58 positions. Notably, after ablating these past-tense heads and projecting the final-layer logits the vocabulary space, present- verbs emerge as the second most probable token (See Figure [2 for details). These results suggest that in English, past-tense heads actively suppress present-tense verbs to disambiguate between verb tenses. However, these heads do not play a similar role in the Chinese task.
In Ferrando Voita (2024), they mention that patching can only emphasize the heads that are important for the original tasks but not the contrastive baseline. In the past tense task, we locate past tense heads that are in charge of disambiguating past tense verbs from present tense ones via path patching. These heads are discovered due to the additional morphological markers between the original and contrastive sentence. However, how do models know to promote that specific verb (with without the marker) without confusing it with others?
To address this question, we compute the total score of each head projecting onto either past tense or present tense verb groups (see Appendix C.I). The primary heads identified are 21 21. 4, and 21.13. We observe that these heads not only emphasize different morphological variants of the target verb but also promote tokens that are orthographically similar or semantically related, such as synonyms and antonyms (see Figure 12 in Appendix). These heads are crucial for both English and Chinese tasks.
To assess their impact, we perform ablation studies on these heads and observe significant impacts on model performance. In English, the average rank of past tense verbs drops by 141.95, while present tense verbs experience a much larger average demotion of 221.39. This disparity suggests that past tense verbs are less affected, possibly because the identified past tense heads remain active, suppressing present tense verbs. As shown in Figure [2] the top- promoted words after ablation are still predominantly general past tense verbs. For Chinese, the model performance also declines, with verbs in general experiencing an average rank demotion of 201.82. These ablation results indicate that the copy heads are essential for both English and Chinese tasks, while the past tense heads are specifically activated in English, where they are crucial for task completion. However, qualitative inspection also suggests that the semantics of the verb is not entirely decoupled from the tense, highlighting need for further work in order produce more nuanced descriptions of the function of these heads in this circuit, and in general.
4.6 FEED-FORWARD NETWORKS
Previous work Merullo et al., 2023) discusses the role of the feed-forward networks (FFN) in re- trieving the correct transformation required for the past tense tasks. Specifically, these FFNs are found in the last few layers of the models where the ranked position of the present-tense verb and past-tense verb switch. Comparing the English and Chinese tasks, we assume that these FFN lay- ers play role in predicting the past-tense verb in English but remain unactivated for the Chinese task. When we ablate out the FFNs from layer 20-24, the accuracy on the English task drops from
We use bayer.head for referring to specific heads of interest. 10 ablation experiments carried out in the paper are done via zero-ablation, which refers to zeroing out the contrihution of certain head.
8

---

Published as a conference paper at ICLR 2025
FFN Ablation
Now visit. Yesterday also
visited
had
went
saw
took
met
did
came
spent
Zero-rank Rate
After
visiting
Before
10
15/9
EN
ZH S01R)
<space>
In
E88
+
Figure Illustration of the effects of late feed-forward layers ablation (layer 20-23). Left: Zero-rank rate changes comparison between English and Chinese past tense task. Right: Top predicted tokens comparison for before and after ablation. We observe that English performance is significantly impacted after ablation but Chinese is barely influenced. Qualitative results the right show that the correction token move backward in English but remain the top prediction in Chinese.
97.44% 47.44% and the zero-rank rate from 58.06% 17.74%. However, for the Chinese task, the zero-rank rate is barely changed (25.08% to 24.19%)
In the English example, when examining top predicted tokens qualitatively in Figure we see that the rank of the correct past tense answer visited moved back by 1 position after ablating the MLP layers. Additionally,
worth noting that morphological variants (E.g. visir, visits, etc.) or semantically relevant tokens (E.g. $ XL, 1 move to the top predictions. These are also tokens that are actively promoted by the copy heads mentioned earlier. In the Chinese example, we see that the top token as the correct prediction still remain the same position, unaffected by the FFN layers removal.
5 RELATED WORK AND DISCUSSION
Mechanistic Interpretability We built on previous work on mechanistic interpretability reverse engineer the mechanism of neural network. Circuits are a significant paradigm of model analysis that has emerged from this field. Originated from the vision model OOlah al. 2020) and continued
to transformer language model (Meng et al. 2023) Wang et al. [023 Hanna et al. 2023 Varma
al. 2023 Merullo et al. 2024 Lieberum et al. 2023 Tigges et 2023a), works in mechanistic interpretability try to characterize the function of individual components in completing certain tasks.
Beyond the work understanding the role of attention heads (Olsson et al. 2022 Chen al. 2024]
Singh et al.
2024 Gould et al.) 2023 McDougall et al. (023a), efforts are made in understanding
neurons and feed forward network in (Vig et al. 2020 Finlayson et al.) 2022] Sajjad et al. 2022
Gurnee et al.
2023 Voita al. 2023 Merullo et al. 2023). These studies mainly
causal methods
with minimal pairs to locate the components and their functionalities (Vig et al., 2020 Chan et al.
2022
Geiger et al. 2021 2023 Meng et al. 2023) Wang et al. 2023 Chan et al. [023]] Cohen al.
2023
However, these methods heavily rely on manual inspection, which limits their scalability to
larger models. Therefore other recent works have attempted automate the causal process (Conmy cet al. 2023 Bills et al. 2023 Syed et al. 2023 Hanna et al. (024) or use input attribution to trace the subgraph of important components lifting the burden of defining minimal pairs (Ferrando & Voita 2024)
Multilingual Interpretablity With English-centric large language models demonstrating out- standing generation and reasoning capabilities, more current efforts have been devoted to expanding such capabilities to multilingual settings (Workshop et al. 2022 Yang et al. 2024 Üstin et al.
2024 Aryabumi et al. 2024). Many works [Wu & Dredze 2019 Winata et al., 2022) have lever-
Note we cannot compute accuracy in Chinese as there is no past-tense alternative verb form against which
to compare.
Both words he translated to visit.

---

Published as a conference paper at ICLR 2025
aged transfer learning to boost performances on low-resource languages alternative to tackling data deficiency problems. Mcanwhile, language models have also been reported to suffer from the "curse of multilinguality" (Conneau 2019 Chang et al. 2023) where languages interfere with each
other given fixed model capacities.
Previous works have attempted investigate cross-lingual representation alignment in BERT based models (Artetxe et al. 2020; Muller et al., [022]] [onneau et al. 2020 Del & Fishel 2021 2022.
Hämmerl et al. 2024). observing the "First Align, the Predict" paradigm (i.e. models first con- verge cross-lingual representation to language neutrality then become language specific again across the layers). Other works also quantify their effects on transfer learning abilities on downstream tasks (Malkin et al.) [2022] [hoennii et al.) [023). More recent works such as |Wendler et al. (2024) observe that multilingual generative models go through language conversion phrases with the reasoning process mainly conducted in English. [Tang et al. (2024) specifically locates these language-specific neurons and manipulates the output language by activating or deactivating dif- ferent language-specifie neurons. [i et al. (2024) also observe that Multi-Layer Perceptron (MLP) is associated with how toxic concepts are encoded crosslingually. However, it remains unclear how models utilize their components when receiving inputs in different languages: when task is prompted in different languages, are the underlying circuits shared or does there exist completely separate paths? While Merullo et al. (2024) has reported the reuse of circuit components across different tasks in the same language, no prior work has investigated this phenomenon at the cross- lingual level. Additionally, considering language construction is unlikely to be exactly the same across languages, how are model components used to reflect these differences? In this paper, with the help of mechanistic interpretability techniques like path patching and information flow routes, we provide an initial analysis to quantify to what extent model components are shared when solving tasks for different languages.
We investigate when and how large language models (LLMs) encode structural similarities between languages. Specifically, we ask (1) whether linguistic tasks that are structurally similar across languages are handled by the same L.M circuitry and (2) whether linguistic tasks that employ language-specific structure are handled by specialized LLM circuitry. Through a series of exper- iments employing different tasks and models in both English and Chinese, we find that indeed cross-lingual structural similarity is represented within the LLM using shared circuits which in- voke the same algorithm independent of language. We find that these shared algorithms emerge even in monolingual models trained on different languages. We also show that when tasks require language-specific caapooentt-spccffcalll past tense morphology in English-LLMS localize the corresponding processing such that it plays causal role in some languages but not in others.
This work builds previous literature (Artetxe et al. 2020 Muller et al.. 2021 Conneau et al. 2020) that discuss the "First Align, then Predict" paradigm, where parameter sharing between mod- els allows for better alignment of cross-lingual structures. Our work in generative models provide supportive evidence by diving deeper into the specific attention and FFN weights that are responsible for such structural similarities. Taken together, our results provide a first step in using mechanistic analysis about LLMs' inner workings in order to ask higher-level questions about how linguistic structures are reflected in state-of-the-art Al models.
Our results are limited to specific tasks and datasets, future work will be necessary to replicate and generalize our findings. Even so, our studies yield several interesting and encouraging insights which can inspire new lines of inquiry. For example, we find that the same circuit emerges across different models trained on entirely different languages. This raises questions about what type of structure is languages, even shared universally? That is, what regularities
shared across cnderwise-dissimile exist in the distribution of both English and Chinese such that they give rise to the same highly specific functional components organized in the same way? While the details of this discussion specific to I.Mss, the spirit shares much with long-running linguistic discussions of universality and of learnability in language (Yang) 2004). Deeper discipliiiyl collaborations could possibly exploit LLMS in order to gain fresh perspectives on such questions. Another important direction concerns practical considerations for developing state-of-the-art technology in languages other than English. This includes not only building LLMs in other languages (Wu et al. 2024). but also ensur- ing their safety and robustness (Yong et al. 2023 Deng et al.) (223) across languages. Insights into specific mechanisms that are shared between languages could guide more principled approaches to transferring parameters or anticipating failure modes. Future work could explore whether circuit overlap like what we document here can be exploited towards these goals.
10

---

Published as a conference paper at ICLR 2025
6 ACKNOWLEDGEMENTS
We thank Benjamin Muller, Zhaofeng Wu, Javier Ferrando, Yong Zheng-Xin, Jacob Li, Rui-Jic Yew, members of the Lunar lab and Health NLP lab at Brown, and the anonymous reviewers for their valuable discussions and feedback on this work.
REFERENCES
Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. On the cross-lingual transferability of monolingual representations. Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.). Proceedings of the 58th Anmual Meeting of the Association for Computational Lin- guistics, pp. 4623-4637, Online, July 2020. Association for Computational Linguistics. doi: 22.18663/11/2220.cc/mma...
UtL//tttss//cclaathho.... .aagy.ogg//// /200/.002222/ acl- awt.mmma......minn....a.aa 421
Viraat Aryabumi, John Dang, Dwarak Talupuru, Saurabh Dash, David Cairuz, Hangyu Lin, Bharat Venkitesh, Madeline Smith, Kelly Marchisio, Sebastian Ruder, et al. Aya 23: Open weight leases to further multilingual progress. arXiv preprint arXiv:2405. 2024.
Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardr. Eliciting latent predictions from transformers with the tuned lens, 2023.
Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O' O'Brien, Eric Hal- lahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar Van Der Wal. Pythia: suite for analyzing large lan- guage models across training and scaling. In Proceedings of the 40th International Conference on Machine Learning, ICML' 23. JMLR.org 2023.
Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models explain neurons in language models.
blob. core. windows. www//eeronneepppeeeeee
paper/index. html 2023.
Lawrence Chan, Adrià Garriga-Alonso, Nicholas Goldwosky-Dill Ryan Greenblatt, Jenny Nitishinskaya, Ansh Radhakrishnan, Buck Shlegeris, and Nate Thomas. Causal scrubbing, a method for rigorously testing interpretability hypotheses. Al Align- ment Forum, 2022.
hhtpps://wwel al forent ottp/ppstss///hhhzccz222777777 causal-scrubbing how.o//nnoeeeo.ooooooooooooooorr
Lawrence Chan, Adrià Garriga-Alonso. Nicholas Goldowsky-Dill, Ryan Greenblatt, Jenny Ni- tishinskaya, Ansh Radhakrishnan, Buck Shlegeris, and Nate Thomas. Causal scrubbing: method for rigorously testing interpretability hypotheses [redwood research]. Align- ment Forum, 2023. URL htttp:::/www..llgmnnffffo.r..../ orarrrumoor.//.ooomoo htt.g///sstt//////tccutu//s wwwp://bwnnn.aaamaoooooo-rrrrrrr
Accessed: 17th Sep 2023.
Tyler A Chang, Catherine Arnett, Zhuowen Tu, and Benjamin Bergen. When multilin- guality curse? language modeling for 250 high-and low-resource languages. arXiv preprint arXiv:2311.09205. 2023.
Angelica Chen, Ravid Shwartz-Ziv Ziv, Kyunghyun Cho, Matthew L Leavitt, and Naomi Saphra. Sud- den drops in the loss: Syntax acquisition, phase transitions, and simplicity bias in MLMs. In The Twelfth International Conference on Learning Representations, 2024. URL https: httprrreeweeeeeeeooummiiaaaaaaee
Rochelle Choenni, Dan Garrette, and Ekaterina Shutova. How do languages influence each other? studying cross-lingual data sharing during Ilm fine-tuning. arXiv preprint arXiv:2305. 13286, 2023.
Rochelle Choenni, Ekaterina Shutova, and Dan Garrette. Examining modularity in multilingual Ims via language-speccaalzee subnetworks. In Findings of the Association for Computational Linguistics: NAACL 2024, pp. 287-301, 2024.

---

Published as a conference paper at ICLR 2025
Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. Evaluating the ripple effects of knowledge editing in language models, 2023.
Arthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonoo. Towards automated circuit discovery for mechanistic interpretability. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https:
A Conneau. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116. 2019.
Alexis Conneau, Shijie Wu, Haoran Li, Luke Zettlemoyer, and Veselin Stoyanov. Emerging cross- lingual structure in pretrained language models. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.). Proceedings of the 58th Annual Meeting of the Association for Computa- tional Linguistics, pp. 6032, Online, July 2020. Association for Computational Linguistics. httpst ptrtps://cclanthooog..rrgg
doi: 10. 12263//112020.all-maia
acl-main. 536
Guy Dar, Mor Geva, Ankit Gupta, and Jonathan Berant. Analyzing transformers in embedding space, 2023.
Maksym Del and Mark Fishel. Similarity of sentence representations in multilingual Ims: Resolving conflicting literature and a case study of baltic languages. Balt. J. Mod. Comput.. 10, 2021. URL https://ap1: semanti
www.:/arrrrr///ooosssi://///// 249991132
Maksym Del and Mark Fishel. Cross-lingual similarity of multilingual representations revisited. In Yulan He, Heng Ji, Sujian Li, Yang Liu, and Chua-Hui Chang (eds.). Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 195, Online only, November 2022. Association for Computational Linguistics. URL.
wrwp://00ooy.r./2222.../ aacl- 15.
Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. Multilingual jailbreak challenges in large language models. arXiv preprini (ex:Xi::310006444 2023.
Javier Ferrando and Elena Voita. Information flow routes: Automatically interpreting language models at scale. arXiv preprint arXiv:2403.0088 2024.
Matthew Finlayson, Aaron Mueller, Sebastian Gehrmann, Stuart Shieber, Tal Linzen, and Yonatan Belinkov. Causal analysis of syntactic agreement mechanisms in neural language models. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1828-183, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/w//2011 1atl://..1/00//111110 lacl-long. lon.lloo....... 144. URL https://acllnthhnn
org. 2021 acl acl-lon.... 144
David Freedman, Robert Pisani, and Roger Purves. Statistics (international student edition). Pisani, R. Purves, 4th edn. WW Norton & Company, New York, 2007.
Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. Causal abstractions of neural networks. In Advances in Neural Information Processing Systems, vol- ume 34, pp. 9574-9586, 2021. URL nttps://papers.ntt hctppssc///aperr22///assss (F5c22744aaa00reea243333338288 Absttattttt
Atticus Geiger, Christopher Potts, and Thomas Icard. Causal abstraction for faithful model interpre- tation. Ms., Stanford University, 2023. URI. http:httss///ii...tttt http:/iwww.rr//b///223/3 04709
Nicholas Goldowsky-Diil Chris MacLeod, Lucas Sato, and Aryaman Arora. Localizing model behavior with path patching. arXiv preprint arXiv:2304. 05966, 2023.
Rhys Gould, Euan Ong, George Ogden, and Arthur Conmy. Successor heads: Recurring, inter- pretable attention heads in the wild, 2023.
Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsi- mas. Finding neurons in haystack: Case studies with sparse probing, 2023.
12

---

Published as a conference paper at ICLR 2025
Katharina Hämmerl, Jindfich Libovický, and Alexander Fraser. Understanding cross-lingual Alignment- -A survey. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguisties: ACL 2024, pp. 10922-10943. Bangkok, Thailand, August 2024, Association for Computational Linguistics. doi: 10. 111663/11/2224.fniinnng 649. URL htt.h/tsss/acllannoooggo.oooo www.ollooooorg/2224..4. findiiiise...iinn--siiaaae 649
Michael Hanna, Ollic Liu, and Alexandre Variengien. How does GPT-2 compute greater-than?: Interpreting mathematical abilities in pre-trained language model. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL http:https///ppoo httpps://openrevieww www.://wwteeoouumimiiimmm p4PckNOR8K
Michael Hanna, Sandro Pezzelle, and Yonatan Belinkov. Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms, 2024.
Yair Lakretz, Dicuwke Hupkes, Alessandra Vergallito, Marco Marelli, Marco Baroni, and Stanislas Dehaene. Mechanisms for handling nested dependencies in neural-network language models and humans. Cognition, 213: 104699, 2021.
Xiaochen Li, Zheng-Xin Yong, and Stephen H Bach. Preference tuning for toxicity mitigation generalizes across languages. arXiv preprint 1XXi::2406.16235,, 2024.
Tom Lieberum, Matthew Rahtz, János Kramár, Neel Nanda, Geoffrey Irving, Rohin Shah, and Vladimir Mikulik. Does eircuit analysis interpretability scale? evidence from multiple choice capabilities in chinchilla, 2023.
Dan Malkin, Tomasz Limisiewicz, and Gabriel Stanovsky. A balanced data approach for evaluating cross-lingual transfer: Mapping the linguistic blood bank. In Marine Carpuat, Marie-Cathering de Marneffe, and Ivan Vladimir Meza Ruiz (eds.), Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4903-4915, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10. 10.18653/1/2022..aaaa www.:/222aaacmmmi..a3... URL https:://aclatthhttttt htwprhhhlllgy..rg/222222 /
naacl-main. 361
Callum McDougall, Arthur Conmy, Cody Rushing, Thomas McGrath, and Neel Nanda. Copy sup- pression: Comprehensively understanding an attention head, 2023a.
Callum McDougall, Arthur Conmy, Cody Rushing, Thomas McGrath, and Neel Nanda. Copy sup- pression: Comprehensively understanding an attention head, 2023b. /ttp//tss://arxii.ooo org/ abs/2310.0aa 04625
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt, 2023.
Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. Language models implement simple word2vec- style vector arithmetic. arXiv preprint arXiv:2305. 16130 2023.
Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. Circuit component reuse across tasks in trans- former language models. In The Twelfth International Conference on Learning Representations, 2024. Utt/https::// http://perreiiiee..eeeefrrrrrrr. www.:/orrummd--poooooot...
Kanishka Misra and Najoung Kim. Generating novel experimental hypotheses from language mod- els: A case study on cross-dative generalization. arXiv preprint urXiv:2408.05086 2024.
Benjamin Muller, Yanai Elazar, Benoît Sagot, and Djamé Seddah. First align, then predict: Understanding the cross lingual ability of multilingual BERT. In Paola Merlo, Jorg Tiede- mann, and Reut Tsarfaty (eds.), Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguisties: Main Volume, pp. 2214-2231, Online, April 2021. Association for Computational Linguistics. doi: 10.18653/v1/2011 eaa.1e.acl-miinn 189. URL https://aclanthooooy..r///// .ew.lmmi............mm 189
Neel Nanda and Joseph Bloom. Transformerlens. TransformerLens)] 2022.
https:///ithhn httts://ggthbbb.oom/eeeeeeaaa www./ooeooennnaaeeeem.em io/
13

---

Published as a conference paper at ICLR 2025
Nostalgebraist. Interpreting gpt: the logit lens, 2020. AclleBbwpiaalvvevv/ieeerrrrtnnne -heeee-- lens
Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter.
Zoom in: An introduction to circuits. Distill, 2020. doi: 2/2911//istill..0022..00
Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodde Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. In-context learning and induction heads. Transformer Circuits Thread, 2022. hutps:/transffomer-crcuisss wwtp:/mmreriiiiiisusu////////ee cetiii-ooootettticinneee -aatttt-eennnnn----iiiaii and- www.etiiiiinnhad//ddeed... html.
Telmo Pires, Eva Schlinger, and Dan Garrette. How multilingual is multilingual BERT? In Anna Korhonen, David Traum, and Lluís Märquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4996 5001, Florence, Italy, July URL https:
2019. Association for Computational Linguistics. doi: 10.18653 //aclanthology. -
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Hassan Sajjad, Nadir Durrani, and Fahim Dalvi. Neuron-level interpretation of deep nip models: A survey, 2022. URL httpp:://axxii.rrggg///e//
wwwrrrivvvoorga..o wtt.vv...rggbb///.//// 13138
Aaditya K. Singh, Ted Moskovitz, Felix Hill, Stephanie C. Y. Chan, and Andrew M. Saxe. What needs to go right for an induction head? mechanistic study of in-context learning circuits and their formation, 2024.
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.046/1 2022.
Aaquib Syed, Can Rager, and Arthur Conmy. Attribution patching outperforms automated circuit discovery, 2023.
Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, and Ji-Rong Wen. sanguage-specingg neurons: The key multilingual capabilities in large language models. arXiv preprint arXiv:2402.16433 2024.
Curt Tigges, Oskar John Hollinsworth, Atticus Geiger, and Neel Nanda. Linear representations of sentiment in large language models, 2023a.
Curt Tigges, Oskar John Hollinsworth, Atticus Geiger, and Neel Nanda. Linear representations of sentiment in large language models, 2023b. URL htttps://aaii..or//bb/////
Igor Tufanov, Karen Hambardzumyan, Javier Ferrando, and Elena Voita. Lm transparency tool: Interactive tool for analyzing transformer language models. arXiv preprint arXiv:2404.07004, 2024.
Ahmet Üstün, Viraat Aryabumi, Zheng-Xin Xin Yong, Wei Yin Ko, Daniel D'souza Gbemileke Onilude, Neel Bhandari, Shivalika Singh. Hui-Lee Ooi, Amr Kayid, et al. Aya model: An in- struction finetuned open-access multilingual language model. arXiv preprint 2rX1v:2402.07827, 2024.
Vikrant Varma, Rohin Shah, Zachary Kenton, János Kramár, and Ramana Kumar. Explaining grokking through circuit efficiency, 2023. https:////// http:://w/wxxwo//////// absa 2309. 02390
Jesse Vig. Sebastian Gehrmann. Yonatan Belinkov, Sharon Qian, Daniel Nevo, Simas Sakenis, Jason Huang. Yaron Singer, and Stuart Shieber. Causal mediation analysis for interpreting neural nlp: The case of gender bias, 2020.
14

---

Published as a conference paper at ICLR 2025
Elena Voita, Javier Ferrando, and Christoforos Nalmpantis. Neurons in large language models: Dead, n-gram, positional, 2023.
Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Inter- pretability in the wild: a circuit for indirect object identification GPT-2 small. In The Eleventh International Conference on Learning Representations, 2023. URL https: http:///ppeeeveeee evenw..eie..... net// forrmniiG-NsVVNNGAAul
Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. Do Ilamas work in en- glish? on the latent language of multilingual transformers. arXiv preprint arXiv:2402. 2024.
Genta Winata, Shijie Wu, Mayank Kulkarni, Thamar Solorio, and Daniel Preotiuc-Piett Cross- lingual few-shot learning unseen languages. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pp. 777-791, 2022.
BigScience Workshop, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, et al. Bloom: A 76b-parameter open-access multilingual language model. arXiv preprint urXiv:2211.55100, 2022.
Shijie Wu and Mark Dredze. Beto, bentz, becas: The surprising cross-lingual effectiveness of bert. arXiv preprint arviv: 1904.00977 2019.
Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger. Dan Jurafsky, Christopher D Man- ning. and Christopher Potts. Reft: Representation finetuning for language models. arXiv preprint arXiv:2404.0359, 2024.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024.
Charles D Yang. Universal grammar, statistics both? Trendy in cognitive sciences, 8(1))451-456. 2004.
Zheng-Xin Xin Yong, Cristina Menghini, and Stephen H Bach. Low-resource languages jailbreak gpt-4. arXiv preprint arXhv:231000246,, 2023.
Qinan Yu, Jack Merullo, and Ellie Pavlick. Characterizing mechanisms for factual recall in language models. arXiv preprint arXiv:2310.15910 2023.
Zhengyan Zhang, Xu Han, Hao Zhou, Ke Pei, Yuxian Gu, Deming Ye, Yujia Qin, Yusheng Su, Haozhe Ji, Jian Guan, et al. Cpm: A large-scale generativ e chinese pre-tned language model. arXiv preprint arXiv:2012.00411 2020.
15

---

Published as a conference paper at ICLR 2025
A TASK BENCHMARK PERFORMANCES
In this section, we present the accuracy and zero-rank rate of both the IOI tasks and Past Tense tasks discussed in & 3 and $ [ In Table I we see that all monolingual models and multilingual models perform well on the IOI tasks. For the past tense task, BLOOM obtains good performance English but not in Chinese. In comparison, Qwen performs slightly worse in English but much better in Chinese. As result, we focus Qwen for our language-specific analysis.
Model
Lang.
Accuracy
Zero-Rank Rate
101 Task
normal
flippdd
normal
flipped
GPT2-small
EN
99.5
99.5
97.5
95.5
CPM-Generatt-disiill
ZH
84.5
86.5
57.5
73.0
BLOOM 560M
EN
100.0
100.0
96.0
91.0
BLOOM-560M
ZH
95.0
100.0
93.0
99.5
Past Tense Task
normal
flipped
normal
flipped
BLOOM- -50M
EN
90.3
51.6
80.7
37.1
BLOOM- -500
ZH
12.9
1.6
Swen2-0.5B-insmmen
EN 96.8
69.4
58.1
24.2
Swen2-0.5B..o -.5-intct
ZH
25.9
21.0
Table 1: Model performances for the IOI and past tense tasks. Note that we unable to compute accuracy for the past tense task in Chinese (marked by -) due to the absence of minimal pairs.
B PATCHING DETAILS OF THE IOI TASK
B.1 MULTILINGUAL MODELS
To extract the IOI circuits the BLOOM model, we use the transformer-lens library (Nanda & Bloom 2022) and follow the practices of Wang et al. (2023). Figure 6 shows the main steps to discover specific function heads for both English and Chinese tasks BLOOM. Throughout each of the steps. we notice the important heads discovered between English and Chinese are highly similar, leading to largely overlapping circuits. Figure 7 validates the functions of the Duplicate Token Heads, Previous Token Heads and Induction Heads found by showing their attention scores on sequences of random tokens. Please refer to more detailed definitions and experiment designs on these metrics in [Wang al. (2023). Additionally, we provide IOI circuits identified in Qwen2-0.5B- instruct by carrying out the same procedure as above (shown Figure g
B.2 MONOLINGUAL MODELS
For path patching on CPM, we add the CPM models to the transformer-lens library and follow the same steps as the multilingual model described above. Figure D presents the comparison between English and Chinese inputs when we patch to the residual stream on GPT2 and CPM respectively. Notice that here the heatmap patterns are quite different because they are trained on completely different data despite the same architecture (with the same number of layers and heads). Then we perform patching to individual heads and follow Wang et al. (2023) to find out their functionalities. Figure 9 Right shows the attention patterns of the name mover heads and copy suppression heads. We see that mover heads 8. 9 and 9. show copying behaviors toward the I0 tokens and 11.7 shows copying behavior toward both 10 and S tokens. The copy suppression head 11. 2 on the other hand suppresses the copying action of the token. Figure I0 shows our validation experiments on the functions of Induction heads, Duplicate Token Heads and S-inhibition heads. Specifically for the S-inhibition head 7. 0, when we ablate the head, the original name movers stop preferring the I0 tokens but start to promote both 10 and S tokens.
16

---

Published as a conference paper at ICLR 2025
English Input
Chinese Input
Z-patching to residual stream
Layers
S-inhibition
Q-patching name movers
Name Mover
Negative Name Mover
V-patching to S- inhibition heads
layans
15
Heads
Figure 6: Path Patching steps for extracting 1OI circuits on BLOOM-560M, Left: English Input.
Right: Chinese Input.
C PATCHING DETAILS OF THE PAST TENSE TASK
C.I PATH PATCHING
For the Past Tense task, we conduct path patching with the English templates since there are no minimal pairs available for Chinese. As shown in Figure Left, two heads layer 19 the most
17

---

Published as a conference paper at ICLR 2025
soguanors Handom tokata (Ergl
soQuEnoos randem tokers (Bngian)
SEcuATIOS rerdom toerss (Engltik)
0.6
0.4
-04
Dusticane takes attertion preb.
Prex attenton prob.
Induction tolen attention
(Chanese)
0.5
Layer
20
Head
Head
Figure 7: Metrics for Duplicate Token Head, Previous Token Head, and Induction Head on English Inputs (Top) and Chinse Inputs (Bottom).
prominent. We later use head attribution and see that they are past tense heads that pay attention general past-tense concepts and suppress present-tense verbs (shown in Figure 12p. In Figure []] Right, we project all the head outputs to the vocabulary space and see how much each head writes the direction of either past tense verbs, present tense verbs English, or verbs in Chinese. Again we notice that the patterns are similar and few heads on Layer 21 writes strongly to both past tense and present tense verbs. These heads also show up in the path patching results but do not influence the final logits as strongly the past tense heads. By examining the top- promoted tokens by these heads, we see that these heads promote orthographically similar or semantically adjacent tokens. These could contain verb forms with various morphological markers as well as synonyms and antonyms (See Figure 12]. More detailed discussions can be seen in
C.2 ABLATION EXPERIMENTS
Figure 12 showws rank changes the verbs and contains qualitative examples when each type of the function -anecion-specific heads are zero-ablated. In general, we notice that past tense heads are specifically activated for English tasks only and therefore the rank of past tense verbs moves forward when these heads are ablated. However, the prediction does not get impacted when we ablate these heads for the Chinese task. Copy heads, the other hand, important for both English and Chinese tasks.
18

---

Published as a conference paper at ICLR 2025
When
Mary
15.98
end
80
John
2948
Pravious Token Head 121551495511111
1
Duplicate
10111503 1.11 1188 (2.2 18.8
store
#b10
S-Inhibition Head 14.5 157 18.5 120 210 22.5
Name Mover Head 21. 2131 2112
gava 3
1/Vig
Negative Name Mover
snack
END
to
# 7
Figure 8: Illustration of IOI circuits of English and Chinese on Swven-00.B-instun model. Blue- framed rectangles or texts are the components used in English IOI and orange- frammed ones are for Chinese IOI.
Log1o1
Projection
Namt Mover
Propecters eatoe of ololen mere Nome Mover
English GPT2
O%
-00%
Prejatie the cupa sing
Casy Suppression
Name Mover
extwoiiing dnddddd
Chinese CPM
J0N
Figure 9: Left: Path patching comparison between GPT2 English task and CPM on Chinese task. Right: Attention probability vs. head projection output into the IO and S space.
Past-tense, present-tense verbs in English and Chinese verbs are all demoted significantly when copy heads are ablated, indicating these heads are shared components implementing the same functions for both English and Chinese circuits.
19

---

Published as a conference paper at ICLR 2025
Induction Head Score
Then, Jack Paul m. Jack gave...
Towan
Induction Heads
Head 5,5
-02
color o 6162274
10
Input: Paul Daniel Paul Daniel Paul Daniel Paul Daniel
Head 1.8
Head 4.2
Duplicate Token Head
10
Propecton of embeddinng vs atention probabiiity on name
Projection of the oubuut 0.30 along the name enteedding va alsemion probutaity nume
S-Inhibition Head
prok
Figure 10: Top: Induction head scores random tokens and its attention pattern. Middle: Duplicate token attention patterns on ABAB random strings. Bottom: S-inhibition head ablation effects on name movers.
20

---

Published as a conference paper at ICLR 2025
Patching
0.1
10
laye
22
15
-0.1
Figure 11: Left: Path Patching on English Past Tense Task. Right: Verb Promotion Heatmaps for English and Chinese Past Tense Task.
Post Tonse Head Ablation
350
Now frestore. Yosterday lalso
120
Intlored
After
rextored
startaed
EMProcR Auwe
We
Yesterday also
300
changed
1dd
wort
did
trled
look
made
wicte
stantol
300
After
+47
Figure 12: Effects of Past Tense Heads and Copy Head Ablation.
21


Reviews:
[{"review_id": "kxrLYfqOg4", "reviewer": "Reviewer_P3M4", "summary": "This paper aims to explore how similar or different circuits are across languages for linguistic tasks in language models. It takes a case-study approach, focusing on two tasks (IOI and past tense task), two languages (English and Chinese), and three language models (two for IOI and one for the past tense task). The past tense task involves morphology, which allows the authors to analyze how it’s handled in different languages.\n\nUsing techniques like path patching, the authors identify circuits and find significant overlaps between languages for both tasks. They also highlight interesting differences, like specific \"past tense\" heads in English.", "questions": "1. **Scope of Claims:**  \n    Could the authors clarify why they make general claims about multilingual models despite the case-study limitations? Would they consider rephrasing these claims to better align with the study's focused scope? Is there a little clash between a generality of the paper title and strict case-study experimental approach?  \n    \n2. **Support or Contrast with Alignment-based studies**\n\tHow do the authors’ findings support or differ from prior studies on multilingual alignment, especially those focusing on hidden state alignment? \n\n3. **Comparison with Coarse-Grained Studies:**  \n    How do the findings here align with or diverge from conclusions in prior, coarse-grained studies on cross-lingual interpretability? Can conclusions from prior work with support from this work's experiments together support general claims or gain novel broader conclusions?\n\n4. **Choice of Circuit-Level Analysis:**  \n    While I am very much in favor of it, I am interested to hear more about why the authors chose a circuit-level approach over other methods and what alternative methodology options they considered. What other mech interp approaches would help support the authors claims in future work (e.g. analysis with SAEs)?  \n    \n5. **Broader Application:**  \n    Do the authors have recommendations on which additional languages, models, or tasks would best extend this study’s findings?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "_Clarity:_  \nThe paper is clear and well-written, with high-quality illustrations and concise, precise language. The flow is logical, and the key points are emphasized effectively.\n\n_Quality:_  \nThe experiments are well-designed and effectively executed. The choice of mechanistic interpretability techniques to explore cross-lingual overlap is sound, and the methods are applied accurately. The IOI and past tense tasks are suitable examples; IOI builds on established monolingual work, while the past tense task introduces unique, language-specific morphological challenges.\n\n_Originality:_  \nThis work is notable for its detailed, fine-grained circuit-level analysis of cross-lingual overlap in models.\n\n_Significance:_  \nFor the NLP community, particularly those working on cross-lingual models and low-resource languages, this work provides a valuable deeper understanding of cross-lingual transfer down to the circuit level. This detailed approach might inspire new techniques for more effective cross-lingual transfer.", "weaknesses": "_Scope of Main Claims:_  \nIn my view, the primary contribution of this work lies in _demonstrating a specific case study_ of circuit-level cross-lingual transfer between languages in related or separate models. While the paper aims to make broader claims about multilingual models, the setup—two languages, two/three models, and two tasks—suits a focused case study rather than generalizable conclusions. Although the paper acknowledges these limitations, the claims in the introduction could be adjusted to better reflect the scope of the experiments.\n\nExpanding to more languages, models, and tasks would strengthen broader claims, but as a focused case study, this paper is effective on its own. I would recommend reframing the claims to align more closely with the experimental scope.\n\n_Related Work on Multilingual Interpretability:_  \nA broader review of related work could enrich the discussion, especially on different approaches that analyze multilingual models by aligning internal representations. For instance, [1] trains a monolingual transformer LM and transfers it to another language by retraining only the embedding matrix, hinting at cross-lingual structure similarities. This aligns with the monolingual IOI task analysis in this paper. [2] and [3] conduct coarse-grained analyses on similar cross-lingual interpretability challenges, while [4] aligns hidden states, and [5] provides further insights into the universality of these findings. [6] surveys cross-lingual alignment, offering broader yet less detailed conclusions on the inner workings of multilingual models. This paper can be seen as building on this older work by diving deeper into specific attention heads and showing detailed cross-lingual structures.\n\nIt would also be helpful to add a few sentences on how the conclusions here either support or contrast with prior work beyond the works already mentioned.\n\n**References**\n\n1 [On the Cross-lingual Transferability of Monolingual Representations - ACL Anthology](https://aclanthology.org/2020.acl-main.421/)\n\n2 [First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT](https://aclanthology.org/2021.eacl-main.189)\n\n3  [Emerging Cross-lingual Structure in Pretrained Language Models - ACL Anthology](https://aclanthology.org/2020.acl-main.536/)\n\n4 [[PDF] Similarity of Sentence Representations in Multilingual LMs: Resolving Conflicting Literature and a Case Study of Baltic Languages | Semantic Scholar](https://www.semanticscholar.org/paper/Similarity-of-Sentence-Representations-in-LMs%3A-and-Del-Fishel/0b749f70dd8cbe22fcfa7efeb571836b195c125d)\n\n5 [Cross-lingual Similarity of Multilingual Representations Revisited - ACL Anthology](https://aclanthology.org/2022.aacl-main.15/)\n\n6 [Understanding Cross-Lingual Alignment—A Survey - ACL Anthology](https://aclanthology.org/2024.findings-acl.649/)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PbCc9bGHIK", "reviewer": "Reviewer_EEXZ", "summary": "This paper compares how LLMs process syntactic phenomena in English vs Chinese. The authors use two tools for analysing the mechanisms underlying LLM computations, namely path patching and information flow routes. They consider two syntactic phenomena, one shared across English and Chinese and another unique to English. For the shared syntactic phenomenon (indirect object identification), they show that LLMs use similar internal circuitry across both languages. For the language-specific phenomenon (morphological markers in English only), they show that circuitry diverges. The results show that multilingual LLMs employ overlapping circuitry for linguistic phenomena common across languages, and that similar internal mechanisms emerge for related syntactic processing even in monolingual models for different languages. The paper also presents a detailed analysis of the LLM computation involved in the studied syntactic processes.", "questions": "(1) By “ablate”, I assume you mean zeroing out the contribution of a head to compare performance with/without the head? I suggest that you just mention t what you mean by “ablate” explicitly somewhere, since you use the word quite often in discussion your results e.g. “To validate our findings, we ablate the heads 19.4 and 19.5 and analyze their impact on the tasks.”\n\n(2) Section 4.5 can be organised a bit better. It contains quite a lot of information and requires the reader to refer to many figures. Potentially split the large paragraph (lines 403–418) into multiple subparagraphs, or organise the section into paragraphs with headings.\n\n(3) Line 297: Fix wording of “This divergence is not able,”.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "I'd like to congratulate the authors on an excellent and super interesting paper!\n\n(1) The analysis conducted by the authors is detailed, extensive, and extremely well presented. It provides concrete insights into the mechanisms underlying multilingual LLMs.\n\n(2) The methodology followed by the authors is sound and extensively documented, providing a valuable framework and example for future researchers to investigate the circuitry of multilingual LLMs.\n\n(3) The paper makes valuable contributions to multilingual interpretability, successfully unifying aspects of mechanistic interpretability with linguistically informed analysis.", "weaknesses": "Due to the heuristic nature of mechanistic interpretability tools like path patching, it’s difficult to conclude with certainty that findings such as those presented in this paper will generalise to other tasks/languages/models. However, the authors are careful to not overstate their claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lmEfuwr3XQ", "reviewer": "Reviewer_qgEk", "summary": "**Paper Summary**:\n- This paper shares some interesting findings on the similarities and differences between the internal structure of LLMs and the linguistic structure of English and Chinese. The authors employ mechanistic interpretability tools to explore the internal circuitry used by multilingual and monolingual LLMs when dealing with the same or different morphosyntactic processes. The findings indicate that there are common circuits and trade-offs when preserving linguistic differences.", "questions": "Do you have any initial ideas on how to conduct cross-disciplinary collaborations based on this work?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Summary Of Strengths**:\n- Interesting topic and findings: I find it very attractive to use mechanistic interpretability to identify the relationship between models and language structures. The conclusion about common components between multilingual and monolingual models provides insightful ideas for further research.", "weaknesses": "**Summary Of Weaknesses**:\n- Limitation: The work is based only on Chinese and English and is limited to a few tasks and datasets. The conclusions could be more convincing if more languages were included.\n- Clarity on future impact: It may be challenging to transfer the learning from this work to current multilingual LLM training efforts, which underscores the need to better understand the findings (e.g., cross-lingual transfer learning, parameter migration, etc.).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lTL6OrAdDa", "reviewer": "Reviewer_n6vb", "summary": "The paper assesses whether English and Chinese monolingual and multilingual language models use similar circuits for similar morphosyntactic processes (indirect object identification; IOI) and different circuits for differing processes (past tense inflection). They use path patching and information flow routes to identify circuits. They find that similar circuits are used for IOI in English and Chinese, for both monolingual and multilingual models. Overlapping circuits (in this case copy heads in attention) with language-specific components (in this case MLP blocks for English inflection) are used for past tense inflection.", "questions": "1. Could you provide some detail for how the attention head types and arrows in Figures 2 and 3 were identified? The figures imply fairly strong claims about how the internal algorithms work, so a description of the method used to identify the directed graph feels somewhat important.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The tasks and methodology appear sound and well thought-out.\n2. The paper is well-written, although some additional details would be very helpful (see weaknesses below).\n3. The ablation results are convincing, e.g. FFN ablation in Section 4.6 dropping performance specifically for English. Also Figure 11 in the Appendix showing the effect of ablating different attention heads.\n4. The results are interesting; language-agnostic and language-specific components have previously been proposed in multilingual language models (e.g. https://arxiv.org/abs/2205.10964; https://arxiv.org/abs/2109.08040; https://arxiv.org/abs/2009.12862), but the concrete demonstrations in this paper (specific circuits and their functions) are very informative.", "weaknesses": "1. The results in Section 3.5 and the last paragraph of Section 3.4 (\"We first use path patching to identify…\"; p. 5) are difficult to interpret without a good amount of background from Wang et al. (2023). A high level overview of how the different types of attention heads are quantified would be extremely helpful.\n2. Related to the previous point, the results indicating similar circuitry (Figures 2 and 3) rely quite a bit on the reader trusting that the algorithms visualized in Figures 2 and 3 are accurate. The description of how the head types and arrows are determined is fairly minimal (\"In both GPT2 and CPM, we identified similar circuits as Wang et al. (2023) outlined in Figure 3\"; p. 5), so evaluating the validity of the algorithms identified in the figures is difficult.\n3. Minor point: \"Anecdotally, we observe that many of the heads that are shared between English IOI and control are also shared with Chinese IOI... further work is needed to test this hypothesis quantitatively\" (p. 5). It could be helpful to have some very preliminary quantification, e.g. n% of the shared heads from English IOI and control are also shared with Chinese IOI, given some threshold for similarity.\n4. Consider moving Figure 11 into the main text if possible, since it is referenced several times in Section 4.5.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "evx2aKefmJ", "reviewer": "Reviewer_5YHy", "summary": "The submission uses interpretability tools to examine if the LLM’s mechanisms behind solving linguistic tasks are similar or differ across languages. Following Wang et al., 2023, the authors use path patching to identify information circuits in multilingual models corresponding to the same morphosyntactic task in two languages. The aim of the experiments is to examine whether such circuits will be the same (or similar) for solving tasks in distinct languages.\nThe paper focuses on two tasks: object identification and verb past tense inflection in English and Chinese. The performance in the tasks is checked via generic prompt filling. Experiments cover two multilingual models: BLOOM (in case of object identification) and Qwen2-0.5B-Instruct (verb past tense inflection).\n\nThe results show that in fact, the found circuits are similar across languages, suggesting that multilingual language models implement a shared mechanism for solving considered linguistic tasks across two languages (Chinese and English).", "questions": "- Does tokenization affect your experimental setting? More specifically, do you require that the queried property (object, past tense verb) be represented as one token?\n\n- Why don’t you present the results for IOI with Qwen2-0.5B-Instruct?\n\n- Why do you filter the English verbs with irregular past inflection?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The work tackles the timely problem of interpreting LMs’ functions across languages. Despite the community's focus on interpretability and multilingual models, not many works have tried to explain the mechanisms behind LMs' multilingualism.\n\n\n- The authors put the work in the context of past approaches in circuit identification (especially Wang et al. 2023), extending the experimental settings to multilingual models.\n\n\n- The experimental part studies the identified circuits in-depth: comparing across languages the overlap of heads in the circuits and the information flows.", "weaknesses": "- The linguistic scope of the experiments is limited. The method is applied only to two relatively simple morphosyntactic tasks in two languages. The results are close to 100% in object identification, which is a sign of saturation (a more complex task would be more informative). Only two distinct languages are considered. Therefore, the result does not show whether typological similarity could affect the similarity of the identified circuits.\n\n- The choice of models used for experiments is chaotic. It is not clear why Qwen2-0.5B-Instruct is not considered in the object identification task. The paper should present the results for both tasks, at least for two considered models.\n\n- The methodology should be described in more detail, focusing on its implications in the multilingual setting. For instance, I understand that the head types were already described in the work of Weng et al. 2023, but the natural question is what linguistic role these types have across different languages.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)