On the Similarity of Circuits across Languages: a Case Study on the Subject-verb Agreement Task (2024)

Abstract: Several algorithms implemented by language models have recently been successfully reversed-engineered. However, these findings have been concentrated on specific tasks and models, leaving it unclear how universal circuits are across different settings. In this paper, we study the circuits implemented by Gemma 2B for solving the subject-verb agreement task across two different languages, English and Spanish. We discover that both circuits are highly consistent, being mainly driven by a particular attention head writing a ‘subject number’ signal to the last residual stream, which is read by a small set of neurons in the final MLPs. Notably, this subject number signal is represented as a direction in the residual stream space, and is language-independent. Finally, we demonstrate this direction has a causal effect on the model predictions, effectively flipping the Spanish predicted verb number by intervening with the direction found in English.

## introduction the widespread use of large language models (llms; @xcite @xcite highlights the importance of research dedicated to interpreting how these models work internally @xcite , especially to ensure they are safe. mechanistic interpretability (mi) @xcite aims to reverseengineer the algorithms implemented by language models. a large set of mi works have focused on circuit analysis @xcite , which locates subsets of components responsible for a behavior while giving human-understandable explanations of their roles. this research has made progress in identifying circuits that handle different tasks @xcite @xcite @xcite . however, it remains 1 code will be released upon acceptance. ## experimental setup in our experiments, we use gemma 2b model @xcite . this model has a large vocabulary size (256k tokens), making it particularly well-suited for circuit analysis, especially when doing activation patching (section 3) in a multilingual setting, since it has a large set of non-english words with a reserved token. regarding the dataset, for the english experiments use the subject-verb agreement (sva) dataset from arora et al. (2024) foot_0 , built on top of syntaxgym @xcite . the dataset consists of contrastive pairs that differ in the subject number, which agrees with the verb form continuation. this allows us to create 'clean' and 'corrupted' versions: clean: the executive that embarrassed the manager has corrupted: the executives that embarrassed the manager singular plural ## methods we start searching for a circuit in gemma 2b for solving the sva in english. to do so we use common techniques in circuit analysis, mainly direct logit attribution, activation patching, and attention pattern analysis. direct logit attribution. every model component adds a vector f c (x) to the residual stream, and the last residual stream state gets projected onto the unembedding matrix, producing the logits distribution. due to the linearity of the residual stream, the direct effect of a component to the logits can be measured by projecting its output onto the unembedding matrix, f c (x)w u . we can also measure the direct attribution to the logit difference (dlda) (yin and neubig, 2022; wang et al., 2023) of the two possible verb continuations (g and b): activation patching. a transformer lm can be seen as a directed acyclic graph (dag) representing a causal model @xcite @xcite , where nodes are model components, and edges representations. during the forward pass on the corrupted input x we can intervene on the value of a node, f c (x), or residual stream state, f l (x) by taking the activation value from the forward pass on the clean input x. this is referred to as denoising activation patching @xcite . we can express the intervention using the do-operator (pearl, 2009) as f (x|do(f c (x) = f c (x))). via a metric m we measure how the prediction changes between both runs: we are interested in finding components that increase the clean verb prediction when patching on the corrupted run. thus, a natural choice for the patching metric m is the logit difference between the clean and the corrupted verbs' logits. in the e m b a tt n 0 m l p 0 a tt n 1 m l p 1 a tt n 2 m l p 2 a tt n 3 m l p 3 a tt n 4 m l p 4 a tt n 5 m l p 5 a tt n 6 m l p 6 a tt n 7 m l p 7 a tt n 8 m l p 8 a tt n 9 m l p 9 a tt n 1 0 m l p 1 0 example 1, this means computing the logit difference between 'has' and 'have', and we expect it to increase as we patch activations from the clean (which includes 'executive') into the corrupted forward pass. ## english subject-verb agreement circuit locating relevant components and residual stream states. we perform activation patching on the residual stream states across the dataset and show the average logit differences foot_1 in figure 1 (a) . we can see that the noun in the subject largely impacts the prediction, and patching at its position in early layers causes the verb prediction to aggressively change to match its number. information from the subject flows towards the last residual stream via the attention block at layer 13 (figure 1 (b)), followed by some action from downstream mlps at the last position (figure 1 (b) ), especially mlp at layer 13 (mlp13). we can also observe that 'that' and the following verb ('embarrassed') get information from the subject at middle layers. we get a more granular understanding of the attention layers that seem relevant by doing activation top promoted tokens positive neuron activation ' are', 'are', 'were', ' were', 'are', 'aren', ' are', ' were', ' weren' top promoted tokens negative neuron activation ' gardent', ' is', ' has', ' sembrano', ' was', ' continúan', ' appartienment', ' isn', ' hasn', ' sostu' ## spanish subject-verb agreement circuit to study the subject-verb agreement task in spanish, we follow the style of the english dataset, where we first prompt gpt4 spanish circuit is consistent with the english circuit. with activation patching we see a similar pattern to that of the english dataset. information from the subject flows to the last residual stream at layer 13, where the attention block shows a large effect (figure 6 ). also similarly, downstream mlps are relevant for correctly solving the task, with mlp13 showing the highest total effect (figure 6 (b)), while mlp15, mlp16 and mlp17 having large direct effects on the logit difference. the contribution of mlp17 is notably greater than in the english dataset (figure 2 ), where we observe non-english specific neurons (appendix d). activation patching on individual attention heads (figure 6 (d )) shows that, as in the english dataset, attention heads l13h7 and l17h4 have a positive influence on the correct verb form, while l17h7 influences negatively. activation steering. in both languages, the same attention head (l13h7) composes with specific neurons in downstream mlps that are responsible for the correct verb form prediction, suggesting that this head writes a 'subject number' signal, which is found via pc1 (figure 5 ). here, we study whether this direction, found in 50 english examples (pc1 english ) has a causal effect on the model predictions, also on spanish sentences. specifically, we do activation steering @xcite @xcite on the attention head output at the last position (n) ## related work our work builds upon and extends several key studies in the field of probing neural language models for syntactic knowledge, particularly focusing on agreement mechanisms in multilingual contexts. causal probing studies have provided evidence for the existence of specific syntactic agreement neurons in language models @xcite @xcite . furthermore, research has demonstrated that models like @xcite rely on a linear encoding of grammatical number to solve the number agreement task @xcite . however, these studies have primarily focused on monolingual models, leaving a gap in our understanding of multilingual contexts. moving beyond monolingual studies, chi et al. (2020) investigated universal grammatical relations in multilingual bert. they developed a structural probe @xcite , learning a linear mapping to a syntactic subspace where syntactic features overlap between languages. this study demonstrated the potential for identifying cross-lingual syntactic similarities in multilingual models. @xcite conducted a causal analysis of syntactic agreement neurons in multilingual language models. via counterfactual interventions, they discovered significant overlap between languages in terms of neurons that causally influence syntactic agreement. ## conclusion in this work, we study how gemma 2b solves the subject-verb agreement task in two different languages, english and spanish. through activation patching and direct logit attribution we find that both languages rely on circuits that are highly consistent. moreover, we provide evidence of an attention head (l13h7) writing a 'subject number' signal as a direction from which downstream neurons read to promote the correct verb number continuation. finally, we show this direction has a causal effect, being able to flip the predicted verb number across languages.