Causal Analysis of Syntactic Agreement Neurons in Multilingual Language Models (2022)

Abstract: Structural probing work has found evidence for latent syntactic information in pre-trained language models. However, much of this analysis has focused on monolingual models, and analyses of multilingual models have employed correlational methods that are confounded by the choice of probing tasks. In this study, we causally probe multilingual language models (XGLM and multilingual BERT) as well as monolingual BERT-based models across various languages; we do this by performing counterfactual perturbations on neuron activations and observing the effect on models’ subject-verb agreement probabilities. We observe where in the model and to what extent syntactic agreement is encoded in each language. We find significant neuron overlap across languages in autoregressive multilingual language models, but not masked language models. We also find two distinct layer-wise effect patterns and two distinct sets of neurons used for syntactic agreement, depending on whether the subject and verb are separated by other tokens. Finally, we find that behavioral analyses of language models are likely underestimating how sensitive masked language models are to syntactic information.

## introduction syntactic information is necessary for robust generalization in natural language processing tasks (for a case study using the natural language inference task, see @xcite . the success of pretrained language models (lms) such as roberta @xcite and gpt-3 @xcite in many nlp tasks has prompted hypotheses that they accomplish their performance through structural representations induced during pre-training, rather than only lexical or positional representations @xcite ; behavioral evidence for lms' syntactic abilities has been found in masked lms (mlms; @xcite @xcite and autoregressive lms (alms; @xcite . evidence for structural representations has been reported for multilingual pre-trained lms @xcite and in sequence-tosequence models @xcite . despite efforts to understand the structural information encoded by pre-trained lms @xcite @xcite @xcite ; inter alia), it remains unclear how and where multilingual models encode this information. most multilingual probing studies are correlational and use dependency parsing or labeling as a proxy task indicative of syntactic information @xcite . this is problematic: models do not need structural or word order information to achieve high performance on dependency labeling @xcite , and training a parametric probing classifier introduces many confounds @xcite . causal probing, however, enables nonparametric analyses of models through counterfactual interventions on inputs or model representations. causal probing studies have argued for the existence of specific syntactic agreement neurons and units in neural language models @xcite @xcite , but these studies have focused on monolingual models-usually (though not always) in english. causal methods allow us to make stronger arguments about where and how syntactic agreement is performed in pre-trained lms, and we can apply them to answer questions about the language specificity and construction specificity of syntactic agreement neurons. in this study, we extend causal mediation analysis @xcite @xcite to multilingual language models, including an autoregressive lm and a masked lm. we also analyze a series of monolingual mlms across languages. we employ the syntactic interventions approach of @xcite on stimuli in languages typologically related to english, such that we can observe whether there exist syntax neurons that are shared across a set of languages that are all relatively high-resource and grammatically similar. our contributions include the following: our data and code are publicly available. foot_0 ## discussion we observe two distinct layer-wise nie patterns for syntactic agreement, depending on whether the subject and verb are adjacent or separated by other tokens. this extends the findings of @xcite to multilingual mlms and alms, as well as monolingual mlms in various languages. going beyond their findings, we ruled out the possibility that these neurons do not simply track semantic plausibility or bigram collocations more generally. while this is not conclusive evidence that these neurons are specialized to syntax, evidence from other behavioral and probing studies also supports the existence of neurons focused on syntax @xcite @xcite found neurons focused purely on syntax, while @xcite found redundantly encoded syntactic information across neurons. it is not clear how much of the neuron overlap we observe is due to redundantly encoded information, but future work could investigate this. a consistent trend across our experiments is that alms encode syntactic agreement in a distinct way from mlms. in alms, there is more cross-lingual and cross-structure neuron overlap than in mlms; more similar layer-wise effect patterns across structures and languages (though they are still distinct); and a greater proportion of neurons which are sensitive to agreement. this could be partially explained by alms' left-to-right processing of natural language input, which more closely resembles incremental inputs to human learners. mlms are able to perform syntactic agreement @xcite , but their fill-in-the-blank pre-training objectives may induce distinct representations of sentence structure as compared to models that process or predict inputs incrementally. why do we observe different indirect effect contours for short-and long-range agreement? perhaps syntactic agreement is encoded using a single mechanism, but the way that syntactic information is used for predicting output tokens depends on the structure of the input or prior output tokens. alternatively, there could be two completely distinct agreement mechanisms that function in different ways entirely. while our findings do not disambiguate between these possibilities (or some other separate type or amount of mechanisms), future work could employ methods like those in @xcite to observe this distinction more explicitly. the findings of @xcite suggest that the model regions that are implicated in saying something are distinct from those implicated in knowing something-that is, knowledge retrieval and predicting particular tokens are separate mechanisms in pre-trained language models. perhaps their method could be extended to study syntactic agreement, such that we can better understand what, exactly, these distinct indirect effect trends represent. ## conclusions we have used causal mediation analysis to observe which neurons track syntactic agreement in multi-lingual pre-trained language models, and in which layers they are concentrated. we found two distinct layer-wise contours for syntactic agreement regardless of the language, multilinguality, or architecture of the model ( §4.1); that syntax-sensitive neurons are shared across languages in autoregressive language models ( §4.2.1); and that the neuron sparsity of syntactic agreement is similar in monolingual and multilingual models ( §4.3). we also found that behavioral analyses of masked language models obscure the extent to which their neurons are sensitive to syntactic agreement ( §4.3). processing (emnlp-ijcnlp), pages 833-844, hong kong, china. association for computational linguistics. zhilin yang, zihang dai, yiming yang, jaime carbonell, russ r salakhutdinov, and quoc v le. 2019. xlnet: generalized autoregressive pretraining for language understanding. in advances in neural information processing systems, volume 32. curran associates, inc.