Naturalistic Causal Probing for Morpho-Syntax (2023)

Abstract: Probing has become a go-to methodology for interpreting and analyzing deep neural models in natural language processing. However, there is still a lack of understanding of the limitations and weaknesses of various types of probes. In this work, we suggest a strategy for input-level intervention on naturalistic sentences. Using our approach, we intervene on the morpho-syntactic features of a sentence, while keeping the rest of the sentence unchanged. Such an intervention allows us to causally probe pre-trained models. We apply our naturalistic causal probing framework to analyze the effects of grammatical gender and number on contextualized representations extracted from three pre-trained models in Spanish, the multilingual versions of BERT, RoBERTa, and GPT-2. Our experiments suggest that naturalistic interventions lead to stable estimates of the causal effects of various linguistic properties. Moreover, our experiments demonstrate the importance of naturalistic causal probing when analyzing pre-trained models. https://github.com/rycolab/naturalistic-causal-probing

## introduction contextualized word representations are a byproduct of pre-trained neural language models and have led to improvements in performance on a myriad of downstream natural language processing (nlp) tasks (joshi et al., 2019; kondratyuk, 2019; zellers et al., 2019; brown et al., 2020). despite this performance improvement, though, it is still not obvious to researchers how these representations encode linguistic information. one prominent line of work attempts to shed light on this topic through probing (alain and bengio, 2017), also referred to as auxiliary prediction (adi et al., 2017) or diagnostic classification (hupkes et al., 2018). in machine learning parlance, a probe is a supervised classifier that is trained to predict a property of interest from the target model's representations. if the probe manages to predict the property with high accuracy, one may conclude that these representations encode information about the probed property. while widely used, probing is not without its limitations. @xcite for instance, probing a pre-trained model for grammatical gender can only tell us whether information about gender is present in the representations, foot_1 it cannot, however, tell us how or if the model actually uses information about gender in its predictions (ravichander et al., 2021; elazar et al., 2021; ravfogel et al., 2021; lasri et al., 2022). furthermore, supervised probing cannot tell us whether the property under consideration is directly encoded in the representations, or if it can be recovered from the representations alone due to spurious correlations among various linguistic properties. in other words, while we might find correlations between a probed property and representations through supervised probing techniques, we cannot uncover causal relationships between them. in this work, we propose a new strategy for input-level intervention on naturalistic data to obtain what we call naturalistic counterfactuals, which we then use to perform causal probing. through such input-level interventions, we can ascertain whether a particular linguistic property has a causal effect on a model's representations. a number of prior papers have attempted to tease apart causal dependencies using either input-level or representation-level interventions. for instance, work on representational counterfactuals has investigated causal dependencies via interventions on neural representations. while quite versatile, representation-level interventions make it hard-if not impossible-to determine whether we are only intervening on our property of interest. another proposed method, templated counterfactuals, does perform an input-level intervention strategy, which is guaranteed to only affect the probed property. under such an approach, the researcher first creates a number of templated sentences (either manually or automatically), which they then fill with a set of minimal-pair words to generate counterfactual examples. however, template-based interventions are limited by design: they do not reflect the diversity of sentences present in natural language, and, thus, lead to biased estimates of the measured causal effects. naturalistic counterfactuals improve upon templatebased interventions in that they lead to unbiased estimates of the causal effect. in our first set of experiments, we employ naturalistic causal probing to estimate the average treatment effect (ate) of two morpho-syntactic features-namely, number and grammatical gender-on a noun's contextualized representation. we show the estimated ate's stability across corpora. in our second set of experiments, we find that a noun's grammatical gender and its number are encoded by a small number of directions in three pre-trained models' representations: bert, roberta, and gpt-2. @xcite we further use naturalistic counterfactuals to causally investigate gender bias in roberta. we find that roberta is much more likely to predict the adjective hermoso(a) (beautiful) for feminine nouns and racional (rational) for masculine. this suggests that roberta is indeed gender-biased in its adjective predictions. finally, through our naturalistic counterfactuals, we show that correlational probes overestimate the presence of certain linguistic properties. we compare the performance of correlational probes on two versions of our dataset: one unaltered and one augmented with naturalistic counterfactuals. while correlational probes achieve very high (above 90%) performance when predicting gender from sentence-level representations, they only perform close to chance (around 60%) on the augmented data. together, our results demonstrate the importance of a naturalistic causal approach to probing. ## probing there are several types of probing methods that have been proposed for the analysis of nlp models, and there are many possible taxonomies of those methods. for the purposes of this paper, we divide previously proposed probing models into two groups: correlational and causal probes. on one hand, correlational probes attempt to uncover whether a probed property is present in a model's representations. on the other hand, causal probes, roughly speaking, attempt to uncover how a model encodes and makes use of a specific probed property. we compare and contrast correlational and causal probing techniques in this section. ## the causal framework the question of interest in this paper is how contextualized representations are causally affected by a morpho-syntactic feature such as gender or number. to see how our method works, it is easiest to start with an example. let's consider the following pair of spanish sentences: the meaning of these sentences is equivalent up to the gender of the noun programador, whose feminine form is programadora. however, more than just this one word changes from (1) to ( 2 ): the definite article el changes to la and the adjective talentoso changes to talentosa. in the terminology of this paper, we will refer to programador as the focus noun, as it is the noun whose grammatical properties we are going to change. we will refer to the changing of (1) to ( 2 ) as a syntactic intervention on the focus noun. informally, a syntactic intervention may be thought of as taking part in two steps. first, we swap the focus noun (programador) with another noun that is equivalent up to a single grammatical property. in this case, we swap programador with programadora, which differs only in its gender marking. second, we reinflect the sentence so that all necessary words grammatically agree with the new focus noun. the result of a syntactic intervention is a pair of sentences that differ minimally, that is, only with respect to this one grammatical property (figure 1 ). another way of framing the syntactic intervention is as a counterfactual: what would (1) have looked like if programador had been feminine? the rest of this section focuses on formalizing the notion of a syntactic intervention and discussing how to use them in a causal inference framework for probing. a note on inanimate nouns. when estimating the effect of grammatical gender here, we restrict our investigation to animate nouns, for example, programadora/programador (feminine/ masculine programmer). grammatical gender of inanimate nouns is lexicalized, meaning that each noun is assigned a single gender, for example, puente (bridge) is masculine. in other words, there is not a non-zero probability of assigning each lemmata to each gender, which violates a condition called positivity in causal inference literature. thus, we cannot perform an intervention on the grammatical gender of those words, but rather would need to perform an intervention on the lemma itself. we refer to gonen et al. (2019) for an analysis of the effect of gender on inanimate nouns' representations. note that a similar lexicalization can also be observed in a few animate nouns, for example, madre/padre (mother/father). in such cases, to separate the lemma from gender, we assume that these words share a hypothetical lemma, which in our example represents parenthood, and combining that with gender would give us the specific forms (e.g., madre/padre). ## approximating the ate in this section, we show how to estimate equation( 6 ) from a finite corpus of sentences s. ## dataset we use two spanish ud treebanks (nivre et al., 2020) in our experiments: spanish-gsd (mcdonald et al., 2013) and spanish-ancora (taulé et al., 2008). we only analyze gender on animate nouns and use open multilingual word-net (gonzalez-agirre et al., 2012) to mark the animacy. corpus statistics for the datasets can be found in ## insights from ate estimators in the following experiments, we first use the estimators introduced in §4 to approximate the ate of number and grammatical gender on contextualized representations. we look at how stable these ate estimates are across datasets, and whether they change across words with different parts of speech. we then analyze whether the ate (as an expected value) was an accurate description of how representations actually change in individual sentences. finally, we compute the ate of gender on the probability of predicting specific adjectives in a sentence, thereby measuring the causal effect of gender in adjective prediction. ## insights from naturalistic counterfactuals in the following experiments, we rely on a dataset augmented with naturalistic counterfactuals. we first explore the geometry of the encoded paired values computed using equation ( 14 ) to measure causal gender bias in masked adjective prediction. morpho-syntactic features. we then run a more classic correlational probing experiment, highlighting the importance of a causal framework when analyzing representations. ## conclusion we propose a heuristic algorithm for syntactic intervention which, when applied to naturalistic data, allows us to create naturalistic counterfactuals. although similar analyses have been run by prior work, using either templated or representational counterfactuals (elazar et al., 2021; vig et al., 2020; bolukbasi et al., 2016, inter alia), our syntactic intervention approach allows us to run these analyses on naturalistic data. we further discuss how to use these counterfactuals in a causal setting to probe for morpho-syntax. experimentally, we first showed that ate estimates are more robust to dataset differences than either our naïve (correlational) estimator, or template-based approaches. second, we showed that ate can (at least partially) predict how representations will be affected after intervention on gender or number. third, we employ our ate framework to study gender bias, finding a list of adjectives that are biased towards one or other gender. fourth, we find that the variation of gender and number can be captured by a few principal axes in the nouns' representations. and, finally, we highlight the importance of causal analyses when probing: when evaluated on counterfactually augmented data, correlational probe results drop significantly.