Implicit causality in GPT -2: a case study (2023)

Abstract: This case study investigates the extent to which a language model (GPT-2) is able to capture native speakers’ intuitions about implicit causality in a sentence completion task. Study 1 reproduces earlier results (showing that the model’s surprisal values correlate with the implicit causality bias of the verb; Davis and van Schijndel 2021), and then examine the effects of gender and verb frequency on model performance. Study 2 examines the reasoning ability of GPT-2: Is the model able to produce more sensible motivations for why the subject VERBed the object if the verbs have stronger causality biases? For this study we took care to avoid human raters being biased by obscenities and disfluencies generated by the model.

## introduction this paper is a case study, highlighting different ways to analyse the linguistic abilities of a language model, with respect to an established linguistic phenomenon, namely implicit causality (ic) bias @xcite . speakers associate either the subject or the object of a verb with the cause of the state or event described by that verb. for example, the verb frighten is a subject-biased verb because native speakers of english tend to see the subject as the cause of the frightening event. thus, given a main clause like in (1a), participants in a sentence completion task would tend to provide a reason referring to the subject, as in (1b). earlier work by @xcite and @xcite investigated the extent to which language models are able to capture native speakers' ic biases. this paper aims to reproduce some of their earlier results, using gpt-2 @xcite as an example. using the same sentence completion task as @xcite , we further investigate how bias, subject gender, and verb frequency influence the behavior of this model ( §3). next, we will more thoroughly assess the quality of the completions generated by gpt-2 ( §4), asking: does the model's performance hold up to further scrutiny? why gpt-2? although it is neither the most recent, nor the best performing open-source language model around (see @xcite for alternatives), gpt-2 is still a very popular choice for many researchers and practitioners. (see https: //huggingface.co/gpt2 for statistics.) this popularity is at least partly due to its size, as the model can be run and fine-tuned on consumer hardware. the model's popularity means that studying its capabilities and limitations may be more impactful (at least in the short term) than studying the capabilities and limitations of larger but less accessible systems. for us, gpt-2 offers the right balance of complexity and efficiency; as shown by @xcite , its outputs are good enough to have a meaningful discussion about the assessment of language model performance, without requiring a large (and expensive) computational infrastructure. contributions next to the value of reproducing earlier work, and providing further details on the ic-related behaviour of gpt-2, the main innovation of this paper is the controlled assessment of output quality. we took great care to separate issues with the fluency and offensiveness of the output from the content of the generated continuations. (see section 6 for the limitations of this study.) this not only makes the task less harmful to our participants, but it also increases their focus on our construct of interest: the reasoning abilities of language models. our code and data from this paper is available at https://github.com/ hienhuynhtdn/gpt2andimplicitcausality/ . ## data we present two studies. study 1 investigates nextword surprisal values, and study 2 looks at continuations that are generated based on a prompt. for both our studies, we provide the model with input sentences of the following form: the subjects and objects are provided by davis and van schijndel (2020), who produced a list of 14 noun pairs that are grammatically male and female (e.g. man, woman or brother, sister). a combination of the nouns and verbs, our set of stimuli consists of 6888 examples (246 verbs × 14 pairs of gender-mismatched nouns × 2 subject genders). ## study 1: ic-bias and pronoun use first, we investigate whether verbs in gpt-2 possess the same subject/object bias as in the human experiment described above. set-up to test the hypothesis, we use the approach from @xcite and checked for each prompt whether the model assigned a lower surprisal to a male or female pronoun (i.e. he or she). 1 if gpt-2 captures the ic bias, then subject-biased verbs with a female subject should lead the model to produce lower surprisal values for she. since each noun pair is used in both orders (either a male or a female noun in subject position), we have a perfectly balanced dataset. results figure 1 shows the results split by subject gender and bias scores from @xcite . gpt-2 generally picks up on the subject or object bias of the verb. the gender produces more subject-based explanations if the verb's ic bias is more subject-biased. there is only one exception: performance for sentences with both subject-biased ## study 2: assessing continuations we now turn to continuations generated by gpt-2 based on the prompts described in section 2. our results above suggest that gpt-2 can generate continuations according to the causality bias pattern, given enough data. based on this result, we now hypothesize that such continuations are better when the ic pattern is clearly present in the data. the stronger the ic bias is, the clearer the pattern of continuations in the training set. of course, this task is much more difficult than simply generating the right pronouns. we now want to see whether the continuations actually make sense in the eyes of human judges. to this end, we collected human ratings for a carefully controlled subset of the continuations generated by gpt-2 (see appendix c for details). irb approval was obtained prior to this study. (ethical considerations in appendix a.) participants we used the prolific participant pool to recruit 75 participants for the sentence rating task. our items were spread across 25 different lists, and each participant was only allowed to provide ratings for one list. we restricted potential participants to native speakers of english, either from the uk or from the usa. after assessing response quality, we recruited five additional participants, to obtain three reliable judgments per item. task and target construct each continuation was assessed by three different participants, who judged whether the continuations were reasonable, given the prompt. we set up our experiment as a rating task, where each participant indicated for a list of 40 items whether they agreed with the statement that the continuation was 'reasonable.' participants could indicate their agreement on a fivepoint likert scale, ranging from 'strongly agree' to 'strongly disagree.' with the addition of some examples in our task description (see appendix g), we targeted our participants' intuitions for what makes a good reason to do something. as we will discuss below, we aimed to avoid any influence of the form of the output as much as possible. 3 prompt selection due to financial limitations, we were not able to obtain ratings for all 6888 continuations. following the recommendations from @xcite , we used a stratified sampling approach. we selected the 5 most frequent noun pairs, and the 10 most frequent verbs for each of the 10 different bias levels (as illustrated in figure 1 ). this selection gives us a sense of the upper bound performance with respect to continuation quality. frequency was again determined using the subtlex-us data @xcite . prompts were constructed in the same way as before (see ex. 2), with each noun pair being presented in both orders. this yields 10 verbs × 10 bias levels × 5 nouns × 2 @xmath0. ## discussion the difference between study 1 and study 2 indicates a qualitative difference between generating pronouns and providing explanations: the latter requires higher-level reasoning which may not be present in language models like gpt-2 @xcite . though the fact that gpt-2 ## limitations the main limitation of our paper is that we focused on only one language model (gpt-2), and only in one language (english). so while our findings provide insights into the capacities of the english gpt-2, they cannot be generalised to other language models or other languages (which is also illustrated by @xcite ). our main contribution is methodological, namely exploring how to assess the linguistic capacities of language models. because our approach treats gpt-2 (mostly) as a black box, our analysis can easily be applied to other models as well. for study 2, a negative effect of verb frequency on the quality of generated continuations was found. as the materials used for the rating study were not gathered to cover the full range of frequencies, this pattern should not be generalized and may reflect a hidden effect. the negative interaction with ic bias suggests the same. the model fit shows an imperfect fit, but without one clear deviation from lin-earity. hence, our findings may well be explained better with more appropriate independent variables. ## conclusion this paper showed two different ways to assess the linguistic capacity of a language model (gpt-2), using implicit causality as a case study. the techniques used above can be applied in a black box setting, without the need to look at the internals of the model. we hope that this paper is useful for others aiming to assess the ability of other language models to capture different linguistic phenomena as well. our findings also showed that automatic assessment methods may not be enough to determine whether semantic phenomena like implicit causality are learned by a language model. human evaluation remains a necessary complement to automatic evaluation (van der @xcite . our paper shows one way to do this without participants being influenced by factors like grammaticality and offensiveness of the output.