Of Models and Men: Probing Neural Networks for Agreement Attraction with Psycholinguistic Data (2024)

Abstract: Interpretability studies have played an important role in the field of NLP. They focus on the problems of how models encode information or, for instance, whether linguistic capabilities allow them to prefer grammatical sentences to ungrammatical. Recently, several studies examined whether the models demonstrate patterns similar to humans and whether they are sensitive to the phenomena of interference like humans’ grammaticality judgements, including the phenomenon of agreement attraction.In this paper, we probe BERT and GPT models on the syntactic phenomenon of agreement attraction in Russian using the psycholinguistic data with syncretism. Working on the language with syncretism between some plural and singular forms allows us to differentiate between the effects of the surface form and of the underlying grammatical feature. Thus we can further investigate models’ sensitivity to this phenomenon and examine if the patterns of their behaviour are similar to human patterns. Moreover, we suggest a new way of comparing models’ and humans’ responses via statistical testing. We show that there are some similarities between models’ and humans’ results, while GPT is somewhat more aligned with human responses than BERT. Finally, preliminary results suggest that surface form syncretism influences attraction, perhaps more so than grammatical form syncretism.

## introduction 2 with the fast development of large language models (llms), interpretability has become @xcite an important issue in natural language processing. interpretability studies aim to explain what llms learn during pre-training, for instance, whether they pick up factual information or develop language skills. one of the promising directions of interpretability research is comparing model responses to human acceptability judgements @xcite . a case in point is the research on the phenomenon of agreement attraction. in this paper, we investigate models' sensitivity to agreement attraction in light of morphological syncretism. we design a probing experiment based on the data from a previous psycholinguistic experiment on humans @xcite . the focus is on russian, a language with rich morphology, which allows us to investigate agreement attraction interacting with different types of syncretism. syncretism is a surface formal identity of grammatically distinct forms like genitive singular polja 'of field (gen.sg)' and nominative plural polja 'fields (nom.pl)' being formally identical. this kind of identity occurs in genitive case and it differs from accusative case syncretism of some other nouns, where it is simply the nominative and accusative case forms that coincide (separately in singular and in plural): lug 'meadow(nom.@xmath0.sg)' and luga 'meadows(nom.@xmath1.pl)'. distinguishing accusative and genitive syncretisms is a unique setup that helps to disentangle the effects of structure (underlying features) from the effects of surface forms. agreement is a grammar rule where grammatical features like number or gender of one linguistic element, the controller, license corresponding features on a syntactically related element, the target. in (1a) singular subject trassa is the controller and requires singular on the verb 'be' and adjective. similarly, in (1b), plural trassy requires plural. this type of phenomena is well acquired by people @xcite , who easily recognize errors in agreement (see e.g. @xcite ). however, the task becomes more complicated if the controller and the target of agreement are separated by some linguistic material. it is yet more complicated, if the surface form of the intervening material coincides with the form of a potential controller due to syncretism. the agreement errors are not recognised so easily in this case. the higher acceptability of incorrect sentences of this type is called agreement attraction and has been under research in psycholinguistics @xcite and in nlp. in nlp similar studies have already been widely conducted on the english language, revealing the inner workings of language models @xcite , as well as the relationship between model errors and human errors @xcite . we tackle a more complex question of whether and how the models parse two kinds of syncretism that could potentially cause agreement attraction for russian. this allows us to finely distinguish the effect of the surface form from the effect of the underlying syntactical structure. we employ the data from a psycholinguistic study of @xcite , measuring russian speakers reading times and cloze test completion. this further allows us to compare model's behaviour to that of humans. our contributions can be stated as following: ## experimental setup we study the attraction phenomenon in llms and compare it to human data available from @xcite . in these experiments, humans' reading time has been measured in relation to the grammatical pattern of the sentence. we follow this approach in our experimental setting, yet we also propose a model-specific interpretability analysis. we reproduce the reading time analysis performed on humans' data, by introducing the readabilitylike metrics for llm. we offer deeper insights into how llms process sentences of every grammatical pattern, by analyzing their attention maps. our statistical analysis methodology mostly follows the one of @xcite , with a few changes made for the sake of results' interpretability. since we test the models on the exact same data on which humans have been tested, we manage to avoid uneven comparisons, yet support the theoretical findings of the original work. ## conclusion we explored how models react to errors in subjectverb agreement, where humans are prone to mis-takes of attraction. these are ungrammatical contexts that look as if agreement happens not with the subject as a whole, but with subject's dependent @xcite . we find that indeed, like humans, models see such sentences as more acceptable than ungrammatical sentences with no attraction, i.e. the ranking in (6) holds for humans and models alike. most importantly, we find in genitive, a pattern similar to what slioussar (2018) finds, where surface syncretism is more predictive of attraction than grammatical number. recall that in our case attraction by surface syncretism obtains for genitive singular, where the attractor is neither nominative nor plural, while grammatical attraction is expected for genitive plural. this is a somewhat puzzling result for humans @xcite and models alike, because other tasks show that both are sensitive to deeper structure. as for overall accuracy, rugpt, a decoder model, was more likely to choose correct sentence continuation, assigning higher probability to the verb form with the correct number. bert, an encoder model, did worse here. attention scores investigation does not present a clear picture, but for rubert comparison between sentences that differ only in attractor are significant. this may be the reason for why its scores are significantly determined by attractor number. we examined a single phenomenon of agreement attraction in subject-verb agreement on a constrained dataset from a psycholinguistic study of @xcite . we confirmed that rubert and rugpt exhibit agreement attraction by grammatical number. an intriguing preliminary finding, resembling slioussar (2018)'s results is that for rugpt agreement attraction seems more sensitive to formal identity than to grammatical number, which could be distinguished in russian genitive forms.