Assessing the Capacity of Transformer to Abstract Syntactic Representations: A Contrastive Analysis Based on Long-distance Agreement (2023)

Abstract: Many studies have shown that transformers are able to predict subject-verb agreement, demonstrating their ability to uncover an abstract representation of the sentence in an unsupervised way. Recently, Li et al. (2021) found that transformers were also able to predict the object-past participle agreement in French, the modeling of which in formal grammar is fundamentally different from that of subject-verb agreement and relies on a movement and an anaphora resolution. To better understand transformers’ internal working, we propose to contrast how they handle these two kinds of agreement. Using probing and counterfactual analysis methods, our experiments on French agreements show that (i) the agreement task suffers from several confounders that partially question the conclusions drawn so far and (ii) transformers handle subject-verb and object-past participle agreements in a way that is consistent with their modeling in theoretical linguistics.

## introduction since @xcite , the long-distance agreement task has been a paradigmatic test for assessing the ability of neural network language models (nnlms) to uncover syntactic information from raw texts: a model able to predict the correct verb form (especially when the verb does not agree with the word just before it) has to, to some extent, acquire a representation of the syntactic structure and encode it in its internal representations. in this work, we seek to identify to what extent the nnlm abstracts its representations from surface pattern recognition to structurally motivated representations. to do this, we focus on two kinds of number agreement in french (both morphologically marked): on the other hand, the agreement between chats and adoptés in (2) is an object relative clause agreement. overall, it involves an agreement between a noun in the main clause and a past participle in the relative clause. to predict this kind of agreement, the model must be able to detect a complex set of patterns across different clauses that brings into play an anaphora resolution and a movement, a set of operations whose nature is fundamentally different from phrase structure embedding involved in the subject-verb agreement in (1). while these two kinds of agreement show very similar surface patterns, their modelings in formal grammar result in completely different representations of the sentence structure, and it is not clear whether and how a sequential language model can identify these abstract representations based merely on the words sequence. even though a tremendous number of studies have brought to light the ability of neural networks to capture and abstract the information needed to predict the subject-verb agreement (see section 6 for an overview), it is only very recently that @xcite showed that they were also able to predict the much rarer past participle agreement across clause boundaries like the agreement between chats and adoptés in (2). building on @xcite , the goal of the present work is to contrast how transformers handle these two kinds of agreement: we aim to determine whether they encode the same abstract structure in their internal representations to capture the information required to choose the correct verb form or, on the contrary, if the abstract structure they encode reflects the distinction made in the theoretical modeling of these two agreements. this contrast will shed a new light on our understanding of the internal working of transformers. the contributions of this paper are twofold. first, we assess incremental transformers' syntactic ability to process two different syntaxsensitive dependencies with similar surface forms in french. our results show that transformers perform consistently well on both agreement phenomena, and crucially, that they are able to abstract away from potential confounds such as lexical co-occurences or superficial heuristics. second, we use linguistic probes as well as targeted masking intervention on self-attention to test where transformer-based models encode syntac-tic information in their internal representations and how they use it. we find that, for both constructions, even though the long-distance agreement information is mainly encoded locally across the tokens between the two elements involved in the agreement, transformers are able to leverage distinct linguistically motivated strategies to process these two phenomena. 2 the rest of this paper is organized as follows: in section 2, we define more precisely the two kinds of agreement considered in this work. then, in section 3, we contrast the capacity of transformers to capture subject-verb and object-past participle agreement. following this, in section 4, by using probing and counterfactual analysis, we examine the way transformers encode and use the syntactic information to process these two linguistic phenomena. we then discuss, in section 5, the impact of some potential confounders, in section 6 the related work, and finally we conclude in section 7. ## long-distance agreement overall, this work aims to assess to what extent transformers rely on shallow surface heuristics to predict agreement or to what extent they infer more abstract knowledge. to study this question, we rely on a contrast between two agreement tasks as exemplified in (1), (2), and in figure 1 . both involve a long-distance agreement but they receive substantially different linguistic analyses. ## do transformers process the two agreements in the same way? in the last section, transformer language model has demonstrated a strong ability for capturing syntactic information that generalizes beyond surface heuristics when processing long-range subject-verb and object-past participle agreements. in this section, we investigate whether transformers acquire and use a meaningful, abstract representation of the sentence to predict the two theoretically distinct agreements we considered in this work or, on the contrary, whether they use one unified agreement mechanism to resolve the two agreement tasks. to do this, we use in section 4.1 linguistic probes @xcite to compare the distribution of the number information across the sentence token representations. then, in section 4.2, we compare the way transformers use the encoded syntactic information to process these two different long-range dependencies. mean probing accuracy object-pp subject-verb prefix 58.6% ±0.1 59.5% ±0.2 context 92.3% ±0.2 93.0% ±0.1 suffix 73.6% ±0.2 78.1% ±0.2 ## discussion our experiments clearly show that transformers are capable of predicting quite well the object-past participle and subject-verb agreements in the case of french object relative clauses. moreover, even though several confounders exist, the results consistently indicates that transformers base their predictions mainly on the tokens involved in agreement rules and they apply different strategies despite the very similar superficial forms of these two linguistic phenomena. we converted our original evaluation sets into nonsensical but grammatically well-formed evaluation sets: following @xcite , we have created a nonce dataset for each agreement phenomenon by substituting each content word of the original evaluation sentence with a random word having the same syntactic category. 8 we then evaluated our model's predictions on number agreement tasks in this nonce setting. the results in ## conclusion this work addresses the question of the capacity of transformers to uncover abstract representation of sentences rather than to merely capture surface patterns. to this end, we investigate the mechanisms implemented by this model to predict two kinds of agreement in french: the subject-verb and the object-past participle agreements. even though these two agreements involve very similar word sequences, their linguistic analyses are fundamentally different. the first set of experiments we reported show that transformers are indeed able to predict these two kinds of agreement and, by comparing their predictions with those of simple surface heuristics, we highlight their ability to capture syntactic information. in a second set of experiments, we used probing and counterfactual analysis methods to provide evidence that transformers are actually succeeding for good reasons: our analyses reveal that (i) the incremental transformer bases its predictions on cues that are linguistically motivated and consistent with french grammar, (ii) the abstract structure uncovered by transformers reflects the distinction made in the theoretical modeling of these two agreements. our work is a first step towards a better understanding of the inner representations of nnlms. designing new probes, supported by causal analysis and involving a wider range of languages, could improve our understanding of such models. in particular, our observation about the linguistically motivated distribution of syntactic information in transformers' representations could be extended to other linguistic phenomenon and languages.