Linguistic Minimal Pairs Elicit Linguistic Similarity in Large Language Models

Authors: xinyu  zhou
delong  chen
samuel  cahyawijaya
xufeng  duan
zhenguang g cai
Venue: EMNLP_2024_Workshop_BlackBoxNLP
Published: 2024-08-20 13:48:07

Abstract: We introduce a novel analysis that leverages linguistic minimal pairs to probe the internal linguistic representations of Large Language Models (LLMs). By measuring the similarity between LLM activation differences across minimal pairs, we quantify the linguistic similarity and gain insight into the linguistic knowledge captured by LLMs. Our large-scale experiments, spanning 100+ LLMs and 150k minimal pairs in three languages, reveal properties of linguistic similarity from four key aspects: consistency across LLMs, relation to theoretical categorizations, dependency to semantic context, and cross-lingual alignment of relevant phenomena.
Our findings suggest that 1)linguistic similarity is significantly influenced by training data exposure, leading to higher cross-LLM agreement in higher-resource languages. 2)Linguistic similarity strongly aligns with fine-grained theoretical linguistic categories but weakly with broader ones. 3)Linguistic similarity shows a weak correlation with semantic similarity, showing its context-dependent nature. 4)LLMs exhibit limited cross-lingual alignment in their understanding of relevant linguistic phenomena. This work demonstrates the potential of minimal pairs as a window into the neural representations of language in LLMs, shedding light on the relationship between LLMs and linguistic theory.

Main paper: 
 introduction: 
 The categorization of linguistic phenomena 1 based on their relevance has been a long-standing endeavor, dating back to Aristotle (Aristotle, 350 BC).* Joint first authors. Xinyu Zhou is now affiliated with Université Paris Cité and Sorbonne Université.1 Linguistic phenomena refer to observable patterns or features in language use. For example, subject-verb agreement is a linguistic phenomenon where verbs must agree with subjects in number and person. An example would be: "The dog barks" (correct) instead of "*The dog bark" (incorrect).This has led to the widely accepted theoretical linguistic consensus of a hierarchical categorization of language structure encompassing syntax, semantics, morphology, etc., which provides a structured way to understand the intricate nature of language, and allows linguists to investigate the interrelationships and commonalities among these linguistic domains (Comorovski, 2013; Li, 2004) .Alongside the theoretical discussions of linguistic phenomena, a growing body of research on quantitative measurement of similarities based on statistical modeling on large-scale corpora has been observed in computational linguistics. Examples include lexical similarity (Holman et al., 2011) , syntactic similarity (Boghrati et al., 2018; Schoot et al., 2016) , semantic similarity (Pennington et al., 2014; Reimers and Gurevych, 2019) , among others. These examples showcase the possibilities of understanding the nature of language through purely statistical methodologies. However, there has been limited research on quantitatively measuring the relationships between different linguistic phenomena. Given that language is a complex system composed of numerous interrelated linguistic phenomena, addressing this gap could lead to a more comprehensive understanding of language structure and its underlying mechanisms.In this work, we aim to uncover and analyze the internal linguistic knowledge of Large Language Models (LLMs) when presented with a wide range of linguistic phenomena. LLMs are large-scale unsupervised language learners without any prior linguistic knowledge, and have demonstrated humanlevel language capability, as evidenced by their leading performance on language understanding benchmarks and impressive language generation fluency (Zhao et al., 2023; Bang et al., 2023) . More specifically, we are interested in how LLMs represent different linguistic phenomena, and whether linguistically similar phenomena are represented similarly in LLMs.Figure 1 : The process of measuring linguistic similarity in an LLM. We extract LLM activations for sentences in linguistic minimal pairs and compute their differences. Since the sentences differ solely in a specific linguistic phenomenon, the resulting difference only contains information about that phenomenon. We then measure the similarity between these activation differences, which we refer to as linguistic similarity.To elicit such representations, we examine the activations in LLMs in response to linguistic minimal pairs (Warstadt et al., 2020) . As shown in Fig. 1 , these pairs consist of sentences that differ only in a word/phrase, with one being grammatical and the other ungrammatical. Since minimal pairs differ only in one particular linguistic phenomenon, information about other aspects (such as topic and semantic meaning) will be canceled out through subtraction. We interpret the remaining differences as the LLMs' internal representation of a specific linguistic phenomenon. By calculating the similarity between multiple such representations, we derive a measure of linguistic similarity between linguistic minimal pairs.We then conduct an extensive analysis of linguistic similarity in LLMs. Our experiment encompasses 100+ LLMs of varying scales and pretraining corpora, utilizing 150,000 linguistic minimal pairs across 3 different languages. We report our observations correspond to the following key questions:1) How consistent is linguistic similarity across different LLMs? LLMs have the highest alignment of linguistic similarity in English, which is the most widely used language for LLM pertaining, while the alignments are comparatively weaker in Chinese and Russian. We further visualized the relationships among these LLMs with UMAP (McInnes et al., 2018) . On Chinese samples, we observed a distinct clustering pattern: bilingual and multilingual LLMs formed one cluster, while English-only models formed another. The above results suggest that the language distribution in the training data influences the linguistic similarity in LLMs.2) Does linguistic similarity align with theo-retical linguistic categorizations? We compared linguistic similarity across three levels of theoretical linguistic categorizations. Our analysis revealed that fine-grained classifications exhibit significantly higher intra-class similarities compared to interclass similarities. However, this disparity diminishes considerably at higher categorization levels. Meanwhile, we can also observe some highly correlated phenomena pairs that are not classified to the same theoretical categorization.3) To what extent does linguistic similarity correlate with semantic similarity? We showed a weak correlation between semantic similarity and linguistic similarity, despite many existing samples with low linguistic similarity and high semantic similarity, and conversely, high in linguistic and low in semantic. The weak correlation indicates that linguistic similarity in LLMs has a contextdependent nature.4) Whether relevant phenomena in different languages enjoy higher linguistic similarities? We compare the linguistic similarity of the shared three linguistic phenomena in English and Chinese. Our UMAP visualization revealed that while English phenomena are clustered within a shared region, they are "attracted" by their relevant phenomena in Chinese.We hope this paper sparks new exploration into LLMs' internal linguistic representations, uncovering deeper insights into their inner workings and potentially informing linguistic theory.To facilitate future research, the activation differences of the 100+ LLMs, pre-computed samplelevel linguistic similarities, and all the codes are made publicly available at https://github.com/ ChenDelong1999/Linguistic-Similarity. 
 
 Appendix:  


Decision: Linguistic analysis of a language model - a topic that is a good fit for the workshop.