Evaluating Grammatical Well-Formedness in Large Language Models: A Comparative Study with Human Judgments (2024)

Abstract: Research in artificial intelligence has witnessed the surge of large language models (LLMs) demonstrating improved performance in various natural language processing tasks. This has sparked significant discussions about the extent to which large language models emulate human linguistic cognition and usage. This study delves into the representation of grammatical well-formedness in LLMs, which is a critical aspect of linguistic knowledge. In three preregistered experiments, we collected grammaticality judgment data for over 2400 English sentences with varying structures from ChatGPT and Vicuna, comparing them with human judgment data. The results reveal substantial alignment in the assessment of grammatical correctness between LLMs and human judgments, albeit with LLMs often showing more conservative judgments for grammatical correctness or incorrectness.

## introduction the rise of llms has been extraordinary, demonstrating proficiency across numerous linguistic tasks such as resolving ambiguities @xcite , addressing queries @xcite , and facilitating multilingual translation @xcite . despite not being initially programmed with a human-like hierarchical syntax structure, these models have managed to identify complex syntactic patterns and generate sophisticated syntactic interpretations @xcite @xcite . however, the critical question remains: do llms truly mirror human linguistic cognition? prominent figures such as @xcite argue that llms and humans process and understand language differently, while others like @xcite suggest that llms might indeed reflect genuine human linguistic processes. recent empirical research has become key to this debate. innovative experiments by @xcite subjected gpt-3 to a battery of psychological tests originally crafted to understand facets of human thought processes, ranging from decisionmaking matrices to reasoning pathways. the outcomes were intriguing, with gpt-3 not just mirroring but at times outperforming human benchmarks in specific scenarios. similarly, @xcite assessed the capacity of llms to understand and respond to false-belief scenarios, which are utilized to gauge human empathy and comprehension. here, the responses from chatgpt echoed the patterns seen in school-going children, though subsequent research from @xcite voiced concerns about the consistency of such responses. further, @xcite subjected chat-gpt to a range of psycholinguistic tests, revealing significant alignment in language use between the model and humans, although differences such as in word length preference were observed (e.g., @xcite ). @xcite assessed chatgpt's ability to compute pragmatic implicatures and found that chatgpt did not demonstrate human-like flexibility in switching between pragmatic and semantic processing. additionally, chat-gpt did not exhibit the effect of communicative context on the rates of computing scalar implicatures, which is a well-established effect for human participants. when examining llm-human similarities, it's crucial to assess the extent to which llms' representations of linguistic knowledge align with those of humans. contemporary linguistic theories often distinguish between the inherent mental systems that enable language comprehension and production and the actual use of language-illustrated by distinctions like "langue vs. parole" from @xcite and "competence vs performance" by @xcite . grammaticality judgement is a central method to assess linguistic representation competence. @xcite highlighted that evidence for linguistic theorizing largely depends on "the judgements of native speakers". while there are other sources of evidence like speech corpus or acquisition sequences @xcite , formal linguists typically favor native speakers' grammaticality intuitions. the prevailing assumption is that our language knowledge comprises abstract rules and principles forming intuitions about sentence well-formedness @xcite @xcite . our study focuses on the representation of grammatical well-formedness in llms. recent research on the grammatical capabilities of language models has primarily focused on binary grammaticality judgments using minimally different sentence pairs. marvin and linzen (2018) evaluated various language models on syntactic phenomena and found that while models handle local dependencies well, they struggle with non-local dependencies. similarly, @xcite introduced the corpus of linguistic acceptability (cola) to test neural network models on binary acceptability judgments, revealing that these models still fall short of human performance on complex syntactic structures. @xcite further examined gpt-3's performance on less frequent grammatical constructions, highlighting its limitations in understanding underlying meanings. while these studies have provided valuable insights into the grammatical abilities of language models, they primarily relied on binary judgment tasks and focused on specific syntactic phenomena. in contrast, our preregistered study ( https://osf.io/75dtk ) adopts a more comprehensive approach by incorporating both binary and graded naturalness judgments, allowing for a finer-grained analysis of language model performance. we collected grammaticality judgment data for over 2400 english sentences from chatgpt and vicuna, comparing them with human judgment data. our findings indicate substantial agreement between chatgpt and humans regarding grammatical intuition, although noticeable differences were also observed. ## experiment 1 in this experiment, we presented chatgpt and vicuna with english sentences of varying grammaticality and asked them to judge the sentences as either natural or unnatural. we compared the llms' judgement data with human judgement data, examining the similarities and differences in their knowledge of sentence grammaticality. ## experiment 2 our second experiment replicated the four-category grammaticality judgement in @xcite with chatgpt and vicuna as the participants. we then compared human performance with that of the llms. ## experiment 3 this experiment aimed to further our understanding of human and llms' knowledge of grammaticality by replicating the previous two experiments using a sliding scale judgement task that was adopted from @xcite . ## general discussion expanding upon @xcite , our study introduced chatgpt and vicuna as llm counterparts for grammaticality judgment, seeking to determine the extent to which llms align with humans in their linguistic knowledge. in general, we observed a strong correlation between human and llm judgments on the naturalness of sentences, which according to @xcite suggests a significant overlap in their grammatical knowledge. however, this overlap does not imply equivalence, as our data revealed consistent statistical differences in ratings between humans and llms across all three experiments. in the binary judgment task, human participants were more conservative than chatgpt. conversely, in the fourpoint and sliding scale tasks, human participants displayed greater variability in their judgments towards both grammatical and less grammatical sentences compared to chatgpt. vicuna's ratings, while generally aligning with those of chatgpt and humans, exhibited less variation across tasks, suggesting a different processing model. we posit that a fundamental distinction between human and llm representations of language lies in the 'distributional knowledge' of sentence grammaticality. humans acquire an understanding of grammaticality through diverse daily language experiences, enriched by a dynamic array of cognitive and contextual cues. in contrast, llms rely predominantly on statistical patterns derived from their training data. this difference in linguistic input is crucial, with human language input being inherently more diverse and dynamic, incorporating a wide array of linguistic registers, dialects, and styles shaped by social interactions and cultural contexts. this exposure enables humans to develop a nuanced and contextually adaptive understanding of language, an aspect of linguistic competence that llms with their data-driven learning processes cannot fully replicate @xcite . moreover, human language processing is inherently multi-modal, incorporating auditory, visual, and contextual cues that enhance comprehension and interpretation. this multi-modal integration includes body language, tone, facial expressions, and environmental context, all of which contribute to a rich, intuitive grasp of language nuances and grammaticality. in contrast, llms such as chat-gpt and vicuna process language purely as text tokens, which are sequences abstracted from their communicative contexts. the tokenization process specific to each model's architecture often strips away nuanced information that humans naturally use to infer meaning, leading to potential discrepancies in understanding subtle linguistic cues or complex semantic structures. additionally, the cognitive processes in humans, including memory, attention, and inference, dynamically interact during language processing, allowing for a rich contextual interpretation of language that adapts in real-time. this level of cognitive engagement in language processing is not mirrored in current llm architectures, which primarily rely on recognizing patterns and statistical generalizations from extensive datasets. these fundamental differences imply that the grammaticality of a sentence is judged against different distributions of possible sentence structures by humans and llms. understanding these variations is crucial for recognizing the limitations and potential biases of llm-generated language assessments. it also underscores the importance of incorporating diverse real-world language data and sophisticated cognitive models into llm training protocols to improve their linguistic adaptability and judgment accuracy. ## conclusion our investigation into the alignment of llms with human grammaticality judgments has revealed both promising correlations and significant nuances in their linguistic capabilities. while llms like chat-gpt and vicuna can effectively mirror human judgments in broad strokes, discrepancies in sensitivity and the conservativeness of their ratings underscore the importance of careful model selection and calibration for specific linguistic tasks.