Large Language Models as Neurolinguistic Subjects: Discrepancy between Performance and Competence (2025)

Abstract: This study investigates the linguistic understanding of Large Language Models (LLMs) regarding signifier (form) and signified (meaning) by distinguishing two LLM assessment paradigms: psycholinguistic and neurolinguistic. Traditional psycholinguistic evaluations often reflect statistical rules that may not accurately represent LLMs’ true linguistic competence. We introduce a neurolinguistic approach, utilizing a novel method that combines minimal pair and diagnostic probing to analyze activation patterns across model layers. This method allows for a detailed examination of how LLMs represent form and meaning, and whether these representations are consistent across languages. We found: (1) Psycholinguistic and neurolinguistic methods reveal that language performance and competence are distinct; (2) Direct probability measurement may not accurately assess linguistic competence; (3) Instruction tuning won’t change much competence but improve performance; (4) LLMs exhibit higher competence and performance in form compared to meaning. Additionally, we introduce new conceptual minimal pair datasets for Chinese (COMPS-ZH) and German (COMPS-DE), complementing existing English datasets.

## introduction large language models (llms) have demonstrated remarkable reasoning, linguistic, arithmetic, and other cognitive abilities. the advent of llms has reignited cross-disciplinary discussions about what sorts of behavior are "intelligence", even if the intelligence exhibited by llms may differ from human intelligence @xcite . llms have drawn the attention of researchers from various fields, including linguistics, cognitive science, computer science, and neuroscience, who investigate how llms develop and exhibit these capabilities. there is currently a heated debate about whether llms understand human language or whether their 1 code and data available here. performance is simply the product of complex statistical relationships @xcite . a central aspect of this debate concerns the nature of llms' linguistic representations. using the semiotic framework of language proposed by de @xcite , which distinguishes between the signifier (form) and the signified (meaning), we can inquire into the extent to which llms comprehend the form and meaning, and how form and meaning intertwist with each other. is llms' understanding of language meaning merely a statistical outcome based on their grasp of language form? when different languages express a shared concept with distinct forms, do llms create similar representations for these variations? how can we better understand the representations of form and meaning in these systems that support the observed patterns of performance? the underlying processes remain unclear due to the opaque nature of neural networks. therefore, we need appropriate methods to assess their true linguistic understanding. drawing inspirations from the cognitive study on human language processing, we propose that the assessment of llms can be divided into two primary paradigms: psycholinguistic and neurolinguistic. as illustrated in figure 2 , the psycholin-figure 2 : psycholinguistic vs. neurolinguistic paradigm. both direct probability measurement and metalinguistic prompting can be considered as psycholinguistic methods, while minimal pair probing @xcite and other diagnostic probing are neurolinguistic. when treating llms as psycholinguistic subjects, their responses may leverage their grasp of form, relying on statistical correlations, to create an illusion of understanding meaning. this enables llms to produce structurally coherent but not necessarily semantically accurate responses, as their "understanding" is shaped by patterns rather than true conceptual processing @xcite @xcite . consequently, psycholinguistic evaluations tend to reflect performance rather than competence, as they assess external outputs that may not fully capture the underlying linguistic knowledge encoded within the model. this mismatch suggests that psycholinguistic evaluation results might not accurately represent the true linguistic competence of llms. in contrast, examining llms as neurolinguistic subjects focuses on internal representations, providing a more direct assessment of competence by moving beyond surface-level biases @xcite . to achieve this, we adapted the decoding probing method by @xcite , referred to as "minimal pair probing", to analyze how llms encode form and meaning across layers. this approach allows for a finer distinction between performance and competence, revealing insights that psycholinguistic methods might overlook. in order to address questions about whether llms maintain consistent underlying representa-tions of the same concept when the form changes across multiple languages, we also create a multilingual minimal pair dataset (comps-zh for chinese and comps-de for german). by evaluating llms in both psycholinguistic and neurolinguistic paradigms, we found: 1) psycholinguistic and neurolinguistic results reveal very different patterns, suggesting both paradigms are necessary for a comprehensive understanding of llms. 2) though more intrinsic than metalinguistic prompting, direct probability measurement may still not accurately assess linguistic competence, as it remains influenced by statistical patterns. 3) llms acquire competence in linguistic form more easily, earlier, and with greater accuracy than in meaning. 4) as linguistic form varies across languages, llms' understanding of the same concept shifts accordingly, with meaning competence linearly correlated to form. this suggests that signifier and signified in llms may not be independent, and maintaining conceptual representations likely depends on statistical correlations with form. ## minimal pair @xmath0 while prior neurolinguistic approaches have explored internal representations, they often employed coarse-grained datasets and primarily focused on decoding linguistic labels from embeddings, providing a general perspective on the linguistic features encoded in lms. in contrast, the minimal pair probing method presented by @xcite integrates minimal pair design with diagnostic probing. this combination leverages the granularity of minimal pair design and the layer-wise insights of diagnostic probing, thereby enabling a more detailed analysis of internal patterns for form and meaning. we adopt minimal pair decoding as the neurolinguistic paradigm in our work. specifically, given an llm f : x 0,1,...,@xmath1, we can extract the hidden state representations f l (s) of the l-th layer of stimuli s. a label z, we have internal representation f l (s i + ) and f l (s i -) for each sentence. a minimal probing classifier g : f l (s) → ẑ is trained and evaluated on d m , with grammatical/conceptual performance measure perf(f, d o , g, d m ). note that our focus is on evaluating the linguistic competence of the llm f itself, i.e., perf(f, d o ), rather than the capacity of the probing classifier g. as suggested by @xcite , even untrained or random representations can yield surprisingly high probing accuracy, raising concerns that the classifier may exploit dataset artifacts rather than meaningful representations. to control for the potential bias introduced by g, we construct a random embedding baseline. specifically, for each sentence in the dataset, we assign a fixed random vector r, sampled from a gaussian distribution with the same mean and standard deviation as the real model embeddings f l (s). importantly, each sentence is consistently assigned the same random vector across occurrences, preserving instance-level identity that the probing classifier might exploit. this allows us to assess the extent to which task performance can be driven by superficial sentence-level cues rather than meaningful representations. we then compute perf(g, d m ) by training g on these random embeddings, which reflects the inherent predictability or "shortcut" potential of the probing task. therefore, our performance score incorporates a correction factor based on this random baseline, defined as: this formula applies a correction term, penalizing cases where the probing classifier performs well even on random embeddings. when perf(g, d m ) = 0.5, the correction factor is 1; if the performance is higher, the factor shrinks toward 0, discouraging overfitting or trivial tasks; if it drops below 0.5, the factor exceeds 1, slightly amplifying the model's score. this ensures that only meaningful representations in f contribute to the final evaluation. ## discussion language performance vs. competence: probing reveals deeper linguistic understanding than direct probability. our results demonstrate that neurolinguistic probing uncovers linguistic competencies in llms that are not captured by psycholinguistic methods. while meta performs the worst and direct performs better, neuro consistently outperforms both, revealing a systematic underestimation of competence when relying on output-based evaluations. hu and levy (2023) argued that direct probability measurement, being more intrinsic than metalinguistic prompting, better reflects competence. however, our findings show that even direct falls short of revealing the full extent of llms' linguistic capabilities. direct relies on the final output layer, which is highly optimized for next-word prediction and thus entangled with task-specific objectives. prior studies @xcite have shown that syntactic and general linguistic information is often better represented in intermediate layers than in the final layer. waldis et al. (2024) also emphasized that output correctness is an insufficient indicator of linguistic understanding, advocating for probing internal representations. our t-sne visualizations corroborate this: clear linguistic clusters emerge in intermediate layers but dissolve in the final layer, reinforcing the view that the last layer is not optimal for assessing competence. these findings suggest that direct, while more grounded than prompting, is still a limited proxy for internal knowledge. in contrast, neurolinguistic probing inspects internal activation patterns across layers and tasks, uncovering the underlying representational structure of form and meaning, and further validates the discrepancy between performance and competence. on the other hand, while meta results underperform, this does not necessarily indicate that the llms lack the underlying linguistic competence. instead, it may reflect limitations in information access, as suboptimal prompts can prevent models from exhibiting their full capabilities. specifically, prompting failures do not always equate to a lack of encoded knowledge. this aligns with prior work @xcite emphasizing the need to distinguish performance conditions from underlying ability. thus, we argue that probing, particularly when applied layer-wise, provides a more accurate and comprehensive assessment of linguistic competence than direct probability alone. form and meaning: observations from saussure's semiotics our results reveal that llms consistently learn linguistic form before they grasp meaning. this may suggest a developmental trajectory where statistical patterns in syntax and grammar are more readily captured by the model than conceptual understanding. second, the models' formal competence is generally superior to their semantic competence. this is evident in their ability to decode grammaticality structures accurately but with less reliable conceptual accuracy. we further observe a linear correlation between form and meaning competence, particularly when linguistic forms vary across languages while meaning remains constant. this suggests that llms' understanding of meaning might rely heavily on form, with conceptual representation anchored to formal structures rather than independent meaning comprehension. these results offer a semiotic and neurolinguistic explanation for llms' longstanding issue of generating "confidently incorrect" responses, i.e., hallucinations @xcite . ## conclusion this study adopts both psycho-and neuro-linguistic approaches to evaluating llms, revealing a distinction between linguistic performance and competence. our results highlight the limitations of llms' semantic understanding and the need for future research to move beyond statistical correlations toward more grounded language representations. by introducing a cognitive neuroscience perspective, along with semiotics, we hope will inspire further research to deepen our understanding of the language capabilities of llms.