What Does Parameter-free Probing Really Uncover? (2024)

Abstract: Supervised approaches to probing large language models (LLMs) have been criticized of using pre-defined theory-laden target labels. As an alternative, parameter-free probing constructs structural representations bottom-up via information derived from the LLM alone. This has been suggested to capture a genuine “LLM-internal grammar”. However, its relation to familiar linguistic formalisms remains unclear. I extend prior work on a parameter-free probing technique called perturbed masking applied to BERT, by comparing its results to the Universal Dependencies (UD) formalism for English. The results highlight several major discrepancies between BERT and UD, which lack correlates in linguistic theory. This raises the question of whether human grammar is the correct analogy to interpret BERT in the first place.

## introduction probing large language models (llms) consists in mapping their internal states to linguistic classes or relations @xcite . most methods use supervised learning for training a probe to predict pre-determined labels @xcite @xcite @xcite . however, critics have deemed this insufficient for determining whether llms actually represent linguistic structures @xcite . for representation proper, the labels should not only be predictable from the llm; they should somehow capture its internal architecture on a high level of abstraction. a possible way forward is to use parameter-free probing, which shuns separate probing classifiers by extracting structural information directly from the llm @xcite @xcite . as a bottom-up approach, this has been interpreted as uncovering the grammar intrinsic to the llm without relying on a priori presumptions derived from linguistic theory. in this paper, i focus on a parameter-free probe called perturbed masking, originally presented and applied to bert by @xcite . while it has received criticism for underwhelming results compared to gold-standard parses @xcite , this overlooks its main goal of uncovering bert's inherent syntax -which may well deviate from linguistic theory @xcite . such deviations do not call for discarding it; instead, they provide insight into how bert's architecture can differ from common linguistic assumptions. i compare dependency graphs derived from bert to the universal dependencies (ud) annotation for english, and uncover major discrepancies related to verbal argument structure, noun phrase structure, modifiers, and prepositions. in particular, bert treats the root (in ud's annotation) as a head far more often than ud. this effect of being "attracted by the root" is especially strong in recursive embeddings, but also extends beyond these. moreover, bert's behavior tends to resist linguistic explanation. for example, despite major disagreements within linguistic theory, argument structure is ubiquitously treated as clause-bound: no feasible analysis assimilates embedded clause arguments to main clause arguments. yet, the bert-parse regularly does exactly this. indeed, the only cases where bert's deviations from ud have a salient linguistic interpretation concern prepositions and some possessive constructions, where dependent-head relations are flipped. the results thus point to the same direction as critiques of supervised probing: the assumption that bert represents grammar in line with familiar linguistic formalisms lacks proper support. when this is not built directly into the experiment design (via pre-determined target labels), probing reveals fundamental disparities between bert and commonly accepted syntactic principles. we are thus prompted to question whether human grammar is an appropriate analogy for bert after all. ## methodology i describe the parameter-free probing technique investigated (section 2.1), the dataset (section 2.2), and the experiment pipeline (section 2.3). ## discussion the results are not easily explained by some trivial non-linguistic property. locality does not account for bert's deviations from ud, since the average head-dependent distance is actually higher in bert-parses (section 3.1). another initial possibility could be that bert mimics naive right-chain performance. 8 however, most examples in sections 3.2-3.5 involve bert assigning the head leftward (i.e. the dependent rightward). sometimes this even goes directly against right-chain-like annotation in ud, as in example (11) (section 3.4). it is also worth raising the controversial status of the ud format itself (c.f. @xcite . the central issue here concerns function words, which ud treats as dependents of content words -going against alternative formats such as surface-syntactic universal dependencies (sud) @xcite where these relations are reversed. the corresponding distinction appears in our results as well, with respect to prepositions and nps (section 3.5). bert's performance might thus accord better alternative formats to ud, such as sud. that said, most discrepancies discussed in section 3 are not specific only to ud. all mainstream syntactic frameworks distinguish between arguments/modifiers of main and embedded clauses (sections 3.2, 3.4), and treat possessors, determiners, numerals, or adjectives as modifying nouns rather than verbs (sections 3.3, 3.4). with the possible exception of (root-modifying) pps (section 3.5), the shifts are not made linguistically coherent by minor changes to the syntactic formalism. ## conclusions and future work this study uncovered several discrepancies between bert and ud. while some were syntactically interpretable, bert's prevailing tendency to treat the root as a head across phrase boundaries lacks a clear linguistic analogy. this puts to question the idea that bert should be interpreted in line with traditional grammatical formalisms. instead, it highlights the need to explain llms in their own terms -avoiding reliance on a priori linguistic assumptions not motivated by llms themselves.