Probing Syntax in Large Language Models: Successes and Remaining Challenges
Authors: Pablo J. Diego Simon
Emmanuel Chemla
Jean-Remi King
Yair Lakretz
Venue: COLM 2025 (2025)
Keywords: Syntax
LLMs
Probing
Evaluation
TL;DR: This work evaluates syntactic representations in LLMs using structural probes. We assess these probes across three benchmarks, revealing that their accuracy is compromised by linear distance and syntactic depth, yet remains invariant to surprisal.

Abstract: The syntactic structures of sentences can be readily read-out from the activations of large language models (LLMs). However, the ``structural probes'' that have been developed to reveal this phenomenon are typically evaluated on an indiscriminate set of sentences. Consequently, it remains unclear whether structural and/or statistical factors systematically affect these syntactic representations. To address this issue, we conduct an in-depth analysis of structural probes on three controlled benchmarks. Our results are fourfold. First, structural probes are biased by a superficial property: the closer two words are in a sentence, the more likely structural probes will consider them as syntactically linked. Second, structural probes are challenged by linguistic properties: they poorly represent deep syntactic structures, and get interfered by interacting nouns or ungrammatical verb forms. Third, structural probes do not appear to be affected by the LLMs' predictability of individual words. Fourth, despite these challenges, structural probes still reveal syntactic links far more accurately than the linear baseline or the LLMs' raw activation spaces. Taken together, this work sheds light on both the challenges and the successes of current structural probes and provides a benchmark made of controlled stimuli to better evaluate their performance.

Published as a conference paper at COLM 2025
Probing Syntax in Large Language Models: Successes and Remaining Challenges
Pablo J. Diego-Simón"
Emmanuel Chemla 1.2
Jean-Rémi King"
Yair Lakretz!
!LSCP, ENS, PSL,, EHESS, CNRS, Paris, France Earth Species Project, Berkeley, California, USA
3 Meta AI, Paris, France
Abstract
The syntactic structures of sentences can be readily read-out from the activations of large language models (LLMs). However, the "structural probes" that have been de- veloped to reveal this phenomenon are typically evaluated on an indiscriminate set of sentences. Consequently, it remains unclear whether structural and/or statistical factors systematically affect these syntactic representations. To address this issue, we conduct an in-depth analysis of structural probes on three controlled bench- marks. Our results are fourfold. First, structural probes are biased by a superficial property: the closer two words are in a sentence, the more likely structural probes will consider them syntactically linked. Second, structural probes are challenged by linguistic properties: they poorly represent deep syntactic structures, and get interfered by interacting nouns ungrammatical verb forms. Third, structural probes do not appear to be affected by the I.LMs' predictability individual words. Fourth, despite these challenges, structural probes still reveal syntactic links far more accurately than the linear baseline or the LLMs' raw activation spaces. Taken together, this work sheds light on both the challenges and the successes of current structural probes and provides a benchmark made of controlled stimuli to better evaluate their performance.
Introduction
The autonomy of syntax. Understanding how word sequences combined to form the meaning of a sentence is a central challenge to linguistics (Chomsky, 1957; 1995). Behavioral (Rayner, 1978; Frazier & Rayner, 1987; Gleitman, 1990) and neuroimaging research (Pallier et al., 201 1; Dehaene & Cohen, 2011; Devauchelle et al., 2009; Caucheteux et al., 2021; Friederici, 2018; Santi & Grodzinsky, 2007) show that this phenomenon requires the brain to build latent structures that link the words of sentences. Critically, such "syntactic" system is thought to be autonomous: the infamous sentence "colorless green ideas sleep furiously" shows that words can be syntactically linked even each word is highly unpredictable, and thus lead to a nonsensical sentence Croft (1995); Chomsky (1957). Yet, the neural and computational implementations of such autonomous syntactic structures remain largely unresolved (Grodzinsky & Friederici, 2006; Kaan & Swaab, 2002).
Syntactic representations LLMs. Large Language Models (LLMs) now offer an unprecedented opportunity to revisit this historical question. Not only these models generate coherent sentences, but their internal activations explicitly represent the syntactic structures long theorized by linguists (Belinkov al., 2017: Tenney et al., 2019; Peters et al., 2018; Hale & Stanojevié, 2024; Hewitt & Manning, 2019; Diego-Simon et al., 2024). Following earlier work on linear probing (Alain & Bengio, 2017: Conneau et al., 2018), the 'Structural Probe' (Hewitt & Manning, 2019) recently showed that LLMs spontancously learn to build a subspace of activations, where the distance between two contextualized word embedding corresponds to the distance that separates syntactic tree. Diego-Sime Simoon et al. (2024) further showed that the angle between these two words can represent the type of syntactic relations (e.g. 'subject of", 'object of").
words in the
*Correspondance to: pablo. berbo..ieeee wiw.ooeeeese.... wwwooooosss.....m.

---

Published as a conference paper at COLM 2025
Linear distance
Probe predictions pass & fail
sings
sings
cat
cat
Depth
The
by
The
by
dog
dog
the
the
Low
Surprisal
Fligh
Figure I: Dependencies form a tree structure linking words in sentences, which the Structural Probe aims to predict. Each dependency connects two words that are separated by certain linear distance. Since the dependency tree is rooted, one word in the dependency serves as the head and the other as the child, introducing the notion syntactic depth. Beyond these structural features, the LLM assigns each word surprisal value based on its downstream prediction. We investigate how linear distance, depth, and surprisal influence probe predictions.
Challenge. However, the performance of such Syntactic Probes has mostly been assessed with aggregated scores over large naturalistic corpora (Guarasci et al., 2021). Consequently, it is unclear whether the syntactic representations revealed by these LLM probes identify abstract representations of syntax (Croft, 1995; Chomsky, 1957), or whether they rather rely other, heuristic or statistical factors.
Approach. To address this question, we evaluate the performance two state-of-the-oe syntactic probes (Hewitt & Manning, 2019; Diego-Simon et al., 2024) under variety of linguistic conditions. To this end, we leverage both naturalistic datasets and controlled stimuli designed manipulate specific linguistic (linear distance, depth, number interference) and statistical (word predictability) features. With these tests, we aim to (I) characterize the strengths and limitations of structural probes (2) compare them with human behavior, and ultimately (3) provide benchmark to better evaluate structural probes.
2
Related work
Understanding syntax with linear structural probes has been an extensive area of research initiated with the seminal work of Hewitt & Manning (2019). These probes have been extended using spectral Müiler-Ebeerteenn et al., 2022), polar (Diego-Sime) al., 2024), and orthogonal (Limisiewiez Mareček, 2021) transformations providing insights about the underlying vectorial spaces hosting syntactic processing (Reif et al., 2019).
This suite of probes rests on the premise that linguistic structure is linearly readable from LLM representations. Recent work suggests that linear representations also encode belief states, both across pre-training (Shai et al., 2024) and during in-context learning (Park et al., 2025).
In parallel, efforts have been made to develop non-linear probes, which relax the linear constraint, offering more flexible view of syntactic encoding (Eisape et al., 2022; White et al., 2021; Chen et al., 2021).
Probing methods, however, are subject to limitations and potential artifacts. Some have shown evidence that probe performance can be misleading, not generalizing to Jabberwocky sentences (Maudslay & Cotterell, 2021) or having marginal effects on information-theoretii scores (Pi- mentel et al., 2020). Revealing that, probes may exploit shallow heuristics, making controlled and linguistically motivated evaluations essential for uncovering deeper insights.
2

---

Published as a conference paper at COLM 2025
3 Methods
3.1 Data
Naturalistic sentences The Structural (Hewitt & Manning, 2019) and Polar (Diego-Simon et al., 2024) Probes were trained on the Universal Dependencies English Web Treebank (UD-EWT) corpus (Silveira et al., 2014). UD-EWT contains 16,622 sentences. The corpus consists of 254,820 words and 16,622 sentences, extracted from online journals, blogs and informative websites. Every sentence in the corpus has been manually annotated following the Universal Dependencies (UD) formalism (Nivre et al., 2017).
Sentences that include email addresses web URLs have been excluded from the dataset, they considered noisy and irrelevant for syntactic analysis. Also, sentences for which UD tokens and LLM tokens cannot be aligned are also excluded. We adhere to the default split provided by UD-EWT, resulting in 1,,82 sentences for training, 1,851 for validation, and 1,869 for testing.
Controlled sentences structures:
We generated controlled datasets for each of the following syntactic
Prepositional Phrase (PP) Subbi]
Center Embedding (CE)
[subi]
The mouse by the cat moves
The mouse that the cat chases moves
Right Branching (RB)
subj The mouse believes that the cat moves
In these syntactic structures, the attractor interferes with the subject of the sentence. Furthermore, both PP and CE structures introduce long-range subject-verb dependencies, which the subject is separated from the main verb. Each dataset contains sentences with varying degrees of syntactic nesting, ranging from sentence and introduces an attractor noun. For the PP and the CE case, adding more nestings implies modifying the linear distance between the head and the dependent in the subject-verb dependency. For the 2 nesting case each structure becomes:
levels. Adding a syntactic nesting modifies the syntactic depth of the
The mouse by the cat beside the fox moves
The mouse that the cat that the fox protects chases moves
The cat believes that the fox expects that the mouse moves
The controlled dataset is generated using a generative grammar tailored to each target syntactic structure: PP, CE, and RB. The number of nestings for each sentence is uniformly sampled from (1,2,3), introducing varying levels of syntactic complexity. Three nestings are already unusual in natural language and difficult for humans to understand. shared vocabulary is used across all structures, consisting of five lexical categories: verb, noun, transitive verb, preposition, and determiner. Lexical items are sampled from these categories for each syntactic structure, with replacement allowed only for the determiner category. To maintain dataset integrity, ensure that all generated sentences are unique.
Finally, dissociate the effects of additional attractors from those of increased linear distance berween the subject and the verb, we augment the dataset simple sentences containing fillers. In these constructions. the fillers are adverbs that modify the main verb. Fillers extend the linear distance without introducing attractors.
*
Simple sentence [2 fillers]: The mouse quickly and silently moves.
"https:: "https://universas versal o daperddecci.e.ooggteeeeee//////

---

Published as a conference paper at COLM 2025
Overall, the controlled dataset consists of 80,000 sentences. To extend the analysis and generate ungrammatical sentences, the same sentences are used, but the main verb form altered to mismatch the subject-verb agreement.
3.2 Probes
Structural Probe Let W = (I0,, W2,..., wy) be sentence consisting of words belonging to dataset D. Let Tw = (Vw, Ew) ) be its corresponding syntactic dependency tree, where Vw the set of words, and Ew is the set of edges connecting Vw in an acyelic manner. The adjacency matrix E {0, 13 txt of Tw is defined as:
if [w Ew,
A.f
(0, otherwise.
For any words W,, w0 W, the syntactic distance dz wj) is defined as the number of edges in the unique path connecting wi and wi in Tw. Formally, it can be written as:
dr(w;; wi) arg min (I((A)))* o1).
(2)
where [[- represents the indicator function.
Thus we can construct the pairwise syntactic distance M IN between all words in sentence.
M. d (w;, 20,)
Let h/ E Rd denote the contextual embedding of the word wi obtained from layer of an LLM with hidden dimensionality d. In cases where word is tokenized several, the word representation h is computed as the average of its subtoken embeddings to ensure a one-to-one correspondence between words and embeddings.
We follow Hewitt & Manning (2019) and introduce a Structural Probe, defined by a linear transfor- mation B5 E mxd, where m < represents the output dimensionality of the Structural Probe.
In the original embedding space H. the syntactic relations between two words w;, wi W in sentence W, are interpreted as the difference between their contextualized word embeddings.
8' (w,, wi) = h! h)
(4)
The Structural Probe maps word representations from H a new space S.
s' (w, w/) Bsd (w;, 056((w;,00) w);
(5)
The pairwise squared euclidean distance matrix M E Rtxt between probed word representations for given sentence W can be computed follows:
M, = ||s (w., 11
(6)
where || represents 1.2 norm.
The goal of the Structural Probe is to find subspace S in which Mij approximates Mj. For that, the Structural Probe will be trained with the following objective.
Ls (Bs)
M,j
My
D
W
WED
=1
(7)
B = arg min Ls- Bg
4

---

Published as a conference paper at COLM 2025
Polar Probe Complete dependency trees directed and labeled acyclic graphs rather than undirected and unlabeled. Thus, each edge (dependency) e E Ew in the graph (dependency tree) is associated with two functions: U and C, which determine its directionality and label (dependency type) respectively.
The Polar Probe (Diego-Simi) Sigo--Simoneo et al., 2024), extends the Structural Probe and predicts a labeled and directed dependency tree, approximating U and C for each predicted syntactic edge. We extend the study to the Polar Probe since it augments Hewitt & Manning (2019)'s framework with additional syntactic information while preserving a linearly readable, distance-bart syntactic code. For more details about its implementation and training refer to Appendix A.
3.3 Training
The Structural and Polar probes were trained on activations from layer 16 Mistral-7B- (Jiang et al., 2023), Llama-22 2b- (Touvron et al., 2023), and BERT-large (Devlin et al., 2019), as this layer has been shown to best encode dependency structures' (Diego-Simon et al., 2024).
Training conducted on the filtered UD-EWT corpus (Silveira al., 2014) using Stochastic Gradient Descent (SGD) with the Adam optimizer (Kingma & Ba, 2017). Probes were trained for 30 epochs with batch size of 200 sentences and learning rate of 0.005. For the Polar Probe, the hyperparameter was set to 10.0 as specified in (Diego-Simon et al., 2024).
Evaluation Following Hewitt & Manning (2019), for each sentence W in the or controlled dataset, the trained probe B& or B to compute its probed pairwise distance matrix M. Then, we apply Kruskal's algorithm Kruskal (1956) using networkx (Hagberg et al., 2008) to obtain the Minimum Spanning Tree (MST), Ew Ignoring edge direction and labels, the probe performance on a given gold edge E Ew is measured by whether e appears in Ew:
Accuracy(e) 1, i E Σw, 0, otherwise.
(8)
3.4 Baselines
To gauge the effectiveness of the probes, compare them against strong baseline, modifying the definition of M E R For more baselines, refer to Appendix A.
Activation Space prediction
this baseline, we use the raw LLM activations directly, without training or applying any additional linear transformation.
Mij
|| (wi wo
3.5 Surprisal calculation
Surprisal, in causal language models, quantifies the information content of word w, via its condi- tional probability given its previous context (Shannon, 1948). We compute word probabilities p(w;) (Pimentel & Meister, 2024), from which surprisal 1 derived.
I(w;) log(p(w:))
3.6 Classifiers
Ridge Regression and Random Forest models trained predict the probe's 's output for ground truth positive instances (e E Ew) using linguistic features such as head depth, head surprisal, child
Mi straa Eaera1-7B-aa 1-8-v00 1 and 1.1 ama - 2 7--hf f are transformer-based L.MMs with 32 layers and 7 billion parameters. HERT-1a rge is masked language model with 24 layers and 340 million parameters.
5

---

Published as a conference paper at COLM 2025
Polar
Head Child
0.8-
Structural
0.8
0.8-
Accuracy
Activation Space
Necuracy
Accaracy
0.6
0.4
0.4-
0.4
0.2-
0.2-
0.2-
0.0
2
10
0.0
4
6
10
2
10
Linear distance
Depth
Surprisal (quantile)
The mouse quickly and sllently moves
The mouse that the cat chases moves
The mouse by the cat moves
1.0
1.0
1.0
Accuraty
0.8-
0.0
Accuracy
Accurap
0.4
0.4
0.4
0.2
0.0
0.0
0.0
10
Linear
Depth
Surprisal (quantile)
Figure 2: The performance of trained linear probes is most affected by the linear distance and depth of syntactic dependencies in both naturalistic (Top) and controlled (Bottom) stimuli. (Top): Accuracy on syntactic dependencies in UD-EWT. (Top Left) By the linear distance between head and child. (Top Middle) By the depth of the head. (Top Right) By the surprisal quantile of the head and the child. (Bottom): Accuracy on the subject-verb dependency sentences. (Bottom Left): Simple sentences with a varying number of fillers. (Bottom Middle): In CE sentences with varying number of nestings, modifying the syntactic depth of the innermost subject-verb dependency. (Bottom Right): In sentences with PP by the surprisal quantile for head and child. Shaded areas indicate the standard error of the mean all plots.
controlled
surprisal, and linear distance. These classifier models reveal which properties an existing syntactic dependency most relevant for probes predict it correctly.
The Ridge classifer model was implemented via scikit-leam's (Pedregosa et al., 2011 I) RidgeClassi fieer with regularization parameter 100.0. In parallel, the Random Forest classifier was configured using scikit-lear''s RandomE ores C1 asaassii fier with 100 estimators, a maximum tree depth of 10, and minimum of 500 samples per split to mitigate overfitting.
Both models were evaluated using 5-fold cross-validation. For each fold, feature importance values were extracted from the Random Forest model and subsequently signed using the corresponding coefficients from the Ridge Regression model.
4 Results
This section structured as follows: First, we study which linguistic and statistical factors best predict probe performance (section 4.1), using both naturalistic (section 4.1.1) and manually-crafted controlled (section 4. 1.2) datasets. Next, we investigate whether sentence parsing by structural probes exhibits processing phenomena similar to those observed in humans (section 4.2).
4.1 Probe performance is impaired by longer and deeper dependencies, but not by the surprisal of its elements.
4.1.1
Naturalistic data
We first study the Structural and Polar Probes on naturalistic dataset, the Universal Dependencies English Web Treebank (UD-EWT) dataset. For each analysis, we evaluate whether the distances between contextualized word embedding within the probe's 's subspace, accurately represents the syntactic distance in the annotated dependency tree. We evaluate the probe accuracy as a function of three linguistic features: (1) the linear distance that seperate two syntactically related words (i.e. the head and the dependent, fig. 2, top left), (2) the syntactic depth of the head (fig. 2, top middle), and (3) the surprisal of the head (fig. 2, top right).

---

Published as a conference paper at COLM 2025
We find that both linear distance and syntactic depth negatively impact probe accuracy, indicating that Long-range and deeply nested dependencies still challenge current probes. Remarkably, for very deep structure (depth=10), the probe's 's accuracy is similar to what can be read out from the original space of activations of the L.M (fig. 2, top middle). By contrast, word surprisal shows little to no impact on probe accuracy. These results mean that the predictability of a word given its context does not affect the syntactic representation identified by the syntactic probes (fig. 2, top right). Overall, these three results suggest that probe performance is more sensitive to structural properties of language than to statistical predictability. Suggesting better performance generalization to nonsensical sentences rather than to sentences containing long-range or deep dependencies.
Linear distance, syntactic depth and word surprisals may not be fully independent from one another. To verify that the above results, we trained Random Forest classi- fier to predict the probe decisions, depend- ing on these three linguistic features (lin- distance, syntactic depth and (head and child) surprisal).
Head surprisal
Feature
Child surprisal
Head depth
fig. 3 shows the signed feature importance for the different input variables. Overall, the results confirm that probe accuracy are most affected by the linear distance and the syntactic depth. By contrast, surprisals lead to weaker and less consistent impact (fig. 2, top right). These results reinforce our earlier findings, suggesting that probe performance most impacted by both syn- tactic depth and linear distance.
Linear distance
-0.5
-0.
-02
Feature Importance
Figure 3: Linear distance and depth of the syntactic dependency are the best predictors of probe perfor- mance Signed mean feature importance in Random Forest classifier trained to predict whether the Polar Probe detects existing dependencies in the UD-EWT dataset. The error bars represent the standard devia- tion of the feature importance values across 5 cross- validation folds.
4.1.2 Controlled sentences
The above analyses are based on syntacti-
cally annotated naturalistic sentences. To independently assess the impact linear distance, syntactic depth and word surprisal on the probes' accuracy, we extend our analyses to a set of controlled sentences, designed such that their syntactic dependencies systematically vary in linear distance and syntactic depth (see section 3.1). In all cases, probe accuracy is evaluated on the subject-verb dependency to ensure consistency across conditions. To manipulate linear distance (fig. 2, bottom left), we used main phrase sentences with a varying number of adverbs as fillers (e.g.. "TThe cat quickly and silently walks"). For syntactic depth (fig. 2, bottom middle), we evaluate the innermost subject-verb dependency in CE sentences with different levels of nesting (e.g., "The cat that the fox chases moves"). Where the subject and verb lie at a linear distance of for the different numbers of nesting. For surprisal (fig. 2, bottom right), analyzed sentences with single PP to avoid fillers (favoring more natural constructions) while introducing long-range agreement.
Once again, our findings align with those from the UD-EWT dataset: linear distance and syntactic depth strongly impact probe accuracy. For linear distance, fig. 2, bottom left we find a linear distance of 3 words is enough to decrease probe accuracy by 0,4 points. A similar effect happens for syntactic depth fig. 2, bottom middle. Notably, for CE sentences, the 'Activation Space' baseline matches with the Structural and Polar probes. We suggest that this due to the subject and the verb being located at adjacent locations. Finally, for surprisal (fig. 2, bottom right), we observe opposing trends, as reflected in fig. 3, where accuracy slightly increases with head surprisal, whereas child surprisal exhibits the opposite effect. When comparing the Structural and Polar probes, we find that the Polar Probe exhibits a slower decay in accuracy with increasing linear distance fig. 2, bottom left. We hypothesize that its more constrained objective, including additional syntactic information, fosters leamning more robust syntactic representations, that rely less on surface heuristics.
7

---

Published as a conference paper at COLM 2025
mouses by em move monses by the call moves
1.0-
1.0
Incongruent Congruent
0.8
0.8
Accuracy
Probability
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
sald odd
Mar ende
suli ool
Nit ubl
Number of nestings
Grammatical
Ungrammatical
1.0
1.0
0.8
0.8
Accuracy
Accuracy
0.6
0.6
0.4
0.4
nesting
0.2
0.2
2 nestings
nestings
0.0
0.0
2
3
4
2
Depth
Depth
Figure 4: Probes share some similarities with human syntactic processing but fail simple PP sentences. (Top left): Accuracy of the Polar Probe on the subject-verb dependency in grammatical PP sentences with varying nesting levels, comparing cases where the final attractor is congruent or incongruent with the subject. Congruent cases are harder for the probe. (Top right): Probability of different Polar Probe predictions in grammatical and ungrammatical with one Predictions are grouped by which word; the attractor, the subject, both, neither-bind to the verb. Because the attractor adjacent to the verb, the probes are misled and produce mostly incorrect parses (red), outnumbering correct ones (green). Ungrammatical cases increase the attraction effect. (Bottom left): Accuracy profile for Right Branching (RB) where the linear distance is constant across nestings, modifying syntactic depth. Probes show primacy and recency effects. (Bottom Right): Accuracy profile for Center Embedding (CE) sentences, inner dependencies, despite being deeper, are also closer together resulting higher probe accuracies. Error bars represent the standard error in all plots.
4.2 Do probes and humans show similar parsing effects?
Psycholinguistic research (Wagers et al., 2009; Bock & Miller, 1991; Francis, 1986; Kimball & Aissen, 1971; Haskell & MacDonald, 2005) has shown that interfering nouns tend bias individuals toward misinterpreting and misproducing long-range subject-verb agreements. These effects are especially pronounced in ungrammatical sentences, where attraction errors are more likely to occur. Moreover, increasing the linear distance between the subject and verb, seen in prepositional phrase and object relative clauses, further generates confusion in syntactic agreement (Bock & Cutting, 1992).
4.2.1 Congruent nouns impact negatively on probe performance
More recent work (Lakretz et al., 2022; Lampinen, 2024; Marvin & Linzen, 2018) has shown that congruent attractor nouns reinforce the confidence of LLMs when predicting verb forms during the next-token prediction task. (refer Appendix A for more details regarding congruence and model prediction). We hypothesize that, contrast, incongruent attractor nouns may cause confusion the syntactic parsing task, which not generative.
8

---

Published as a conference paper at COLM 2025
To test this hypothesis, measure sensitivity of probes to the congruency between the subject and the last attractor in PP sentences. Therefore, in congruent cases, the last attractor functions as an interfering noun positioned adjacent to the verb.
As depicted in Figure 4, top left, the Polar Probe exhibits greater difficulty with congruent cases, suggesting tendency to associate the verb with the attractor when surface forms compatible. Conversely, in incongruent cases, the absence of interference aids in disambiguating the correct subject-verb dependency, leading improved probe performance. Notably, as indicated by the star notation in Figure 4, top left, the difference between the two groups is highly significant across the different levels of nesting. Lastly, Figure top left illustrates the impact of attaching additional PPs on subject- verb accuracy; as expected, accuracy decreases with the number of PPs due to both the greater linear distance and the presence of more attractors.
4.2.2 Ungrammatical verb forms impact negatively on probe performance
Analogous to the interfering effects observed in grammatical sentences, previous works (Marvin Linzen, 2018; Hu et al., 2024; Ryu Lewis, 2021) have shown that ungrammaticality also fosters attraction effects in LLMs when evaluated in generative setting. Mirroring findings from psycholinguistic studies. As illustrated in Figures 4, top middle and 13, top middle, we find that in PP sentences with varying levels of nesting, the Polar Probe exhibits consistently higher error rate when encountering ungrammatical verb forms.
4.2.3 One PP suffices to elicit attraction effects
Remarkably, we note that even in simple sentences with single PP, the Polar Probe tends to bind the verb to both the subject and the attractor, resulting in a wrong parse (Figures 4, top middle and 13, top middle).
Such binding strategy comes at the expense of accurately capturing the case relation within the PP. Surprisingly, the case relation is challenging for the probe to parse as revealed by further dissection the UD-EWT dataset into dependency types. Refer to Appendix A for more details. Overall we find such result as an indicator of the extent to which probes are sensitive to linear distance. Such sensitiveness comes with prediction errors and points solid separation between humans and probes.
As shown in Figure 16, top middle such failure is somewhat mitigated by the use of masked language model, where attention heads operate bidirectionally.
4.2.4 Accuracy profiles for Right Branching and Center Embedding
Controlled sentences with Right Branching (RB) and Center Embedding (CE) structures each include subject-verb dependency per level of nesting. In RB sentences, syntactic depth increases while the linear distance between the subject and the verb remains constant. In contrast, in CE sentences, the subject-verb linear distance decreases as syntactic depth increases. These structural differences lead to distinct accuracy profiles with respect to the number nestings and syntactic depth.
As shown in fig. 4, top right and consistent with the results presented in fig. accuracy RB sentences declines with increasing syntactic depth. Notably, U-shaped accuracy profile emerges, suggestive of primacy and recency effects documented in human cognition (Murdock, 1962: Atkinson & Shiffrin, 1968; Baddeley & Hitch, 1974), where items at the beginning and end of a sequence tend be processed more accurately than those in the middle.
In contrast, accuracy in CE sentences improves with deeper nesting. This finding underscores the dominant influence linear distance over syntactic depth as shown fig.
5 Discussion, Limitations and Future work
Our analyses reveal that accuracy of syntactic probes is primarily sensitive to structural rather than to statistical properties. In fact, in both naturalistic and controlled datasets, find that the model's surprisal to the words in the dependency barely has an impact on probe accuracy. In contrast, probe accuracy decreases as the linear distance and syntactic depth of the dependency increases.

---

Published as a conference paper at COLM 2025
Notably, in relatively simple sentences with one Prepositional Phrase (eg: The keys to the cabinet big.), the probe tends to bind both the subject and the attractor to the verb, yielding a wrong parse (Section 4.2.3).
Such result puts forward a key challenge for linear probes, their sensitiveness to linear distance and syntactic depth makes them prompt to errors. In the PP case, the probes' sensitiveness to linear distance becomes especially apparent- the attractor being adjacent to the main verb is enough to confuse probes to yield a wrong syntactic parse.
To address these challenges, and better evaluate probes, we introduce a set of controlled sentences benchmark. These carefully designed stimuli enable linguistically motivated and systematic evaluation of probe performance.
to
Despite these challenges, probes also exhibit notable successes. They capture syntactic structure more accurately than the model's raw activation space, and perform far beyond linear distance baseline. Moreover, we find some similarities between probes and human behavior. Specifically, these similarities manifest as increased syntactic errors under both noun interference and presence ungrammatical verb forms (Wagers et al., 2009; Bock Miller, 1991). However, this resemblance remains superficial due to the current failure cases observed in linear structural probes.
Several factors likely underlie the current challenges of structural probes. First, their training data is strongly skewed toward dependencies with a linear distance of one, introducing bias into the learned linear probe during its training. Second, if LLMs encode syntax along a nonlinear manifold in hidden space, any strictly linear probe will inevitably distort that representation. Third, some errors may originate not from the probes themselves but from the LLMs, which may fail to encode syntactic structure with full fidelity. Lastly, prompting LLMs has been shown affect their downstream behavior (Lampinen, 2024), contextualizing the sentences using a carefully designed prompt might result in linguistically richer representations, potentially enhancing the performance of syntactic probes.
We posit that exploring non-Euclidean probes that better align with the geometry of LLM represen- tations is a promising direction for future work. Likewise, training syntactic probes on controlled stimuli to better capture long-range dependencies and deeper hierarchical structures remains an important area for continued research.
Acknowledgments
This project was provided with computer and storage resources by GENCI at IDRIS thanks to the grant 2023-AD01 1014766 on the supercomputer Jean Zay's the V100 and A100 partition (PDS). This project has received funding from the European Union's 's Horizon 2020 research and innovation program under the Marie Goldowska-Currd grant agreement No 945304 (PDS).
This project received funding from PSL University under the grant agreement 02 (EC).
This project received funding from the Département d' Études Cognitives (DEC) at ENS under the grant agreement FrontCog, ANR-17-EURE-ON (EC).
This project received funding from the French National Research Agency (ANR) under the grant agreement ComCogMean, Projet-ANR-23- 2----------------- (EC).
References
Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes, 2017. URL https: h/wp//ppeeeee..ecoooo/.ep. net forum? (
R.C. Atkinson and R.M. Shiffrin. Human Memory: A Proposed System and its Control Processes, pp. 89-195. Elsevier, 1968, doi: 10.0016:00799742210)))444444
URL http:// dx. doi org/
09.1016/50079-744 (08) 60422-3.
Alan D. Baddeley and Graham Hitch. Working memory. In Gordon H. Bower (ed.), New York: Academic Press, volume 8 of Psychology of Learning and Motivation, pp. 47-89. Academic Press, 1974. doi: http:://oo.aog//...0.00oo0o00/s1
URL https:
//www. www.sccienced... rect. www.ttooo/ccieene.eeeee htt.a//cww.......o.........oe/np
10

---

Published as a conference paper at COLM 2025
Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sajjad. and James Glass. What do neural machine translation models learn about morphology? In Regina Barzilay and Min- Yen Kan (eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 861-872, Vancouver, Canada, July 2017. Association for Computational Lin- URL https: aall anthelogy.. www.yy..rrr/1.... o01p11100000////////
guisties. doi: 10. 10.18833/11/11--11
Kathryn Bock and J. J.Coer Cutting. Regulating mental energy: Performance units in language production. Joumnal of Memory and Language, 31(1):99-127, 1992. ISSN 0749-596X. doi: https: http:////00.00060007999990000009 scienne//arrtcllee////////999999
URL https: wwww/w.ww.......o//...o www.sccinnediieec..... com/
Kathryn Bock and Carol A Miller. Broken agreement. Coguitive Psychology, 23(1):45-93, January
1991. ISSN 0010-0285. doi: 11000000110008899999900000 10. 1tt.111111111-100 0280 (91) 90003-7.
URL http: /dx doi org/
Charlotte Caucheteux, Alexandre Gramfort, and Jean-Remi King. Disentangling syntax and semantics in the brain with deep networks. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 1336-1348. PMLR, 18-24 Jul 2021. URL https: http://pocceeeig..../ palespress vt///v1399///// wwt.:///ccceeeeeeeeccc html.
Boli Chen, Yao Fu, Guangwei Xu, Pengjun Xie, Chuanqi Tan, Mosha Chen, and Liping Jing. Probing (bert) in hyperbolic spaces. In International Conference on Learning Representations, 2021. URL
https: /ttps:://ppnnevieww
forum? 7Vdd7yy
Noam Chomsky. Syntactic Structures. De Gruyter, December 1957. ISBN 9783112316009, doi: 10.1515/9783112316600 URL http://ds: dwi. 011.:1.1111111111 /ttp:///00..111//111111111000000
Noam Chomsky, Language and nature. Mind, 104(413):1-61, 1995, doi: 10.1093/mind/104.43..1
Alexis Conneau, German Kruszewski, Guillaume Lample, Loic Barrault, and Marco Baroni. What you cram into single $& vector: Probing sentence embeddings for linguistic properties. In Iryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2126-2136 Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10.18653/v1/18--11 URL https:tt/// htt//aaaaaaattoooooeeeeaaa org/p18-
William Croft. Autonomy and functionalist linguistics. Language. 71(3):490, September 1995. ISSN 0097-8507. doi: 10.2307/416218. URLhttp: htt.t:://///. ///////xxxxxx doi org 10. 2307/416218
Stanislas Dehaene and Laurent Cohen. The unique role of the visual word form area in reading. Trends in Cognitive Sciences, 256))225- June 2011. ISSN 1364-6613, doi: 10. 10.16///.tics. 2011.04.003 URL http:: doi www.oorg//....
tics. 2011. 04. 003.
Lune-Dominique Devauchelle, Catherine Oppenheim, Luigi Rizzi, Stanislas Dehacne, and Christophe Pallier. Sentence syntax and content in the human temporal lobe: An fmri adaptation study in auditory and visual modalities. Journal of Cognitive Neuroscience, 21000 May 2009. ISSN 1530-8898. doi: 10. 10.1162/joen.2209.20000 21070. URL http: dxx doi woo.ooorg/...1.. jocn. 20...220700000
Jacob Devlin, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pp. 4171 4171-4186 Minneapolis, Minnesota, June 2019, Association for Computational Linguistics. doi: 10. 18653/v1/N19-1441 URL https: / aac lnnthology org/ 1E//NN99112////
Pablo J. Diego-Simon Stéphane d' Ascoli, Emmanuel Chemla, Yair Lakretz, and Jean-Remi King. A polar coordinate system represents syntax in large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https: httpps://ppeeeeeeeeeeeee /ppp///eee/ee.eee//////// net net/fol met/ffoum? d=x27 dwx.mm 8..
11

---

Published as a conference paper at COLM 2025
Tiwalayo Eisape, Vineet Gangireddy, Roger Levy, and Yoon Kim. Probing for incremental parse states in autoregressive language models. In Yoav Goldberg, Zomitsaa Kozareva, and Yue Zhang (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 2801-2813, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.186633w1/2022.. http:///2222.nnseeeeeeeee. findings-emmll.2055 URL https:://ccaatthhttt/ wwwpegg...../r/eeeeee 2022.
aoddings-emnii 203/.
W. Nelson Francis, Proximity concord in english. Journal of English Linguistics, 19:309 317, 1986.
URL https:://///
/api. http://app...mma.......mmmm wwwant wcscholar..ccccccccccoo.....a... :
Lyn Frazier and Keith Rayner. Resolution of syntactic category ambiguities: Eye movements in parsing lexically ambiguous sentences. Journal of Memory and Language, 26(5):505 October 1987. ISSN 0749-594 -966.. doi: 10. 10.1016/0749- 596x(87)90137799 URL http: //t:://xxxx doi org/ 10016007495596x(879901101 10. 10..101//01 007//////---2 9- 596X (87) 90137-9.
Angela Friederici. The neural basis for human syntax: Broca's area and beyond. Current Opinion in Behavioral Sciences, 21:88-92, 2018. ISSN 2352-1546. doi: https://doi.or//10.0011 2.10beha.201..33..0 URL ht tpst:ps// /wwww. sciencediree www.ceereet.ccoccceeeeeeeeeeeeee cotsss/ceeeeeeeerraicceee.. p21//23521546173000
The Evolution of Language.
Lila Gleitman. The structural sources of verb meanings. Language Acquisition, 1(1):3-55, January 1990. ISSN 1532-7817. doi: 10. (0.1207/115322811110000 2. URL http::/dx..e... http:/xxxdoooo//1111..///
2
Yosef Grodzinsky and Angela D Friederici. Neuroimaging of syntax and syntactic processing. Current Opinion in Neurobiology, 16(2):240-246, 2006. ISSN 0959-4388. doi: https://doi.
1tt000001661..comb2006.000333330 URL https::/p/// /wwww//www/////////////////// sci enceeii rect. wrc.t.com/ttt.ec.. con/science// Cognitive neuroscience.
Raffaele Guarasci, Stefano Silvestri, Giuseppe De Pietro, Hamido Fujita, and Massimo Esposito. Assessing bert's 's ability to learn italian syntax: study on null-subject and agreement phenomena. Journal of Ambient Intelligence and Humanized Computing, 14(1):289 303, May 2021. ISSN doi: 10. 10.10077/11265--21--633399
1868-5144 5148-
URL http: dx oro.0..001....000
s12 1126522-0 021 021- 03297-4, 4.
Aric Hagberg, Pieter Swart, and Daniel S Chult. Exploring network structure, dynamics, and function using networkx. Technical report, Los Alamos National Lab,(LAN)), Los Alamos, NM (United States), 2008
John Hale and Miloš Stanojević. Do LLMs learn a true syntactic universal? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing, pp. 17106-17119, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.1 18653/6 100533//12022.e///m w/2022.emmmlpmmiinnn..mmm .950. URL httpss: acl anthhloogee org/2024/222200 emn emnlpmma i n.. 950/.
Todd R. Haskell and Maryellen C. MacDonald. Constituent structure and linear order in language production: Evidence from subject-verb agreement. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(5):891-904, 2005, ISSN 0278-7393. doi: 00.1037/0278-739333.555 URL http: / / dx doi. org 10. 0001100777222 027001278000 7393. 31 . 5. 891.
John Hewitt and Christopher D. Manning. structural probe for finding syntax in word repre- sentations. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume (Long and Short Papers), pp. 4129-4138, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 1118665/11/N19 1419. 1419/
URL httph:tt////mm. http://aaaaaattttaea//ee
Jennifer Hu, Kyle Mahowald, Gary Lupyan, Anna Ivanova, and Roger Levy. Language models align with human judgments key grammatical constructions. Proceedings of the National Academy of Sciences, 121(36):22000917122, 2024. doi: 10. 10.1073/pms.2440011111,, URL https: //www. /wwp:pppa.s... //tpaaa.ooo/oo//aa/////////////s
24009171
12

---

Published as a conference paper at COLM 2025
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud. Marie-Ama Lachaux. Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL https: //arxiv...vvv// org/abs/ 2310 0. 06825.
Edith Kaan and Tamara Y Swaab. The brain circuitry syntactic comprehension. Trends in cognitive sciences, 6(8):350-356, 2002.
John Kimball and Judith Aissen. think, you think, he think. Linguistic Inquiry, 2(2):241-246, 1971.
ISSN 00243892, 15309150. URL http: /www. jstor.stoo..o wwwp:jjstorrrg/ttttteeoooo wttpr/orr///aaateeeeee///e/
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017. URL https:/ wwwpvv.rrvvvvva////a/e../...a. org/ abs 1112.41122. 121....0 6980.
Joseph B. Kruskal. On the shortest spanning subtree of a graph and the traveling salesman problem. Proceedings of the American Mathematical Society, 7(1):48 50, February 1956. ISSN 1088- 6826. doi: 10. 001100000000099900 9939- 956-0078686-77 URL http:/ /dx. www. 10990/ 1090/ S0C02
Yair Lakretz, Théo Desbordes, Dieuwke Hupkes, and Stanislas Dehaene. Can transformers process re- cursive nested constructions, like humans? In Nicoletta Calzolari, Chu-Ren Huang. Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, and Seung-Hoon Na (eds.). Proceed- ings of the 29th International Conference on Computational Linguistics, pp. 3226-3232 Gyeongju, Republic of Korea, October 2022. International Committee on Computational Linguistics. URL https: / /tt///alaaaa/// antho 1 ogy org/ 2022 2oo..coliing.........o.......... col.ng-.....n.
Andrew Lampinen. Can language models handle recursively nested grammatical structures? case study on comparing models and humans. Computational Linguistics, 50(3):1441 December 2024. doi: 10. 10.1162/coli a00tc2ll.l_000.....__000
URL hwtpt::t////........ttt ac 1 anthology. www. 2024. cll 4..44..0 8/.
Tomasz Limisiewiez and David Mareček. Introducing orthogonal constraint in structural probes. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the I 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). pp. 428-442, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v 1/2021 acc--long.3 URL https: acl www.annoooloyyy..ooooooooooooole org/ 2021 acl-long.36/
Rebecca Marvin and Tal Linzen. Targeted syntactic evaluation language models. In Ellen Riloff, David Chiang. Julia Hockenmaier, and Jun ichi Tsujii (eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 192-1202, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 11151//11/11111111 URL https: htt///aaaaanntoooa org/ D18 1151/.
Rowan Hall Maudslay and Ryan Cotterell. Do syntactic probes probe syntax? experiments with jabberwocky probing. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani- Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Associa- tion for Compulational Linguistics: Human Language Technologies, pp. 124-131, Online, June 2021. Association for Computational Linguistics. doi: 10. 10.18853/v1/2221 www.nl-main.n..... 11. URL ht attps://eet htttp::/accaattholoeeo
org/2021 mwacc-mminnn111
Max Milleer-Ebersten, Rob van der Goot, and Barbara Plank. Probing for labeled dependency trees. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.
7711-7726, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653. v1/2022.acl-oogg552. URL https: : / /aaalaaantti aalahhhhooogy org/ 202/220222222 ac mocel-long.533 532/.
Bennet B Murdock. The serial position effect of free recall. J. Exp. Psychol., 64(5):482-488 November 1962.
13

---

Published as a conference paper at COLM 2025
Joakim Nivre, Daniel Zeman, Filip Ginter, and Francis Tyers. Universal Dependencies. In Alexandre Klementiev and Lucia Specia (eds.), Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Tutorial Abstracts, Valencia, Spain, April 2017. Association for Computational Linguistics. URL https: : / /acll moclannhooogy org/ E1 E17-5001/.
Christophe Pallier, aane-Domniique Devauchelle, and Stanislas Dehaene. Cortical representation of the constituent structure of sentences. Proceedings of the National Academy of Sciences, 10866):2522-2527, January 2011. ISSN 1091-6490. doi: 10.1073/pmas. 101871 1108. URL.
http:: /tx. doi. www.ooog//....o.... 1 1 /t/00///////./////// 1011701111100
Core Francisco Park, Andrew Lee, Ekdeep Singh Lubana, Yongyi Yang, Maya Okawa, Kento Nishi, Martin Wattenberg, and Hidenori Tanaka. In-context learning of representations. In The Thirteenth International Conference Learning Representations, 2025. URL https: // //openreview.o net/ for comm2dd=pllmmm HJJ.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-lcarn: Machine learning in Python. Journal of Machine Learning Research, 12:2825 27:3025-2830, 2011.
Matthew E. Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. Dissecting contextual word embeddings: Architecture and representation. In Ellen Riloff, David Chiang. Julia Hockenmaier, and Jun ichi Tsujü (eds.). Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1499-1509, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10. 10.18653/11/D18-1111 URL https::/// / accl//acclaaaaa anthology.
org/Di 2t////88-17991
Tiago Pimentel and Clara Meister. How to compute the probability of word. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 18358-18375, Miami, Florida, USA, November 1020. URL
2024. Association for Computational Linguistics. doi:
https: http///aaaaanttooogyoooooo org org/2024 2024. emnlp-ma in..nii.... 1020/.
Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, and Ryan Cotterell. Information-theoree probing for linguistic structure. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4622, Online, July 2020. Asso- ciation for Computational Linguistics. doi: (0.18653/v 1 1/2/2/022ll1////.. acl-aan..l-m URL https: wow.. ogy. org/ 2020 0 acl -aaimain maainn.. 420/.
Keith Rayner. Eye movements reading and information processing, Psychological Bulletin, 85(3): 618 660, 1978. ISSN 0033-2909, doi: 10. 1037/0033--209.855 85.3.618. URL http: /xxx doi. org/10. 100110177740 00077/003322200
3. 618.
Emily Reif, Ann Yuan, Martin Wattenberg, Fernanda B Viegas, Andy Coenen, Adam Pearce, and Been Kim. Visualizing and measuring the geometry of bert. In H. Wal- lach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https: http:///oooeeedeeeeee wwwediinnnnnnneeeeeeeeeeeeeee neurips. file/159chh (00e1fffb66111 b4 4d3 (40ddd84444220 wwurips..ccpaperr... wccpppperrfilleeeeeeeeeeeeepeeee warrrffles/pppppe/ee /ttlee//pppr///19//// .po..
Soo Hyun Ryu and Richard Lewis. Accounting for agreement phenomena in sentence comprehen- sion with transformer language models: Effects of similarity- interference on surprisal and attention. In Emmanuele Chersoni, Nora Hollenstein, Cassandra Jacobs, Yohei Oseki, Laurent Prévot, and Enrico Santus (. Proceedings of the Workshop on Cognitive Modeling and Compu- tational Linguistics, pp. 61-71, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v 1/20053/3///222lmmmcccc. 1.6. URL https: aclanthslogy org/ 2rg/2021. cmcl-1. 6/.
Andrea Santi and Yosef Grodzinsky. Working memory and syntax interact in broca's 's area. Neurolm- age, -- August 2007. ISSN 1053-8119. doi: 10. ner.:of..uu0im 24.0044.4777 URL http::/ddx doi wwt.oooorrro...000 6/j nwwroimage.e..e 2007. 04. 047.
14

---

Published as a conference paper at COLM 2025
Adam Shai, Paul M. Riechers, Lucas Teixeira, Alexander Gietelink Oldenziel, and Sarah Marzen. Transformers represent belief state geometry their residual stream. In The Thirty-eighth Anmual Conference on Neural Information Processing Systems, 2024. URL htttpa://opeeeeeiiiee net fet./for forum? i d=YTB 3==YTB7REN 7RELBUC.
C. E. Shannon. A mathematical theory of communication. Bell System Technical Journal, 27 (3):379 423, July 1948. ISSN 0005-8580 doi: 10.1002j 7305,, 1bb8..b. URL http://dke
doi. wtw.da/ooo..o......o/oooo 1tt./...////////o...........ooot 101.110000//111133..1
Natalia Silveira, Timothy Dozat, Marie-Catherine de Marneffe, Samuel Bowman, Miriam Connor, John Bauer, and Christopher D. Manning. gold standard dependency corpus for English. In Proceedings of the Ninth International Conference Language Resources and Evaluation (LREC-2014), 2014.
Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Sam Bowman, Dipanjan Das, and Ellie Pavlick. What do you learn from context? probing for sentence structure in contextualized word representations, In International Conference on Learning Representations, 2019. URL https: https:://ppeeeeeeeeeeeeee /ttp////pprreeiieee/eeeee net forum? idd=8J25gRRcKX.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cris- tian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, lliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023. URL wwwpt:tpp/.ooo...... /ttp://aaxvvvv//. www. aaws/////////e/eeee 2307 .2.307..92288 09288.
Matthew W. Wagers, Ellen F. Lau, and Colin Phillips. Agreement attraction in comprehension: Representations and processes. Journal of Memory and Language, 61(2):206-237, August 2009. ISSN 0749-596X. doi: 10.
URL http:/ / doi. .ar.l.org////
jm1m 2009.00000000000 .4.000...0...000.0 002.
Jennifer C. White, Tiago Pimentel, Naomi Saphra, and Ryan Cotterell. non-linear structural probe. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani- Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 132-138, Online, June 2021. Association for Computational Linguistics. doi: 18653/v / 1/2021 maacl-mmiii 12. URL https: / /awcllanttttt weceeannhhloggg org/ 202 www.paaaaaa....... maacl-main. 12/.
15

---

Published as a conference paper at COLM 2025
A
Appendix
A.1 Polar Probe implementation
Each edge (dependency) e E Ew in the graph (dependency tree) is associated with two functions: U and C, which determine its directionality and label (dependency type) respectively.
u Ew - (-1,1), e+ u(e), C: Ew -> C, et c(e).
(10)
Similarily the Structural Probe, the Polar Probe Diego-Simon al. (2024) defined as Bp R Red linearly transforms the LLM's embedding space H. into subspace P following eqs. (5) to (7).
pl (wi, wi) Bpo (w; wj), Mij = l|p (w,, w,)112
(1I)
Additionally to the structural objective ( eq. (7)), the Polar Probe introduces angular objective A so that p' (w, wi) where {æ0, E Ew encodes information about U and C. For that, set of edges Qe are extracted across sentences from dataset D. Then, the following objective is minimized:
(Bp) = [e
1
Σ cp'ENe
((pp
(12)
-l(((e) = c(e')))
where -(.,)) denotes the cosine similarity, and write pt shorthand for p' (w,, wi) for an edge e= fW; E Ew Ωg. by slight abuse of notation.
Therefore, the Polar Probe is the result of the following objective function:
Bp arg min (Bp + AL (Bp)
(13)
Bp
A.2 Additional baselines:
Linearly informed random prediction
In this baseline, for two words at positions i and j in sentence, define their distance to be the absolute difference in their positions, plus a small random perturbation Eiy;
M, = li -i- jl + Eij.
Here, Eij is drawn i.i.d. Eij is sampled i.i.d. for < from a noise distribution and we set Eij Ej and еÚ 0 keep M symmetric and zero-diagonal.
Random prediction
To establish a lower bound, we assign completely random distances:
M
Cijs
16

---

Published as a conference paper at COLM 2025
104
103
Frequency
10
10
100
10
20
30
40
Linear distance
Figure 5: Histogram of linear distances for dependencies in the naturalistic sentences
Singular
0.6
Plural
prob(s)
0.5
0.4
0.3
0.1
SS
RS
Grammatical number
Figure 6: Probability for predicting the token 's" for different grammatical number combinations of the subject and the attractor.
con
Dependency type
adve
abl
nmod
ad
38
xcomp
obj
-54
cop
compound
det
amod nuromod
flat
9
13
17
21
Linear distance
Figure 7: Polar probe accuracy for dependencies by their linear distance and dependency type.
17

---

Published as a conference paper at COLM 2025
1.0
Polar
Stracturn)
0.8
Activation Space
Accuracy
Random
0.6
Linear Randem
0.4
0.2
0.0
4
8
10
Linear distance
Figure 8: Accuracy function of linear distance in UD-EWT
1.0
0.8
Accuracy
0.6
0.4
0.2
0.0
4
8
10
Depth
Figure Accuracy as a function of syntactic depth in UD-EWT
Head surprisal
Child surprisal
Feature
Head degree
Child degree
Head position
Head depth
Linear distance
05
0.4
0.1
Feature Importance
Figure 10: Feature importance values in Random Forest model with additional features.
18

---

Published as a conference paper at COLM 2025
B Llama2-7B
1.0-
1.0-
1.0-
Seretund
Artentan
Acruracy
0.6-
0.6-
Acruracy
0.6
0.4
0.4
0.4
0.0
2
10
0.0
0.0
10
Linear
Depth
surprisal (quantile)
mouse quickly silently moves
mouse moves
mouse
1.0
1.0-
1.0
0.8
Ancuracy
0.6
0.6
Apeurasy
0.6
0.1
0.4
0.2
0.2
10
2
3
7
10
Linear distance
Depth
Surprisal (quantile)
Figure 11
surprisal
Child surprisal
Child surprisal
Peature
Head surprisal
Foature
Child degree
Head position
Head depth
Head degree
Head depth
Linear distance
Linear distance
46
65
.1
48
4a
Feature Importance
Feature Importance
Figure 12
19

---

Published as a conference paper at COLM 2025
1.0
Grammatical
Attr only unly
Incongruent Congruent
1.0
Ungrammatioal
0.8
Sulj Attr
Netther
Accuracy
Probability 0.2
0.8
0.6
0.6
0.4
0.4
0.2
0.0
0.0
2
3
Number of nestings
Number of nestings
1.0
1.0
0.8
0.8
Accuracy
Accuracy
0.6
0.6
0.4
0.4
no nesting nesting
0.2
2 nestings
0.2
3 nestings
0.0
0.0
3
4
Depth
Depth
Figure 13
20

---

Published as a conference paper at COLM 2025
C
Mistral-7B
C.1
Structural Probe
1.0
Incongruent
Grammatrral
Attr only
1.0
M
Ungrammatical
Subj only
Congruent
0.8
Nettber
Accuracy
Probability
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
2
3
Number of nestings
Number of nestings
1.0
nesting nesting nestings nestings
1.0
0.8
0.8
Accuracy
Accuracy
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
2
3
4
2
4
Depth
Depth
Figure 14
21

---

Published as a conference paper at COLM 2025
D BERT-
1.0
1.0
Polar
Structural
0.8
0.8
Activation Space
Accuracy
Accuracy
0.6
0.4
0.4
0.2
0.2
0.0
0.0
2
6
8
10
6
10
Linear distance
Depth
The mouse quickly and silently moves
The mouse that the cat chases moves
1.0
1.0
Accuracy
0.8
Accuracy
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
7
10
Linear distance
Depth
Figure 15
22

---

Published as a conference paper at COLM 2025
1.0
Incongruent Congruent
Grammatical
Attr only
1.0
Ungrammatioal
unly
Suly Attr
Netther
Accuracy
Probability
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
2
3
Number of nestings
Number of nestings
1.0
1.0
0.8
0.8
Accuracy
Accuracy
0.6
0.6
0.4
0.4
no nesting nesting
0.2
2 nestings
0.2
3 nestings
0.0
0.0
3
4
Depth
Depth
Figure 16
E
Controlled Dataset
E.1
Prepositional Phrase
Sentence
Structure
Attraciors
Fillers
some neighbours by some buys rost the teacher below some friends around bakers
re
2
tailor above the boy near teacher by the kids resis
PP
3
boy quickly and carefully stand
PP
the grandma helow friend gently eagerly playfully happily and nervonsly plays
2
behind soldier the playfally sileatly
Table I: Examples of PP sentences in the controlled dataset.
E.2
Center Embedding
Sentence
Structure
Attractoes
Fillers
some teuchers that the tuiler admines walk
some players neighborr
C0
the teacher that some moms that a talke that the guests show finds lift stands
3
0
the grandmas somse engineers toudly canefully and beldly help smile
60
an artist that the player swittl happily joyfully silently gently quierly and nervously heurs sits
2
the artist that the neighbor that the soldier joyfully boldly and quickly kicks meets walks
ce
Table 2: Examples of CE sentences in the controlled dataset.
23

---

Published as a conference paper at COLM 2025
E.3
Right Branching
Structure
friend assumes that the hikes prodiets
db
0
singer imagines the guesis foel that neighbor realizes
1b
2
the realize that writer predicts that some teachers think engineer imagines
1b
3
buy feels that snd shy and hrae engineer expects
1b
notices friendly young angry funny and wild believe some grandinas feel that some neighborr sotace that the funay kind and shy boys know
ab
Table Examples RB sentences in the controlled dataset.
24


Reviews:
[{"review_id": "oPUwcUv0uJ", "reviewer": "Reviewer_D9Dc", "summary": "This paper presents a thorough investigation of structural probe and its extension, polar probe. It begins by evaluating both probes on naturalistic sentences and finds that linear distance and syntactic depth significantly affect the probe's ability to capture sentence structure, while word predictability does not appear to influence performance. Based on these observations, the paper proposed three controlled stimuli: prepositional phrase, center embedding, and right branching, and use them to further validate the observations. The results suggest that the structural probes are biased by the linear distance between two syntactically related words and the syntactic depth of the head.Additional factors such as congruent nouns and ungrammatical verb forms also influence probe performance. Moreover, the paper provides a analysis of structural probes and human parsing, offering insights into their similarities and differences. The authors propose to use controlled sentences as a future benchmark to evaluate the performance of structural probes.\n\nIn summary, this work presents several interesting observations and highlights the potential limitations of current structural probe design.  However, I believe the paper would benefit from more comprehensive experiments and deeper discussion. In particular, I have concerns about whether the performance drop observed on the controlled stimuli is expected, which raises questions about the validity of the methodology and the strength of the conclusions.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper studies an interesting and important problem, understanding the success and limitations of structural probes of large language models. The experiments offer several valuable observations and insights for the future design of structural probes.\n2. The paper is well-organized, and the writing is generally clear and easy to follow.", "reasons_to_reject": "1. Since this paper aims to examine whether structural and polar probes rely on superficial properties of sentences and to evaluate the robustness of these probes in capturing linguistic features, it is important to ensure the generalizability of the findings and the comprehensiveness of the experiments. I noticed that the experiments are conducted only on Mistral-7B-v0.1. Can the observation be generalized to other models such as LlaMa? Moreover, for the targeting probe, the related work mentioned other extensions of structural probes such as spectral probes.Why was the polar probe chosen for this study? It would be helpful to add a justification for this choice. Moreover, while the structural probe and polar probe are both evaluated in this paper, there is no in-depth discussion on comparing those two probes. For example, structural probe exhibits a larger performance drop on controlled sentences compared to the polar probe (as shown in Figure 2 Bottom Left). Is there any explanation for this difference?\n\n2. Please note that I am not an expert in syntactic structure analysis. In your experiments, how do you ensure that the construction of the controlled stimuli does not affect the original dependency tree and syntactic structure of the sentence?  Additionally, the probes are trained on naturalistic sentences from UD-EWT, while the controlled stimuli may be considered out-of-distribution. If so, wouldn’t a performance drop be expected? As a benchmark, how do you determine that robustness to these controlled sentences is a desirable behavior? While the empirical results suggest some similarity between structural probe behavior and human cognition (Section 4.2), it would be helpful to include human evaluation on the controlled stimuli to further support this argument.", "questions_to_authors": "1. Line 149, is $B_p^*$ the polar probe?  It is not mentioned earlier in the main content.\n2. I noticed that the scale of linear distance in Figure 5 ranges from 0 to 50, whereas in Figure 2, it ranges from 0 to 10. Why does this difference occur?\n3. It would be helpful to include qualitative examples where the structural probe fails, particularly at longer linear distances or increased syntactic depth.", "extra_scores": {}}, {"review_id": "SS7dq2illt", "reviewer": "Reviewer_dAJ4", "summary": "This paper examines the performance of structural probes on three benchmarks that the authors have composed, with the goal to investigate how structural and statistical factors systematically affect the probes’ performance. The benchmarks are constructed regarding three syntactic strucrtures: PP, CE, and RB, each of which contains three nesting levels {1,2,3}, indicating the difficulty for correctly probing the syntactic relation between subject and verb. \nThe main findings reveal that the probe performance is sensitive to structural features such as distance between words and syntactic depth, rather than lexical-level statistics, such as surprisal. There are similarities between probes and human parsing errors.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The research question is well-motivated and clearly defined. \n- To explore what kind of syntactic knowledge is learned by LLM meaningful to linguistic theories.\n- The benchmarks collected will be useful for future work.", "reasons_to_reject": "- The writing of the later part can be further polished. Some of the arguments in discussion are not solid, for example\n  - What does “geometric structure syntax” mean at line 303?\n  - At line 304, “probes are trained without any initial context” is obvious, right? If it is a sophisticated model that takes more context as input, it won’t count as a “probe”.\n  - I do not see why the paper ends with a reference to prompting papers (e.g., Kojima et al., 2022). Maybe I missed somthing here. I knowthat strctural priming is about how syntactic structures that have occurred in previous context tend to occur later more likely, and how is this classicial problem investigated in LLM? Maybe the authors can provide some brief review in a related work section.", "questions_to_authors": "line 12: and of => of\nline 177: perplexity => surprisal", "extra_scores": {}}, {"review_id": "bvToCFeIQl", "reviewer": "Reviewer_qy93", "summary": "This paper compares three (two + a baseline probe) syntactic structural probes on Mistral representations of naturalistic and curated constructed sentences.\n\nI'm not confident that I properly understood the paper, which is reflected in my review", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The research question is important. Researchers typically take the outputs of these structural probes at face value, but this paper instead looks at what biases the probes themselves may introduce.", "reasons_to_reject": "A major point of confusion for me, which is also criticism, is the use of Mistral as the gold standard. If the authors are working to understand the biases introduced by the two probes, then they must assume that Mistral has the \"correct\" representations, an any effects that they observe are deviations from this correct representation. It's the reverse of the problem that the authors are addressing (at least in my understanding) where work assumes that the probe is accurate and any deviations are the result of the underlying model. Wasn't clear to me how the authors. For example, is the sensitivity to linear distance (Fig 2 BL) primarily a property of Mistral or of the probes?\n\nRelated to that, I think it would be particularly important to repeat this study with other LLMs, which maybe could help differentiate\n\n\nI found the interpretation of Fig 3 kind of odd, or at least the description of it. The argument is that since head depth and linear distance dominate in feature importance, \"these results reinforce our earlier findings, suggesting that structural properties of syntax primarily drive probe performance.\" However, \"structural\"/hierarchical distance is usually contrasted with linear/string distance, so this description of the results feels misleading.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q7Yz3nb85L", "reviewer": "Reviewer_tL7W", "summary": "The paper proposes an investigation about how the syntactic information obtained from structural probes on embeddings from LLMs layers are biased or affected by specific syntactic representations in the analysed sentences.\nThe work is important to better understand ho LLMs \"understand\" syntax and syntactic phenomena in the sentence and where this knowledge is embedded.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The results highlighted by the paper provide more detailed hints on both the information that structural probes are capable of obtaining from LLMs, as well as how some syntactic constructions are difficult to be understood by LLM.", "reasons_to_reject": "Some parts of the paper are not very detailed and this could made the paper difficult to be read and understand by non domain experts.", "questions_to_authors": "The sentence at line 37: \"However, the performance of such Syntactic Probes has only been assessed with aggregated scores over large naturalistic corpora.\" doesn't includes some some studies, where detailed tests over sentences which include specific syntactic phenomena peculiar of some languages have been selected. In example, two studies that tested subject omissibility and subject-verb agreement, building tailored senteces sets to verify if and where the LLMs' layers encode the knowledge about these kinds of sentences, are presented in:\n\nGuarasci, Raffaele, et al. \"BERT syntactic transfer: A computational experiment on Italian, French and English languages.\" Computer Speech & Language 71 (2022): 101261.\n\nGuarasci, Raffaele, et al. \"Assessing BERT’s ability to learn Italian syntax: A study on null-subject and agreement phenomena.\" Journal of Ambient Intelligence and Humanized Computing 14.1 (2023): 289-303.\n\nYou should mention these studies and improve the sentence at line 37, taking into account such kind of studies.\n\nPeriod is missing at the end of line 47.\n\nLine 90: you should highlight that long and complex dependencies are very difficult to be understood also by humans. Although in constructions with relative clause the distance between pronoun that\nintroduces the subordinate and antecedent could be virtually increased to infinity still producing a grammatically correct sentence, the results sentence becomes very  complicate and slightly unusual (such the sentence at line 90: \"The cat believes that the fox expects that the mouse moves\").\n\nLine 91: what is the generative grammar used? Add the detail and a reference.\n\nLine 130: Why the details of the polar probe are included in an appendix, while structural probe's details are included in the main text. Considering the most recent publication of the polar probe (2024), maybe it could be more interesting than well-known structural probe (2019). Maybe, both descriptions should be included in the main text of the paper.\n\nLine 139: some brief detail of Mistral-7B-v0.1 LLM should be included (total number of layers, architecture, language, pretraining corpus, etc.).\n\nLine 143: how did you set the hyperparameters vaules of for the training of the probes?\n\nLine 157: You should introduce and  present the surprisal concept, also specifying how it is exploited in your evaluation.\n\nLine 161: Why do you adopt Classifiers to predict output for ground truth positive instances? This point is not very clear to me and more details and a deeper explaination is required in the paper.\n\nResults: some sentences' examples from your datasets should be used to better explain and support the results obtained (used for a sort of qualitative explaination), allowing to better understand the results of your study. Why don't you include them?", "extra_scores": {}}]

Author rebuttal:

Meta-review: The reviewers largely positively evaluate this work and its experiments testing the behavior of structural probes across various attributes of syntactic dependencies like parse depth and linear distance. The pros of this work are in running controlled studies of the behavior of a well-known language model analysis tool on modern language models. Reviewers generally approved of the paper’s clarity. For cons, see below.

I won’t override the unanimous opinions of the reviewers, so I’ll recommend acceptance.  I want to provide some editorial thoughts for the authors, though, since I’ve spent a good amount of time thinking about structural probe research.

After reading the paper myself, I am left somewhat confused – does the decay of the structural probe’s parse accuracy show that the probe is failing, as claimed by the authors? Or is the probe accurately reflecting the underlying language model’s failure to parse? A probe’s parsing failure is not necessarily a failure to give insight into the model, if the model is just failing to build the tree correctly. Arguably, modern models almost never make syntactic errors, so maybe the premise of this work is that a good probe should reflect perfect parsing, and thus not decay in accuracy. This is discussed by Reviewer qy93, but unlike reviewer qy93, I do not think this concern is mitigated by evaluating more models. For, e.g., the 3-depth center embedding synthetic data, with no semantic cues, is really confusing for humans and machines both (also noted by Reviewer tL7W), so it’s plausible to me that models really do just fail to (implicitly) parse them correctly. At a higher level, I’m just not sure what the successes of the probes are (success shows up only in the title) and whether the failures of parsing accuracy are probing failures, or language model failures that are correctly captured by the probes. One way to study this would be to compare probe results to behavior, e.g., when the probe gives an incorrect parse, does the model’s behavior show corresponding inaccuracies? If so, the probe is correctly reflecting the model, otherwise it’s a failure.

This is a nit, but I’m confused that the authors stated “red ideas sleep furiously” as an “infamous” Chomsky quote – the cited work (Chomsky, 1957) has “colorless green ideas sleep furiously,” so the authors should use that quote, if only to be faithful to the cited work.
Decision: Accept