Do Syntactic Probes Probe Syntax? Experiments with Jabberwocky Probing (2021)

Abstract: Analysing whether neural language models encode linguistic information has become popular in NLP. One method of doing so, which is frequently cited to support the claim that models like BERT encode syntax, is called probing; probes are small supervised models trained to extract linguistic information from another model’s output. If a probe is able to predict a particular structure, it is argued that the model whose output it is trained on must have implicitly learnt to encode it. However, drawing a generalisation about a model’s linguistic knowledge about a specific phenomena based on what a probe is able to learn may be problematic: in this work, we show that semantic cues in training data means that syntactic probes do not properly isolate syntax. We generate a new corpus of semantically nonsensical but syntactically well-formed Jabberwocky sentences, which we use to evaluate two probes trained on normal data. We train the probes on several popular language models (BERT, GPT-2, and RoBERTa), and find that in all settings they perform worse when evaluated on these data, for one probe by an average of 15.4 UUAS points absolute. Although in most cases they still outperform the baselines, their lead is reduced substantially, e.g. by 53% in the case of BERT for one probe. This begs the question: what empirical scores constitute knowing syntax?

## 'twas brillig, and the slithy toves recently, unsupervised language models like bert @xcite have become popular within natural language processing (nlp). these pre-trained sentence encoders, known affectionately as bertoids @xcite , have pushed forward the state of the art in many nlp tasks. given their impressive performance, a natural question to ask is whether models like these implicitly learn to encode linguistic structures, such as part-of-speech tags or dependency trees. there are two strains of research that investigate this question. on one hand, stimuli-analysis compares the relative probabilities a language model assigns to words which could fill a gap in a clozestyle task. this allows the experimenter to test whether neural models do well at capturing specific linguistic phenomena, such as subject-verb agreement @xcite or negative-polarity item licensing @xcite . another strain of research directly analyses the neural network's representations; this is called probing. probes are supervised models which attempt to predict a target linguistic structure using a model's representation as its input (e.g. @xcite @xcite ; if the probe is able to perform the task well, then it is argued that the model has learnt to implicitly encode that structure in its representation. 1 work from this inchoate probing literature is frequently cited to support the claim that models like bert encode a large amount of syntactic knowledge. for instance, consider the two excerpts below demonstrating how a couple of syntactic probing papers have been interpreted: 2 further work has found impressive degrees of syntactic structure in transformer encodings @xcite our position in this paper is simple: we argue that the literature on syntactic probing is methodologically flawed, owing to a conflation of syntax with semantics. we contend that no existing probing work has rigorously tested whether bert encodes syntax, and a fortiori this literature should not be used to support this claim. to investigate whether syntactic probes actually probe syntax (or instead rely on semantics), we train two probes ( §4) on the output representations produced by three pre-trained encoders on normal sentences-bert @xcite , gpt-2 @xcite , and roberta @xcite . we then evaluate these probes on a novel corpus of syntactically well-formed sentences made up of pseudowords ( §3), and find that their performance drops substantially in this setting: on one probe, the average bertoid uuas is reduced by 15.4 points, and on the other the relative advantage that bert exhibits over a baseline drops by 53%. this suggests that the probes are leveraging statistical patterns in distributional semantics to aide them in the search for syntax. according to one of the probes, gpt-2 falls behind a simple baseline, but in some cases the leads remains substantial, e.g. 20.4 uuas points in the case of bert. we use these results not to draw conclusions about any bertoids' syntactic knowledge, but instead to urge caution when drawing conclusions from probing results. in our discussion, we contend that evaluating bertoids' syntactic knowledge requires more nuanced experimentation than simply training a syntactic probe as if it were a parser @xcite , and call for the separation of syntax and semantics in future probing work. ## syntax and semantics when investigating whether a particular model encodes syntax, those who have opted for stimulianalysis have been careful to isolate syntactic phenomena from semantics @xcite @xcite , but the same cannot be said of most syntactic probing work, which conflates the two. to see how the two can be separated, consider the famous utterance of @xcite : ## generating jabberwocky sentences to tease apart syntax and semantics when evaluating probes, we construct a new evaluation corpus of syntactically valid english jabberwocky sentences, so called after @xcite who wrote verse consisting in large part of pseudowords (see app. a). in written language, a pseudoword is a sequence of letters which looks like a valid word in a particular language (usually determined by acceptability judgments), but which carries with it no lexical meaning. for our jabberwocky corpus, we make use of the arc nonword database, which contains 358, 534 monosyllabic english pseudowords @xcite . we use a subset of these which were filtered out then manually validated for high plausibility by @xcite . we conjugate each of these words using hand-written rules assuming they obey the standard english morphology and graphotactics. this results in 1361 word types-a total of 2377 varieties when we annotate these regular forms with several possible fine-grained part-of-speech realisations. to build sentences, we take the test portion of the english ewt universal dependency (ud; @xcite treebank and substitute words (randomly) with our pseudowords whenever we have one available with matching fine-grained part-ofspeech annotation. 4 our method closely resembles @xcite , except they do so to analyse parsers in place of syntactic probes. an example of one of our jabberwocky sentences is shown in figure 2 , along with its unlabeled undirected parse (used by the probes) which is taken from the vanilla sentence's annotation in the treebank. ## two syntactic probes a syntactic probe is a supervised model trained to predict the syntactic structure of a sentence using representations produced by another model. the main distinction between syntactic probes and dependency parsers is one of researcher intentprobes are not meant to best the state of the art, but are a visualisation method @xcite . as such, probes are typically minimally parameterised so they do not "dig" for information (but see @xcite . if a syntactic probe performs well using a model's representations, it is argued that that model implicitly encodes syntax. here we briefly introduce two syntactic probes, each designed to learn the syntactic distance between a pair of words in a sentence, which is the number of steps between them in an undirected parse tree (example in figure 2 ). @xcite first introduced syntactic distance, and propose the structural probe as a means of identifying it; it takes a pair of embeddings and learns to predict the syntactic distance between them. an alternative to the structural probe which learns parameters for the same function is a structured perceptron dependency parser, originally introduced in @xcite , and first applied to probing in hall maudslay et al. (2020). here we call this the perceptron probe. rather than learning syntactic distance directly, the perceptron probe instead learns to predict syntactic distances such that the minimum spanning tree that results from a sentence's predictions matches the gold standard parse tree. the difference between these probes is subtle, but they optimise for different metrics-this is reflected in our evaluation in §5. we train the probes on normal uds, then evaluate them on jabberwocky sentences; if the probes are really learning to extract syntax, they should perform just as well in the jabberwocky setting. ## o frabjous day! callooh! callay! in this work, we trained two syntactic probes on a variety of bertoids, then evaluated them using jabberwocky sentences, and showed that performance dropped substantially in this setting. this suggests that previous results from the probing literature may have overestimated bert's syntactic abilities. however, in this context, we do not use the results to make any claims about bert; we contend that to make such a claim one ought train the probes on jabberwocky sentences, which would require more psuedowords than we had available. instead, we advocate for the separation of syntax and semantics in probing. future work could explore the development of artificial treebanks for use specifically for training syntactic probes, which minimise for any confounding statistical biases in the data. we make our jabberwocky evaluation data and code publicly available at https: //github.com/rowanhm/jabberwocky-probing. 'twas brillig, and the slithy toves did gyre and gimble in the wabe; all mimsy were the borogoves, and the mome raths outgrabe.