Does BERT Rediscover a Classical NLP Pipeline? (2022)

Abstract: Does BERT store surface knowledge in its bottom layers, syntactic knowledge in its middle layers, and semantic knowledge in its upper layers? In re-examining Jawahar et al. (2019) and Tenney et al.’s (2019a) probes into the structure of BERT, we have found that the pipeline-like separation that they asserted lacks conclusive empirical support. BERT’s structure is, however, linguistically founded, although perhaps in a way that is more nuanced than can be explained by layers alone. We introduce a novel probe, called GridLoc, through which we can also take into account token positions, training rounds, and random seeds. Using GridLoc, we are able to detect other, stronger regularities that suggest that pseudo-cognitive appeals to layer depth may not be the preferable mode of explanation for BERT’s inner workings.

## introduction at the same time, there is nevertheless scepticism about the premises of probing itself. as hewitt and liang (2019) pondered: "when a probe achieves high accuracy on a linguistic task using a representation, can we conclude that the representation encodes linguistic structure, or has the probe just learned the task?" furthermore, pimentel et al. (2020) "cast doubt on whether probing makes sense as a scientific endeavour," because from an information-theoretic perspective, bert cannot introduce new linguistic information by processing the input sequence. in our view, this debate does not need to hinder the endeavour of uncovering structure within bert. regardless of stance, all parties agree with the existence and importance of different levels of information within bert itself or its embeddings. there is, however, room for improvement in the investigative methods of both @xcite and @xcite (j&t), which seem to have been limited to observational confirmations of what they sought to find. our own exploratory analysis has revealed that bert is indeed linguistically founded, although not in a way that suggests a classical pipeline architecture, other than what factors through our own functional understanding of nlp's terminology and subtasks. in addition to examining bert layers, as j&t did, we have also examined bert's structure through the lenses of the choice of random seed, training iterations, and, most importantly, token position. we also present several statistical tests of j&t's own conclusions. we propose gridloc, 1 a self-attention-based probing method that can probe across all of these aforementioned dimensions. using this novel probing approach and our statistical testing suite, a much more comprehensive picture of the structure of bert arises. specifically: • bert's task-specific features appear in different token positions in an idiosyncratic but consistent pattern for each task; • the attending task-specific features exhibit variance across different sentences, different training durations, and different random seeds; • probe results for tree depth, in particular, show an anomalous distribution of linguistic evidence when taking both layers and token position into consideration. ## attributing the cnlp to bert that there is or was a classical or traditional nlp pipeline is a rather naïve view to take of the history of natural language processing. while there were discussions of independent stages of tokenlevel, syntactic and semantic processing already in the earliest work on machine translation, the pioneers who first engaged with these stages of analysis were of the considered opinion that careful restraint had to be exercised at every level of analysis so that as much of the inherent ambiguity of linguistic input could be carried forward as possible, in the interests of both efficiency and accuracy @xcite . it was not until the late 1970s and early 1980s that an excessive reliance on classical logic by the nlp technologists of the grammatico-logical movement, together with contemporaneous psycholinguistics and cognitive science research pointing to a modularity of linguistic structure in human sentence processing @xcite @xcite , led to a pipeline-based view that, almost immediately, was apologized for as a convenient abstraction, "incremental" @xcite , "highly flexible" and "even opportunistic" (marslen-wilson and tyler, 1987) (see @xcite for a more detailed discussion). while the smoother numerical allocations across tasks in bertology work, and the distributional graphs drawn by @xcite in particular, may at first seem to be commensurate with or even suggest these more nuanced views of a language processing pipeline, the haze of smoke surrounding the visualization of probing evidence makes it extremely difficult to draw precise conclusions from these figures, as we will argue below. nevertheless, probe methods in general @xcite @xcite @xcite are to be credited as one of the few means that we have of approaching the seemingly impossible task of interpreting the neural feature representations within bert. generally, a probe can be performance-based or attention-based. ## gridloc to leverage all of these degrees of freedom, we present here a novel probing method called grid-loc. figure 3 (a) layer attention weights of the same probe of the first three bigram shift test-split sentences. the layer attention weight distributions differ widely. seed: 0, epoch: 7. seed: 1, epoch: 8. (b) distribution of the layer with the highest attention score over the bigram shift test-set sentences for two probing runs with different random seeds. both probes are generated at their top performing (by validation) epochs. distributions can exhibit substantial variance (left: σ = 2.16, right: σ = 0.78). for the run with seed 0 (left), there is also a spike in sentences at the 12th layer that is not observed in the run with seed 1 (right). token position to understand the task-specific feature distribution across token for each layer, we exploit the aforementioned self-attention pooling @xcite method to learn a token attention: layer then, we learn a sentence-specific layer attention: from the attended contextual embedding of the entire sequence. finally, we can train the probe classifier on the fully attended representation of the input sequence: ## experimental setup we used all 10 tasks in senteval @xcite as described in §2.1. to be consistent with j&t's results, we conducted our experiment using the uncased bert-base model and @xcite hyperparameters. 3 ## conclusion did bert rediscover an nlp pipeline? not in a naïve, architectural sense. gridloc reveals a structure in bert that is more intricate than a flowchart of a pipeline could accurately portray, and yet it does seem to be linguistically founded. we find that probing results regarding bert layers are unstable, diverging across sentence input, random seeds and the early iterations of training. the distribution of linguistically motivated task features along token positions, on the other hand, is relatively more stable. moreover, gridloc's results on tree depth provide preliminary evidence of poss being used to conduct novel but linguistically generalizable inference concerning a derivative syntactic phenomenon.