Probing via Prompting (2022)

Abstract: Probing is a popular approach to understand what linguistic information is contained in the representations of pre-trained language models. However, the mechanism of selecting the probe model has recently been subject to intense debate, as it is not clear if the probes are merely extracting information or modelling the linguistic property themselves. To address this challenge, this paper introduces a novel model-free approach to probing via prompting, which formulates probing as a prompting task. We conduct experiments on five probing tasks and show that PP is comparable or better at extracting information than diagnostic probes while learning much less on its own. We further combine the probing via prompting approach with pruning to analyze where the model stores the linguistic information in its architecture. Finally, we apply the probing via prompting approach to examine the usefulness of a linguistic property for pre-training by removing the heads that are essential to it and evaluating the resulting model’s performance on language modeling.

## introduction pre-trained language models such as bert @xcite , gpt-2 @xcite , and gpt-3 @xcite have increased the performance of data-driven natural language processing (nlp) models on a wide variety of tasks. due to their strong performance on many language-based tasks that require some linguistic understanding, it is natural to hypothesize that the models must implicitly encode some linguistic knowledge. one avenue of research that attempts to uncover the linguistic knowledge encoded in these models is called probing @xcite @xcite . a common form of probing is diagnostic probing. under this approach a classifier is trained on top of a pre-trained language model to perform a target linguistic task, which is closely related to the linguistic property in question. the predictive accuracy of the classifier is then taken as an indicator of how much knowledge about the target linguistic property is encoded in the language model representations. however, diagnostic probing has its limitations. an inherent challenge in the endeavor is discerning what is encoded in the pre-trained representations from what is learned by the probe itself @xcite @xcite . the probe could, in principle, learn the task on top of random representations. while the probe is trained to extract linguistic properties from the representations, there is no simple way to restrain the probe from learning the task on its own during training. previous research tackles the challenge using random model baselines @xcite and control tasks @xcite . @xcite try to create a more selective probe using pruning. in this work, we address the above limitation with a novel probing framework that we call probing via prompting (pp). drawing inspiration from recent work on prompting @xcite , we reformat a suite of probing tasks into question-answer pairs and instruct the model to answer the questions with a prefix @xcite . in effect, prompting acts as a model-free probe. thus, the use of prompting instead of a diagnostic probe allows us to work around the dilemma of teasing apart what the representations contain versus what the probe learns. in the empirical part of the paper, we conduct experiments on five linguistic tasks and show that all these properties are indeed encoded in the popular pre-trained language model, gpt-2. we then use a prefix to instruct the model to predict the label ("date"). figure 1 : illustration of different probing paradigms. here, we show an example of named entity labeling: classifying a given entity span into pre-defined categories. furthermore, we show that probing via prompting appears to lead to insightful probing results. in the language of @xcite , pp obtains high selectivity, i.e., the results show high accuracy on the target task, but, as expected, pp does not work well with random representations. these results suggest that pp learns little information on its own and the extracted linguistic properties are indeed encoded in the model. at a high level, this work takes the position that model-free probing methods such as pp are useful for accurately and faithfully locating and identifying the linguistic properties embedded in these representations and helping us understand how neural language models process text. in contrast to diagnostic probes, which require designing random model baselines and control tasks to control for the learning ability of the probe model, model-free probes like pp are less capable of learning the linguistic task themselves, and thus can naturally be more selective than diagnostic probes. ## probing via prompting we now introduce our probing via prompting framework (pp), illustrated in fig. 1b . ## experiments we empirically benchmark our pruning method against several previously proposed works. ## analysis now that we have demonstrated the basic utility of pp, we attempt to use our methodology to determine where in the representations the information is encoded. thus, following @xcite , we search for a subnetwork that optimizes the performance on the task of interest and analyze the resulting subnetwork. since it has been shown that different attention heads in the transformer capture different linguistic phenomena @xcite , we prune attention heads instead of individual weights. concretely, we use differentiable subset pruning (dsp) proposed by @xcite , which allows us to directly control the number of heads to keep. pruning is performed jointly with prefix tuning. essential and non-essential heads. we call the k heads that survive pruning essential heads for the task, and those that are removed non-essential heads. with n pp (ℓ), we denote the distribution that is proportional to the number of essential heads in each layer ℓ under the pp scheme. we define n lr (ℓ) analogously. ## related work bowman, 2018), diagnostic classification @xcite @xcite , and others @xcite @xcite . however, the interpretation of probing results has been called into question @xcite : do the representations encode the linguistic information or does the probe learn the task on its own? a commonly held belief @xcite @xcite ) is a simple model (e.g. a linear one) is not capable of learning the task itself and thus is preferred, but @xcite argues that one should always choose the best possible probe because it reveals the most linguistic information present in the representations. @xcite ; voita and titov (2020) explicitly model the trade-off between accuracy and model complexity. @xcite propose to search for a subnetwork within the model rather than train an auxiliary model, but a task-specific classification head is still required. prompting. the deep contextual word representations are typically derived from either an lm @xcite or a masked lm @xcite . a common use of these pre-trained language models is finetuning. however, an alternative approach called prompting has recently gained much popularity. instead of accommodating a language model for downstream tasks, prompting adapts downstream tasks to be more like lm with the aid of a prompt. in this way, the pre-trained model can be used to perform few-shot or even zero-shot learning @xcite @xcite . most papers construct templates with blanks for the model to fill. for example, lama @xcite creates cloze-style templates to probe knowledge; @xcite put task descriptions and examples in the prefix and then the model performs various tasks by finishing the sentence; @xcite enumerate every possible text span in a sentence, create a template for each of them, and fine-tune the model to perform named entity recognition (ner). however, creating such templates requires a large amount of time and human expertise, and does not necessarily do well on the task of interest. therefore, many researchers focus on generating prompts automatically @xcite @xcite . the prompts must not consist of natural language, but can also be continuous vectors @xcite @xcite . we refer the reader to @xcite for a more thorough survey about prompting. in this work, we apply the method of @xcite to learn continuous prompts to instruct the model to predict linguistic structure. pruning. neural network pruning aims to reduce the model size and increase inference speed by removing redundant network components, such as individual parameters @xcite @xcite , convolutional channels @xcite @xcite , and attention heads @xcite @xcite . in addition to model compression, pruning has also been used for analysis @xcite analyze the linguistic roles the unpruned heads play; @xcite look at the location of unpruned weights. similarly, we examine the network components that survive pruning, but we apply head pruning @xcite instead of weight pruning @xcite since attention heads are believed to be more linguistically interpretable than weights. ## conclusion with the growing popularity of probing, there have been increasing concerns that high probing performance on a linguistic property cannot be attributed to representations encoding the property, since the probe can learn the probing task on its own. in this work, we propose a novel probing via prompting method, which drastically reduces the probe's ability to learn and, thus, mitigates this problem. we conduct experiments on five linguistic tasks and show that these properties are indeed encoded in one popular pre-trained language model, gpt-2. however, they might not be encoded in a natural progression in the model as previously believed. for further study, we hope to develop tools that can more accurately and faithfully locate and identify the linguistic properties embedded in the model and help us understand the way neural language models process text.