Prompting Language Models for Linguistic Structure (2023)

Abstract: Although pretrained language models (PLMs) can be prompted to perform a wide range of language tasks, it remains an open question how much this ability comes from generalizable linguistic understanding versus surface-level lexical patterns. To test this, we present a structured prompting approach for linguistic structured prediction tasks, allowing us to perform zero- and few-shot sequence tagging with autoregressive PLMs. We evaluate this approach on part-of-speech tagging, named entity recognition, and sentence chunking, demonstrating strong few-shot performance in all cases. We also find that while PLMs contain significant prior knowledge of task labels due to task leakage into the pretraining corpus, structured prompting can also retrieve linguistic structure with arbitrary labels. These findings indicate that the in-context learning ability and linguistic knowledge of PLMs generalizes beyond memorization of their training data.

## introduction the rapid increase in the scale of pretrained language models (plms) has led to a new paradigm of nlp modeling: in-context learning, or prompting (e.g., @xcite . in this setting, the model is used to perform a task directly via the predictions of the lm head without additional finetuning on the target task, often with a few demonstrations of the desired behavior provided within the input. this setup has led to impressive few-shot performance on various tasks ranging from classification to summarization and generation @xcite . due to their broad success on tasks requiring language understanding, we hypothesize that these models also contain significant linguistic knowledge. however, we are not aware of existing prompting methods that can directly test this hypothesis on autoregressive plms. behavioral analysis of plms @xcite uses meth-figure 1 : sequence tagging via structured prompting. each predicted label is appended to the context along with the next word to iteratively tag the full sentence. we propose a new approach, structured prompting, that iteratively prompts autoregressive plms to probe for word-and span-level linguistics framed as sequence tagging tasks (section 2). at timestep t, a label for the t-th word in the sequence is decoded from the lm; the model prediction is then fed back into the model along with the next word to progress to timestep t + 1. we evaluate our approach on three sequence tagging tasks: pos tagging, sentence chunking, and ner. our experiments show that plms can perform effective fewshot sequence tagging in the structured prompting setup, and that performance increases with the demonstration set size and model size, consistent with other prompting methods (section 4). we further analyze structured prompting by ex-amining how the model generalizes to various representations for labels (section 5) as well as by analyzing the presence of task data in the pretraining corpus and how this affects model performance (section 6). these experiments show that structured prompting can recover linguistic information from the model without using standard task labels, indicating that plms contain this knowledge in a general manner beyond memorization of the task from pretraining data. interestingly, while plms perform best with meaningful labels (such as original task labels or full class names in english), the model can also in-context learn from arbitrary labels. additionally, the model exhibits strong prior knowledge of the task labels' mapping onto the underlying classes, likely due to the prevalence of task data in the pretraining corpus. the contributions of this work are therefore threefold: (1) we introduce a new paradigm, structured prompting, that probes plms for sequence knowledge without further training, (2) we find that this approach recovers linguistic structure from plms in a few-shot manner, and (3) we present an analysis to quantify the effect of label form and pretraining data on in-context learning performance. overall, our findings provide insight into both the linguistic generalizations learned by plms and how in-context learning works in general. ## structured prompting of pretrained language models we propose a sequential method for performing sequence tagging with plms via in-context learning, which we refer to as structured prompting (figure 1 ). the model is given k (context, tagged sequence) pairs as the task demonstration and the example sentence to be labeled. the model then iteratively tags the words in the example with constrained decoding over a fixed set of labels. more specifically, given a set of labels l and an input sequence c containing k demonstration pairs as well as the full text of the example sentence @xmath0, ..., s n , at each time step t the language model m encodes [c; s t ] and labels s t with @xmath1, s t ). we then update the input sequence by appending the current word s t and the predicted label lt to the end of c. multi-token labels are scored with the average log-likelihood over all tokens p m (ℓ|c) = 1 |ℓ| |ℓ| @xmath2, y 0 , ..., y i-1 ), where y j is the jth subword token in ℓ. this approach to in-context learning tags an en-tire sequence with a single pass over the context. it also allows the model to condition on past predictions while labeling the current word. as we demonstrate in section 4, these features allow us to apply large autoregressive language models to a broad class of core nlp tasks in a few-shot manner. ## structured prompting results we measure the performance of structured prompting on three sequence tagging tasks. this evaluation aims to (1) validate that structured prompting follows prior prompting setups in terms of model and k-shot scaling trends and (2) investigate the extent to which the approach extracts these structures from the model. we then quantify the types of errors made with structured prompting. ## when does structured prompting work? we now investigate how structured prompting surfaces linguistic structure from plms, using the behavior of gpt-neox on pos tagging and ner as a case study. we find that (1) in some cases, the model generalizes to labels not seen in the demonstration, and (2) the label form has a large effect on performance. specifically, the model can learn in context when arbitrary labels represent classes but will ignore label mappings in the demonstration that contradict its prior task knowledge. ## sources of linguistic knowledge in pretraining corpus the results in section 5 demonstrate that the choice of label form can greatly affect structured prompting performance and implies that the model contains prior task knowledge. we analyze contexts in which the labels for pos tagging and ner appear in the pile @xcite to better understand what, if any, task information gpt-neox learns from pretraining. our analysis shows that task information occurs in the pretraining data, both as labeled examples (section 6.1) and in other related contexts (section 6.2). however, we find no evidence of test data leakage. given these findings, we evaluate the model in a new setting that substitutes an english description of each class (e.g., "adjective", "person") for the label in order to control for label leakage while still providing meaningful labels (section 6.3). ## related work prompting plms for sequence information recent work has applied various prompting approaches to sequence tagging tasks, primarily focusing on ner @xcite . however, these approaches also require further training, most often by learning new prompt embeddings for the task @xcite @xcite . other work has finetuned language models to apply them to sequence tagging tasks @xcite . in contrast, our approach requires no additional parameters to be learned. more similar to our work is the sequence tagging method in @xcite , though their approach prompts the model separately for each word in the sentence. additionally, similar approaches to prompting have been proposed for other tasks; these methods decompose a target task and repeatedly prompt the model on subtasks, building on the model's outputs to generate the final prediction @xcite . however, these approaches solve a different subset of nlp tasks and use the outputs from the intermediate prompting steps differently (i.e., by conditioning on them in future prompting steps, whereas in structured prompting each output is a predicted label). probing pretrained models there is extensive work on probing models for their underlying knowledge @xcite @xcite . the approach has become particularly popular for analyzing masked plms (e.g., @xcite , with behavioral probes (e.g. @xcite in particular using the lm setup to elicit knowledge from the model. however, prompting autoregressive plms @xcite @xcite , though technically similar to behavioral probing, is usually not framed as probing the underlying model for knowledge. some exceptions are @xcite , which uses prompting techniques to probe the lm for knowledge base relations, and @xcite , which replaces diagnostic probes with trained prompt embeddings for model analysis. we extend this framing by applying structured prompting as a behavioral probe for linguistic structure. ## conclusion we propose structured prompting, a general paradigm for sequence tagging with autoregressive plms. our experiments show structured prompting performs well on three few-shot sequence tagging tasks. further analysis shows that (1) the approach can elicit linguistic structure in many settings, including when the labels are unrelated to the task, and (2) while labeled task data is present in the pretraining corpora, using informative labels not found in task data gives similar performance to using the task labels. these findings indicate that the model's knowledge of linguistic structure is more general than the memorization of the task data. more generally, our approach provides a method to probe plms for sequence knowledge without training new or existing parameters.