Prompting Language Models for Linguistic Structure
Year: 2023

Abstract:
Although pretrained language models (PLMs) can be prompted to perform a wide range of language tasks, it remains an open question how much this ability comes from generalizable linguistic understanding versus surface-level lexical patterns. To test this, we present a structured prompting approach for linguistic structured prediction tasks, allowing us to perform zeroand few-shot sequence tagging with autoregressive PLMs. We evaluate this approach on part-of-speech tagging, named entity recognition, and sentence chunking, demonstrating strong few-shot performance in all cases. We also find that while PLMs contain significant prior knowledge of task labels due to task leakage into the pretraining corpus, structured prompting can also retrieve linguistic structure with arbitrary labels. These findings indicate that the in-context learning ability and linguistic knowledge of PLMs generalizes beyond memorization of their training data.

heading: 1 Introduction
text: The rapid increase in the scale of pretrained language models (PLMs) has led to a new paradigm of NLP modeling: in-context learning, or prompting (e.g., Brown et al., 2020; Raffel et al., 2020). In this setting, the model is used to perform a task directly via the predictions of the LM head without additional finetuning on the target task, often with a few demonstrations of the desired behavior provided within the input. This setup has led to impressive few-shot performance on various tasks ranging from classification to summarization and generation (Liu et al., 2021a).
Due to their broad success on tasks requiring language understanding, we hypothesize that these models also contain significant linguistic knowledge. However, we are not aware of existing prompting methods that can directly test this hypothesis on autoregressive PLMs. Behavioral analysis of PLMs (Belinkov et al., 2020) uses meth-
ods similar to prompting to measure knowledge stored in language models (Gulordava et al., 2018; Petroni et al., 2019), but this technique is difficult to generalize to tasks that predict more complex structures. Additionally, current approaches for applying PLMs to linguistic structured prediction tasks finetune on the downstream task (e.g., Ma et al., 2022), which confounds measuring underlying model knowledge.
We propose a new approach, structured prompting, that iteratively prompts autoregressive PLMs to probe for word- and span-level linguistics framed as sequence tagging tasks (Section 2). At timestep t, a label for the t-th word in the sequence is decoded from the LM; the model prediction is then fed back into the model along with the next word to progress to timestep t + 1. We evaluate our approach on three sequence tagging tasks: POS tagging, sentence chunking, and NER. Our experiments show that PLMs can perform effective fewshot sequence tagging in the structured prompting setup, and that performance increases with the demonstration set size and model size, consistent with other prompting methods (Section 4).
We further analyze structured prompting by ex-
6649
amining how the model generalizes to various representations for labels (Section 5) as well as by analyzing the presence of task data in the pretraining corpus and how this affects model performance (Section 6). These experiments show that structured prompting can recover linguistic information from the model without using standard task labels, indicating that PLMs contain this knowledge in a general manner beyond memorization of the task from pretraining data. Interestingly, while PLMs perform best with meaningful labels (such as original task labels or full class names in English), the model can also in-context learn from arbitrary labels. Additionally, the model exhibits strong prior knowledge of the task labels’ mapping onto the underlying classes, likely due to the prevalence of task data in the pretraining corpus.
The contributions of this work are therefore threefold: (1) we introduce a new paradigm, structured prompting, that probes PLMs for sequence knowledge without further training, (2) we find that this approach recovers linguistic structure from PLMs in a few-shot manner, and (3) we present an analysis to quantify the effect of label form and pretraining data on in-context learning performance. Overall, our findings provide insight into both the linguistic generalizations learned by PLMs and how in-context learning works in general.

heading: 2 Structured Prompting of Pretrained Language Models
text: We propose a sequential method for performing sequence tagging with PLMs via in-context learning, which we refer to as structured prompting (Figure 1). The model is given k (context, tagged sequence) pairs as the task demonstration and the example sentence to be labeled. The model then iteratively tags the words in the example with constrained decoding over a fixed set of labels.
More specifically, given a set of labels L and an input sequence c containing k demonstration pairs as well as the full text of the example sentence S = s0, ..., sn, at each time step t the language model M encodes [c; st] and labels st with ℓ̂t = argmax
ℓ∈L PM (ℓ|c, st). We then update the input sequence by appending the current word st and the predicted label ℓ̂t to the end of c. Multi-token labels are scored with the average log-likelihood over all tokens PM (ℓ|c) = 1|ℓ| ∑|ℓ| i=0 PM (yi|c, y0, ..., yi−1), where yj is the jth subword token in ℓ. This approach to in-context learning tags an en-
tire sequence with a single pass over the context. It also allows the model to condition on past predictions while labeling the current word. As we demonstrate in Section 4, these features allow us to apply large autoregressive language models to a broad class of core NLP tasks in a few-shot manner.

heading: 3 Experimental Setup

heading: 3.1 Prompt Formatting
text: We use a lightweight prompt format with limited natural language guidance about the task provided to the model as shown in Figure 1; the letters “C” and “T” in the figure represent the inputs “Context” and “Tagged” respectively. For each task, we represent each tag with the token or sequence of tokens corresponding to the surface form of the label provided by the dataset.
In general, our preliminary experiments with varied prompt formats had little effect on performance. Specifically, performance was stable across the choice of delimiter and other minor formatting differences. However, we note that including the word in the “Tagged” sequence is important; on GPT-J, performance degrades by 84% on POS and 79% on NER when decoding the label sequence without repeating the word (i.e., “Tagged: DET NOUN...”).

heading: 3.2 Sequence Tagging Tasks
text: We consider the following English tasks framed as sequence tagging problems in evaluating the proposed structured prompting method. For tasks involving tagging spans of text, we label each token in the span using the BIO label format: given a span of m tokens labeled ℓ, the first token is labeled as the beginning of the span with “B-ℓ”, the remaining m-1 tokens are labeled as inside the span with “I-ℓ”, and tokens not included in the span are labeled as outside the span or “O”).
Part-of-Speech (POS) Tagging We evaluate POS tagging performance on English Universal Dependencies (UD) with the UPOS tagset (Nivre et al., 2020). Specifically, we use the treebank annotated on the GUM corpus (Zeldes, 2017).
Sentence Chunking Chunking, or shallow parsing, partitions the words in a sentence into nonoverlapping spans of syntactic meaning. We evaluate PLMs on chunking with the CONLL2000 dataset from Sang and Buchholz (2000), which frames chunking as a BIO tagging task.
Named Entity Recognition (NER) We evaluate the ability of structured prompting to extract named entities from PLMs with NER. This is measured as a BIO tagging task on the CONLL2003 dataset (Sang and De Meulder, 2003).

heading: 3.3 Models
text: We report performance on seven language models, ranging from 125 million to 175 billion parameters.
GPT-Neo This set of PLMs contains models trained on the Pile (Gao et al., 2020) that from 125 million to 2.7 billion parameters (Gao et al., 2020), 6.7 billion parameters (Wang and Komatsuzaki, 2021), and 20 billion parameters (Black et al., 2022). We use the GPT-Neo models available through Huggingface (Wolf et al., 2019).
GPT-3 We also perform structured prompting with the GPT-3 models (Brown et al., 2020) via the OpenAI API. We use the base GPT-Curie (∼6B parameters) and GPT-Davinci (∼175B parameters) models that have undergone no additional instruction finetuning on POS tagging. Due to the cost of running these models through the API, we generate the GPT-Davinci output with unconstrained top-1 sampling rather than the constrained decoding setup described in Section 2.
In preliminary experiments, we also tested structured prompting on several OPT models (Zhang et al., 2022). We found their performance was significantly worse and did not scale with model size (up to 66B parameters) on POS tagging and NER. We leave a more thorough examination of this behavior discrepancy for future work.

heading: 3.4 Additional Experimental Details
text: We report the mean and standard error across m runs for each experiment. For each of these runs, k demonstrations are sampled from the training dataset at random, with the condition that the k demonstrations cover the label space of the task if possible. We use k = 10 sentences as demonstrations and perform m = 5 runs per experiment unless otherwise stated.
Each model is evaluated on 1000 examples randomly sampled from the task test set (see Appendix A.1 for a discussion on how this choice affects performance estimates). The evaluation subset is held fixed across all five runs, and the evaluation data and selection of demonstrations for each run are fixed across models for each task.
To obtain the tag sequence for each example, we greedily take the top-1 label (with the highest log likelihood) for each word. We also enforce hard constraints for the span-labeling tasks involving BIO tagging (chunking, NER) to ensure a valid BIO tag sequence (e.g., I-X tags can only follow a previous B-X or I-X tag). Empirically, we find that enforcing BIO constraints makes little difference in the method’s overall performance; however, we use them as they ensure valid output sequences. Appendix A.2 compares model performance with and without BIO constraints.

heading: 4 Structured Prompting Results
text: We measure the performance of structured prompting on three sequence tagging tasks. This evaluation aims to (1) validate that structured prompting follows prior prompting setups in terms of model and k-shot scaling trends and (2) investigate the extent to which the approach extracts these struc-
tures from the model. We then quantify the types of errors made with structured prompting.

heading: 4.1 Overall Results
text: Figure 2 presents the results of our primary structured prompting evaluation. We consider the performance of GPT-NeoX (Black et al., 2022) compared to task baselines: overall majority, in which each word is labeled with the most frequent tag in the training set, and per-word majority, where each word is labeled with the tag it most commonly appeared within the training data (left panel).1 All baselines are calculated on the full training set and so use more labeled data than the PLM; the perword majority is a particularly strong baseline as words frequently occur with the same tag.
Structured prompting performs effective fewshot sequence tagging We find that GPT-NeoX significantly outperforms each baseline on POS tagging and NER, and the model slightly underperforms the per-word majority baseline on sentence chunking by 4.2 points. Overall, the approach performs worse for the BIO span-labeling tasks than for word-level POS tagging. We hypothesize that the former tasks are more complex, as they require the model to determine spans and more detailed linguistic knowledge.
Structured prompting scales with model and demonstration size We observe that the performance of structured prompting improves with scale across GPT-Neo models (center panel). Model performance also improves with additional demonstrations (right panel); both of these trends are con-
1For BIO tasks, the majority labels correspond to “O” (NER) and “I-NP” (chunking). The CONLL evaluation script only scores labeled spans, giving an overall majority F1 of 0.
sistent with prior prompting results (e.g., Black et al., 2022). However, the extent to which additional demonstrations help varies: NER improves more with larger sizes of k than POS and chunking, likely because labeled spans are more sparse in NER. Notably, in the zero-shot case the model achieves around 17% accuracy on POS tagging when randomly predicting labels would yield 5.8%.
Structured prompting with GPT-3 Table 1 compares two GPT-3 models to the GPT-Neo series on POS tagging.2 We first compare the 6B parameter GPT-Curie (Gao, 2021) to the similarly sized GPT-J model in a 5-shot setting. We find that GPTCurie underperforms GPT-J by 12.7 points; both models also underperform the per-word majority baseline in this setting.
We then evaluate the largest GPT-3 model, GPT-Davinci, on POS tagging with greedy unconstrained decoding of the entire output sequence. Davinci performs reasonably well and scores similarly to Curie despite the more difficult decoding setting; many errors arise from format errors in the generated output for longer sentences. If we only
2Each experiment reported in this section is repeated across three runs rather than five.
evaluate examples that occur prior to these format errors, performance on that subset of the evaluation data is 72.85 ± 1.3 at k=5 and 78.04 ± 0.8 at k=10.

heading: 4.2 Error Analysis
text: Figure 3 presents an error analysis of structured prompting; complete analyses for other tasks are provided in Appendix A.3. We first break out performance across runs and evaluate how the choice of in-context examples affects performance (left panel). For POS tagging, the choice of demonstrations makes a difference, with some sets performing better than others across models and a performance gap of 4.8 accuracy points between the best and worst run on the 20B parameter model. NER exhibits similar results to POS; however, chunking performance of different demonstration sets is much more varied and inconsistent across models.
Next, we examine common error types in structured prompting with confusion matrices (center and right panel). We zero out the diagonal (representing correct predictions) and normalize the matrices for clarity. Many of the mistakes made by the 20B parameter model on POS tagging are for syntactically similar roles, such as confusing proper nouns for nouns and labeling auxiliary verbs as verbs. However, for BIO tagging the models are not always well-calibrated: on NER, the model most often mislabels “O” tokens, indicating that the model overpredicts named entities.
Given that the choice of demonstrations affects PLM performance, another consideration is: how consistent are the error types across runs? To investigate this, we calculate the pairwise Spearman correlations between the confusion matrices of each run. These correlations are very high for the 20B parameter model, indicating the model makes similar types of error across runs: on average ρ = 0.77 for POS tagging, 0.83 for NER, and 0.88 for chunking; all pairwise correlations have p-values << 0.001. Additionally, the models seem to become more robust across demonstration sets at scale; confusion matrix correlations for the 2.7B model are lower (ρ = 0.71, 0.64, 0.66 for POS, NER, and chunking, respectively).

heading: 5 When Does Structured Prompting Work?
text: We now investigate how structured prompting surfaces linguistic structure from PLMs, using the behavior of GPT-NeoX on POS tagging and NER
as a case study. We find that (1) in some cases, the model generalizes to labels not seen in the demonstration, and (2) the label form has a large effect on performance. Specifically, the model can learn in context when arbitrary labels represent classes but will ignore label mappings in the demonstration that contradict its prior task knowledge.

heading: 5.1 Effect of Seen Labels
text: In Section 4.1, we see that the model obtains above random chance accuracy on zero-shot POS tagging, suggesting that the model does not need to observe the label to associate it with the correct class. To analyze this, we compare the model’s performance when the label is and is not seen in the demonstration, averaged across k-shot runs.
Model performance on unseen tags, and the gain in performance after observing the tag, varies greatly by label class (Figure 4). For some classes in POS tagging, such as ADJ and PUNCT, the model obtains around 50% accuracy without seeing the label. However, unseen performance on AUX in POS tagging and MISC in NER is close to 0%. Furthermore, while observing tags like LOC in NER greatly improves performance, other tags like ADJ and MISC improve much less when seen.

heading: 5.2 Effect of Label Form
text: We hypothesize that the behavior observed in Section 5.1 depends on how informative the label form is for the class. Therefore, we compare the model
performance on (1) the original task labels; (2) shuffled task labels, where we shuffle the label surface forms but maintain underlying class correspondences to words; and (3) proxy labels, where we represent the classes with arbitrary tokens – here, consecutive integers ranging from 11 to 27 (POS) and from 11 to 14 (NER). (Figure 5).
Label shuffling confuses GPT-NeoX Shuffling the labels greatly hurts overall model performance, with POS scores decreasing overall by 50.5%, and NER by 65.9%. Some classes are more robust to the shuffled labels than others: the AUX and DET parts-of-speech score within the standard error of the original class performance, whereas ADJ accuracy drops by 96.2% to near zero.
Interestingly, most mistakes made in the shuffled setting (61.4%) result from the model predicting the true class label rather than the shuffled one from the demonstration. This occurs more frequently for classes whose performance severely degrades when shuffled: 93.9% of errors on the NOUN class are due to this phenomenon, and across classes, there is a strong correlation between performance degradation and the percent of errors predicting the true label (ρ = 0.69, p < 0.05). This result suggests that PLMs ignore in-context label mappings when the model already associates the label with a specific class, similar to findings in Min et al. (2022).
GPT-NeoX in-context learns with arbitrary proxy labels Model behavior with the proxy labels is closer to the original labels, with perfor-
mance decreasing by 25.8% on POS and 30.5% on NER. Indeed, on many labels that significantly degrade with label shuffling, the model performs significantly better on the proxy labels (NOUN and CCONJ in POS tagging, PER in NER). These results demonstrate that the model is able to perform in-context learning to extract linguistic structure, even when the tags are uninformative.

heading: 6 Sources of Linguistic Knowledge in Pretraining Corpus
text: The results in Section 5 demonstrate that the choice of label form can greatly affect structured prompting performance and implies that the model contains prior task knowledge. We analyze contexts in which the labels for POS tagging and NER appear in the Pile (Gao et al., 2020) to better understand what, if any, task information GPT-NeoX learns from pretraining.
Our analysis shows that task information occurs in the pretraining data, both as labeled examples (Section 6.1) and in other related contexts (Section 6.2). However, we find no evidence of test data leakage. Given these findings, we evaluate the model in a new setting that substitutes an English description of each class (e.g., “adjective”, “person”) for the label in order to control for label leakage while still providing meaningful labels (Section 6.3).

heading: 6.1 Task Data Contamination
text: A likely location for task labels to occur is leaked task examples from pretraining data sources. To test this, we search the Pile for instances of labeled POS and NER data (Table 2, the full results are given in Appendix A.4).
POS Tagging Since the POS data is obtained from UD treebanks, we search the Pile for each label as it would appear in the treebank (with tab whitespace on either side of it, see CCONJ example context). We find a significant amount of UD data formatted in this manner: up to 33,000 occurrences for an individual label (NOUN). This is unsurprising given that Github – where UD treebanks are hosted – is a data source for the Pile. However, we find no evidence of test data leakage across any of the POS label occurrences when compared to the GUM treebank (Zeldes, 2017).3
3We also compare the test set against the Pile via other methods (exact document match and searching for individual lines); none of these match any test data against the Pile.
We also perform a closer analysis of the CCONJ label: we compare each occurrence against all nine English treebanks in UD and manually examine it. We find that many CCONJ occurrences can be found in the English Web Treebank (EWT; Silveira et al., 2014) (1052/118/155 from the train/dev/test splits); others match with Parallel Universal Dependencies (PUD; Zeman et al., 2017) (10 occurrences from test set) and ParaTUT (Sanguinetti and Bosco, 2014) (1 occurrence from development set).
Our manual analysis finds that most of the CCONJ occurrences are in non-English documents (77%); other languages whose treebanks we see include Finnish, German, and Arabic, among many others.4 We also observe that every tab-separated instance of CCONJ occurs in the UD treebank format, indicating that this automatic filter is a reasonable estimate of UD data leakage across labels.
NER Task data leakage for NER is much more limited than POS: the most frequent label occurs 5,655 times in the Pile (other than “O” which occurs very frequently in many contexts). Since the CONLL format separates the tags with spaces instead of tabs, it is more difficult to filter for data leakage. Instead, we manually evaluate 100 examples for the BIO labels and give the proportion of the sample that is relevant for NER.
Only a subset of relevant occurrences includes labeled data – our analysis found that labeled data is not common, and most cases are single example sentences annotated in various ways that do not necessarily follow the CONLL format (see I-
4This is unsurprising: though the Pile is characterized as an "English text corpus" (Gao et al., 2020), prior work has found similar corpora derived from the web contain significant amounts of non-English text (Blevins and Zettlemoyer, 2022).
MISC example context). Similar to POS tagging, we also find labeled examples in non-English languages; notably, some of the examples observed are incorrectly labeled.5 This highlights that while the model sees task data during pretraining, the quality and accuracy of that data are unverified.

heading: 6.2 Labels in Other Contexts
text: During the data analysis, we also observe tags from our tasks in settings other than labeled data. Other relevant contexts are task documentation or descriptions (see NOUN, DET, and B-ORG example contexts) and code related to the task (I-LOC example context). These contexts are particularly interesting, as they provide information that may help the model learn by explaining the task in natural language or code, rather than via input/output pairs.
We also observe instances of labels that are unrelated to the task. This is more common for the POS tags; whereas, for NER labels, up to 80% of the sampled contexts are related to the task. The topic of these unrelated contexts varies widely across labels, from biomedical and legal texts (see BPER example context) to unrelated source code and news articles.

heading: 6.3 Relationship Between Labels and Classes
text: Due to the quantity of task data uncovered in the Pile, we would like to control for the effect of pretraining on labeled data. To this end, we evaluate GPT-NeoX on semantically meaningful labels not previously seen in labeled contexts; specifically, we replace the task labels with the English name for
5For example, the phrase “l’entreprise/O SpaceX/O...” occurs in a WebText2 document; however, SpaceX is a named entity that should be labeled as B-ORG.
∆ diagonals report performance with that set. †: delta is within standard error; *: p << 0.001.
each class (e.g., adjective, B-location), which we refer to as the words label set. The model achieves an accuracy of 78.11 ± 1.46 on POS tagging and an F1 score of 56.88 ± 0.86 for NER in this setting.
In Table 3, we compare the performance between these label sets and evaluate how correlated individual class performances are across these sets. We observe an identical ranking across label sets in POS tagging and NER. On NER, the difference in model performance between the true labels and words as labels is within standard error. However, on POS there is a small but significant decrease of 5.4 points between the two; this drop in performance likely quantifies the benefit of observing the POS task data in the Pile.
The correlation study shows that performance across classes on the original, proxy, and words label sets for POS tagging are all strongly correlated (ρ > 0.9). However, their correlations with the shuffled labels are less significant; this difference is likely due to the prior task knowledge GPT-NeoX has for UD labels leading to predicting the actual label of the class rather than the shuffled one, as seen in Section 5.2.

heading: 7 Related Work
text: Prompting PLMs for Sequence Information Recent work has applied various prompting approaches to sequence tagging tasks, primarily focusing on NER (Cui et al., 2021; Ma et al., 2022). However, these approaches also require further training, most often by learning new prompt embeddings for the task (Li et al., 2022; Liu et al., 2022b;
Chen et al., 2022). Other work has finetuned language models to apply them to sequence tagging tasks (Liu et al., 2022a). In contrast, our approach requires no additional parameters to be learned. More similar to our work is the sequence tagging method in Shliazhko et al. (2022), though their approach prompts the model separately for each word in the sentence. Additionally, similar approaches to prompting have been proposed for other tasks; these methods decompose a target task and repeatedly prompt the model on subtasks, building on the model’s outputs to generate the final prediction (Zhou et al., 2022; Press et al., 2022). However, these approaches solve a different subset of NLP tasks and use the outputs from the intermediate prompting steps differently (i.e., by conditioning on them in future prompting steps, whereas in structured prompting each output is a predicted label).
Probing Pretrained Models There is extensive work on probing models for their underlying knowledge (Belinkov et al., 2017; Blevins et al., 2018; Gulordava et al., 2018, inter alia.). The approach has become particularly popular for analyzing masked PLMs (e.g., Liu et al., 2019, 2021b), with behavioral probes (e.g. Petroni et al., 2019; Balasubramanian et al., 2020) in particular using the LM setup to elicit knowledge from the model.
However, prompting autoregressive PLMs (Brown et al., 2020; Schick and Schütze, 2021; Gao et al., 2021), though technically similar to behavioral probing, is usually not framed as probing the underlying model for knowledge. Some exceptions are Alivanistos et al. (2022), which uses prompting techniques to probe the LM for knowledge base relations, and Li et al. (2022), which replaces diagnostic probes with trained prompt embeddings for model analysis. We extend this framing by applying structured prompting as a behavioral probe for linguistic structure.
Analysis of Prompting Methods The results of the structured prompting setup ablations are consistent with prior work. Specifically, our observation of the model’s prior label knowledge is similar to Min et al. (2022). We expand on their findings by showing that the model can still perform in-context learning with proxy labels where the model has no prior mapping for the task.
Other work has also documented the presence of task data in common pretraining corpora (Dodge et al., 2021), shown the effect of pretraining term
frequencies on in-context performance (Razeghi et al., 2022), and demonstrated the ability of LMs to learn from task data during pretraining (Magar and Schwartz, 2022). Similarly, we document the presence of task data and labels in the Pile and find that this signal can help task performance due to the model prior over the labels.

heading: 8 Conclusion
text: We propose structured prompting, a general paradigm for sequence tagging with autoregressive PLMs. Our experiments show structured prompting performs well on three few-shot sequence tagging tasks. Further analysis shows that (1) the approach can elicit linguistic structure in many settings, including when the labels are unrelated to the task, and (2) while labeled task data is present in the pretraining corpora, using informative labels not found in task data gives similar performance to using the task labels. These findings indicate that the model’s knowledge of linguistic structure is more general than the memorization of the task data. More generally, our approach provides a method to probe PLMs for sequence knowledge without training new or existing parameters.
Limitatons
Data Leakage As discussed in Section 6.1, we find evidence of labeled task data for POS tagging and (to a more limited extent) NER in the Pile. We attempt to control for this leakage by evaluating with class names as labels rather than the original tag set; however, due to the cost of training recent PLMs and their large pretraining corpora, it is impossible to control for data leakage when prompting existing models completely.
Both Brown et al. (2020) and Chowdhery et al. (2022) discuss the presence of task data in their pretraining corpora when training PLMs and the difficulty of controlling for it in their evaluations. For downstream users, this issue is further compounded in cases where the pretraining data is unavailable, as it is impossible to even check for contamination in those cases (such as our GPT-3 experiments).
Experimental Limitations with GPT-3 We only perform a subset of our evaluations of structured prompting on GPT-3, due to the cost of running the models in the API; this also means we do not run comprehensive prompt ablations to better tailor the setup for these models. Additionally, the results
(i.e., lower performance than comparable GPT-Neo models) are difficult to interpret due to the black box nature of the GPT-3 models – it may be due to pretraining data differences (as mentioned in the previous limitation), the lack of prompt engineering for the models, or some other discrepancy.
English-only Experiments The experiments in this paper focus on English sequence tagging tasks, and it is unclear how well the proposed method generalizes to other languages. We find evidence of task-relevant data in pretraining corpora in nonEnglish languages, which suggests there is signal for the approach to work in other languages. However, prior work shows that PLMs behave much worse when prompted outside of English (Lin et al., 2022; Shi et al., 2022) but does not address the effect of pretraining data on this phenomenon.

heading: Acknowledgements
text: We would like to thank Sewon Min and Ari Holtzman for their helpful conversations about the work.

heading: A Further Ablations and Analysis
text: In this section, we test additional factors that may affect the performance of our proposed method.
A.1 Choice of evaluation set
For computational reasons, the models are evaluated on a fixed subset of 1000 randomly sampled test examples for each task. As using a smaller evaluation set can introduce noise into our performance estimates, we run a similar experiment on a number of the smaller models but resample the evaluation examples across five runs in addition to varying the demonstrations (Table 4). We find that varying the evaluation examples has a minimal effect on both the average performance and standard error on both POS tagging and NER.
A.2 Ablating BIO Constraints
During this work, we found that limiting the potential output tag space from the model with global
BIO constraints made little difference in model performance for both NER and chunking (Table 5). Specifically, in every case, the difference between the two settings was within the standard error of the means across runs, with NER performing slightly better with the constraints and chunking performing slightly worse.
A.3 Full Results of Error Analysis
We provide additional error analysis results from Section 4.2 in Figure 6.
A.4 Full Results of Pretraining Data Analysis
The complete data analysis for labels not shown in Section 6 is detailed in Table 7.

heading: B Complete Results of Structured Prompting Experiments
text: We provide the full numerical results for the experiments in Section 4.1 in Table 6.

heading: C Responsible NLP Miscellanea
text: This section details information from the Responsible NLP Checklist not covered elsewhere in the paper.
Compute Costs The computational cost of each prompting experiment on the GPT-Neo series of models varies depending on the task and size of the underlying PLM: run times for a single experiment range from around 43 minutes for POS tagging on the 125M parameter model to approximately 50 hours for chunking with GPT-NeoX (20B parameters). The smaller GPT-neo models (fewer than 6B parameters) are run on a single Nvidia RTX-6000, and larger models are run on one or more Nvidia A40 GPUs.
For the GPT-3 POS tagging experiments, we run the models through the OpenAI API. When performing constrained decoding through the API, each example requires multiple calls per word in the sentence to decode the label forms, since model state caching for custom decoding is not available. For GPT-Curie (k=5), with constrained decoding, on average 230M tokens are submitted to the API per run; with Davinci (k=10, where we only performed unconstrained decoding), an average of 1.2M tokens are submitted per run.
Intended Usage of Artifacts To the best of our knowledge, our experiments all fall within the intended use cases of the GPT-Neo models and the Pile dataset, as well as the usage policy of the OpenAI API.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
In the required Limitations Section (after Conclusions)
7 A2. Did you discuss any potential risks of your work? This work presents and analyzes a general prompting technique for core NLP tasks (sequence tagging); there are very limited risks with regards to this work.
3 A3. Do the abstract and introduction summarize the paper’s main claims? The Abstract and Section 1 (Introduction)
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Sections 3 through 6 (used existing artifacts)
3 B1. Did you cite the creators of artifacts you used? In Sections 3 through 6 when discussed
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. We did not create or release any new artifacts
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Appendix C
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. We did not collect any new data
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. Left blank.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Left blank.
C 3 Did you run computational experiments? Sections 3 through 5
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Section 3 and Appendix C
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Section 3 (not model hyperparameters, but prompting format decisions)
C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? No response.
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 3
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: We leave a more thorough examination of this behavior discrepancy for future work