Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrations (2023)

Abstract: In-context learning (ICL) is an important paradigm for adapting large language models (LLMs) to new tasks, but the generalization behavior of ICL remains poorly understood. We investigate the inductive biases of ICL from the perspective of feature bias: which feature ICL is more likely to use given a set of underspecified demonstrations in which two features are equally predictive of the labels. First, we characterize the feature biases of GPT-3 models by constructing underspecified demonstrations from a range of NLP datasets and feature combinations. We find that LLMs exhibit clear feature biases—for example, demonstrating a strong bias to predict labels according to sentiment rather than shallow lexical features, like punctuation. Second, we evaluate the effect of different interventions that are designed to impose an inductive bias in favor of a particular feature, such as adding a natural language instruction or using semantically relevant label words. We find that, while many interventions can influence the learner to prefer a particular feature, it can be difficult to overcome strong prior biases. Overall, our results provide a broader picture of the types of features that ICL may be more likely to exploit and how to impose inductive biases that are better aligned with the intended task.

## introduction in-context learning (icl) is an increasingly popular paradigm for adapting large language models (llms) to downstream tasks @xcite . it works by prompting llms with a small set of examples that demonstrate the input and output of a task, without requiring any parameter updates. however, a key limitation of icl is that it can only incorporate a small number of demonstration examples (e.g., 16) due to the context length limit of ## data construction we choose datasets to cover four different nlp tasks, including both single-sentence and sentencepair classification. for sentiment analysis, we use imdb @xcite and yelp @xcite datasets; for toxicity classification, we use the civilcomments dataset @xcite ; for natural language inference, we use the mnli dataset @xcite ; and for question answering, we use boolq @xcite . for each dataset, we select the original classification label as the default feature and denote it as h 1 . we select alternative comparison features (h 2 ) using existing metadata or simple, deterministic functions, chosen to reflect realistic sources of ambiguity or spurious correlation that have been studied in prior work @xcite @xcite , as well as common shallow features such as length, capitalization, and the presence of particular words or punctuation marks. all datasets and features we use are listed in task dataset hypotheses single-sentence classification sentiment analysis imdb + yelp sentiment (positive vs. negative) domain (imdb vs. yelp) length (short vs. long) terminal punctuation (exclamation vs. period) contains word ("nice"/"food") capitalization (lowercase vs. uppercase) toxicity classification civilcomments toxicity (toxic vs. non-toxic) gender (female vs. male) sexuality (lgbtq vs. non-lgbtq) religion (muslim vs. christian; muslim vs. jewish) race (black vs. white; asian vs. white) length (short vs. long) capitalization (lowercase vs. uppercase) sentence-pair classification natural language inference multinli entailment (entailment vs. non-entailment) domain (government vs. fiction; government vs. telephone) lexical overlap (overlap vs. non-overlap) hypothesis length (long vs. short) hypothesis negation (contains "not", "n't", "no") question answering boolq answer (yes vs. no) question word ("is/was" vs. "do/does/did") lexical overlap (overlap vs. non-overlap) question structure ("is x the same as y") passage length (short vs long) for alternative features, we consider: domain of the text (from the genre meta-data); lexical overlap, following the definition in hans @xcite ; whether the hypothesis is shorter or longer than the premise; and whether the hypothesis contains negation words ("no", "not", "n't"). ## comparing interventions our findings that llms can have strong feature biases have important implications: when the llms' biases do not align with users' intended task, such biases would hurt performance. to resolve such misalignment, we explore a set of intervention methods designed to encourage the model to prefer one feature over another, examining whether the haccuracy for the intended feature indeed increases. ## conclusion in this work, we constructed underspecified prompts from real-world datasets to study feature biases of large language models. we found that the instruction-tuned instructgpt model prefers the "default" task features over distractor features more often than the base gpt-3 model, and we demonstrated the effectiveness of various intervention methods in steering models to use the specified feature. these results not only shed new insights into the working mechanisms of icl, but also have practical takeaways for discouraging models from exploiting unintended features such as demographic biases or shallow statistical cues.