Crafting In-context Examples according to LM s’ Parametric Knowledge (2024)

Abstract: In-context learning can improve the performances of knowledge-rich tasks such as question answering. In such scenarios, in-context examples trigger a language model (LM) to surface information stored in its parametric knowledge. We study how to better construct in-context example sets, based on whether the model is aware of the in-context examples. We identify ‘known’ examples, where models can correctly answer from their parametric knowledge, and ‘unknown’ ones. Our experiments show that prompting with ‘unknown’ examples decreases the performance, potentially as it encourages hallucination rather than searching for its parametric knowledge. Constructing an in-context example set that presents both known and unknown information performs the best across diverse settings. We perform analysis on three multi-answer question answering datasets, which allows us to further study answer set ordering strategies based on the LM’s knowledge of each answer. Together, our study sheds light on how to best construct in-context example sets for knowledge-rich tasks.

## introduction large language models (llms) can perform competitively on knowledge-rich tasks such as question answering via in-context demonstrations @xcite . in such scenarios, in-context examples are used not only to teach the llm the mapping from inputs to outputs, but also to invoke the llm's parametric knowledge @xcite . given such role of in-context examples, we examine how the llm's parametric knowledge of in-context examples impact the effectiveness of in-context examples. let's imagine a very challenging in-context example set, where llms cannot answer any of figure 1 : we study how an lm's knowledge of incontext examples impacts its effectiveness. on the top box, we construct three sets of in-context examples, unknown, halfknown, and known, differing in their difficulty (section 3). on the bottom box, we construct two in-context examples, which contain the same question and answer set, but the answers are sorted differently: one in increasing amount of parametric knowledge and one in reverse (section 4, 5). we pose a suite of research questions connecting parametric knowledge of an lm on in-context examples and its impact on model predictions. figure 1 provides our study overview. we mainly focus on multi-answer qa datasets @xcite @xcite since they allow selecting and ordering the answers among multiple answers. furthermore, we evaluate on a math qa dataset @xcite and two nli datasets @xcite , which require reasoning from llms. we first compare providing 'known' or 'unknown' in-context examples (section 3). we operationalize 'known' in-context examples as those lm can correctly predict with in-context learning. we do not observe a clear winner between two choices, with results varying depending on the dataset. throughout all datasets, however, providing a set of in-context examples that have a mixture of known and unknown information leads to superior performance compared to a set consisting solely of known or unknown in-context examples. our next analysis focuses on the ordering of multi-answer set while fixing in-context example set (section 4, 5). compared to randomly ordering valid answers, semantically meaningful ordering brings substantial changes in model predictions. even alphabetical ordering of answer set changes predicted answers substantially, prompting the model to generate 1.5 more answers on average than when shown a randomly sorted answer set. we further find that ordering the answer set of in-context examples in descending order of model knowledge often leads to performance gains. together, our work suggests best practices for crafting in-context examples, with relation to their parametric knowledge, for knowledge-intensive tasks. ## experimental settings we first describe our evaluation setting which centers around multi-answer qa datasets. ## known examples vs. unknown examples prior work has studied a few characteristics of successful in-context example set, such as label distribution in the in-context example set @xcite . we evaluate in-context examples with respect to model's parametric knowledge, whether a 'known' or 'unknown' in-context example is better. we operationalize 'known' ones as the ones where llms can get the answers correctly from its own parametric knowledge, and 'unknown' ones as those that the model did not answer it correctly. ## ordering answers based on lm's knowledge prior work suggests that the ordering of in-context examples significantly impacts the performance, with more relevant examples being most beneficial when placed last @xcite . yet, no prior work has studied how the ordering of answer set in in-context examples affects model generation and task performances. we investigate this here. following our previous study, our focus is on parametric knowledge of lms being prompted. specifically, we question whether placing answers based on how well the model knows about answers improves the performance. we present strategies to order the answer set of each example, a pair of question q and its gold answer set a * = {a * 1 , a * 2 , ..., a * n }, which will be used as an in-context example. 5 we present two baselines and two methods (perplexity, greedy) for ordering the gold answer set of each in-context example based on model's parametric knowlege. baselines the random baseline randomly orders answers and alphabet orders answers alphabetically. while alphabetical ordering is not input: lm m, prefix p, gold answer set a * = {a * 1 , . . . , a * n }, where each gold answer is a token sequence (i.e., a * @xmath0, . . . wi |a * i | )) output: ordered answer indices of the gold answer set 1: i1 ← {w1 1 , ..., wn 1 } 2: u ← 1 3: while i1 ̸ = ∅ do 4: t ← 0 5: repeat 6: t ← t + 1 7: ot ← argmax @xmath1: p ← [p; ot] 9: it+1 ← {wi t+1 |wi @xmath2: until ∃a * @xmath3, . . . , ot) {this assigns ku the index of completed answer} 11: i1 ← i1 \ {w ku1 } 12: u ← u + 1 13: return {k1, . . . , kn} relevant to model's parametric knowledge of the answer, prior work @xcite has shown that consistent ordering of labels can improve the performance of fine-tuned llm's predictions. knowledge-aware ordering we decide ordering based on the perplexity of individual answer given the prefix, or by performing greedy constrained decoding, given the prefix, on a smaller, restricted vocabulary set. we use the same prefix as in section 3.2, a concatenation of five in-context examples. each ordering strategy will yield two orderings of answers, which either sorts the answers in the descending order of model's parametric knowledge or ascending order (denoted as reverse). ## results for answer ordering strategies having introduced strategies for ordering answers for in-context examples, we study how this impacts the generation of answers with llama2 and opt. we first evaluate whether the generated answers mimic the ordering of answers in in-context examples. then, we evaluate whether the ordering impacts the size and the accuracy of predicted answer set. we also report whether two model's parametric knowledges are in sync, meaning, if one model knows about one fact, does the other model likely to know the same fact? we overall observe such patterns, particularly for quest dataset. throughout in-context learning, the model is expected to learn the pattern shown in the demonstrations @xcite . we assess the generated answers to observe if the model has followed the particular ordering shown in in-context examples. metric we introduce a metric ϕ(s, d s t t , d e , m). this measures how much lm m follows the answer ordering strategy s on evaluation dataset d e when using in-context examples from training dataset d t whose answered are ordered according to s t . 6 when s matches s t , this metric will measure how much predicted outputs mimic the answer ordering strategy of in-context examples. let's denote âi = {a i 1 , a i 2 , ..., a im } be the list of predicted m answers for the i-th example of an evaluation dataset d e , following its generation order from model m . we reorder the predicted answers from âi with respect to s and denote f (a ij ) to be the index of a ij in the newly ordered set. for each consecutive answer pair in âi , we evaluate whether their order is preserved after reordering. then we count the number of consecutive answer pairs that have preserved the ordering, which is the number of pairs that violate the ordering. then, we compute micro average over d e . ϕ(s, d s t t , d e , m) = 100 • @xmath4, and we provide the results for opt model in we further observe that the model is decoding answers such that it will present confident answer first (following the orders of greedy and perplexity), even when answers in in-context example is randomly ordered. even after introducing consistent ordering (presenting less confident answer first), the model shows propensity to present confident answer first (values for reverse greedy and reverse perplexity are below chance (50) consistently). ## conclusion we present comprehensive studies on knowledgeaware prompt design for multi-answer qa tasks. our findings underscore the benefits of having incontext examples that the language model is familiar with. first, the halfknown set aids the model in effectively accessing its parametric knowledge. second, employing knowledge-aware ordering of presenting answers in descending order of the model's knowledge enhances the overall process of answer generation.