Generative Calibration for In-context Learning (2023)

Abstract: As one of the most exciting features of large language models (LLMs), in-context learning is a mixed blessing. While it allows users to fast-prototype a task solver with only a few training examples, the performance is generally sensitive to various configurations of the prompt such as the choice or order of the training examples. In this paper, we for the first time theoretically and empirically identify that such a paradox is mainly due to the label shift of the in-context model to the data distribution, in which LLMs shift the label marginal p(y) while having a good label conditional p(x|y) . With this understanding, we can simply calibrate the in-context predictive distribution by adjusting the label marginal, which is estimated via Monte-Carlo sampling over the in-context model, i.e., generation of LLMs. We call our approach as generative calibration. We conduct exhaustive experiments with 12 text classification tasks and 12 LLMs scaling from 774M to 33B, generally find that the proposed method greatly and consistently outperforms the ICL as well as state-of-the-art calibration methods, by up to 27% absolute in macro-F1. Meanwhile, the proposed method is also stable under different prompt configurations.

## introduction the learning paradigm has revolutionized in the era of large language models (llms), in which one of the most exciting is in-context learning (icl) @xcite . compared to regular supervised learning, llms can learn implicitly by prompting a few training examples as demonstrations, i.e., in context. this paradigm allows users including amateurs to fast-prototype a task solver with much less annotated data, while sometimes also gaining remarkable performances. there are plenty of studies empirically analyzing icl @xcite @xcite @xcite . notably, icl is found to be pathogenically sensitive to the prompt configuration, e.g., the template, choice, and even the order permutation of the training examples can cause the performance to vary from nearly chance to state-of-the-art @xcite @xcite . this instability motivates lots of efforts on searching for a better prompt in terms of better few-shot training examples @xcite @xcite @xcite , better order permutation of those examples @xcite , and slot position in the template @xcite . another promising direction is calibration @xcite @xcite that adjusts the icl predictive distribution via an estimated bias. compared with prompt optimization methods, such a stream is usually much more lightweight and gains substantial and consistency improvement, while also reducing the instability. however, many of those ideas are based on heuristic intuitions, and there are few principled investigations characterizing how the prompt affects the predictive distribution. this paper fills this gap. in specific, we focus on the text classification task. we study the in-context model p(x, y), i.e., the joint distribution of the input x and label y given a prompt consisting of a few training examples, while the true data distribution is denoted as q(x, y). specifically, we identify that the in-context model p(x, y) poses label shift @xcite to the data distribution q(x, y) both theoretically and empirically. our theoretical analysis (section 3.1) is based on the bayesian interpretation of @xcite , showing that the in-context model p(x, y) is not ensured to be a valid estimate of the true data distribution q(x, y), due to the prior preference of llms and the limited amount of training examples. next, we empirically find that the in-context label conditional p(x|y) is a good approximation of the data label conditional q(x|y), while the in-context label marginal p(y) generally deviates from the true one q(y), which provides persuasive evidence of label shift in the in-context model (section 3.2). then, with this understanding, we can calibrate the in-context classifier p(y|x) by simply adjusting the label marginal p(y), which is naturally obtained by marginalizing out the input x and can be effectively estimated via monte-carlo sampling, i.e., generating new instances from the in-context model. we call our approach generative calibration (gc). actually, most previous state-of-the-art calibration approaches @xcite @xcite implicitly assume that the icl predictive distribution has a shift on label marginal. however, none of them validate the soundness of that assumption, while we for the first time verify this systematically. what's more, their estimates of the label marginal are too heuristic to be solid, while our monte-carlo estimate is an unbiased one to the in-context label marginal p(y). we conduct exhaustive experiments on 12 text classification tasks and 12 llms scaling from 774m to 33b. the results show that the proposed gc greatly and consistently improves the performance of icl (by up to 27% absolute in macro-f1) and outperforms previous state-of-the-art calibration methods (by up to 9% absolute in macro-f1) for all llm scales. meanwhile, gc is also stable towards changes of the choice and order of training examples in the prompt, and exceeds or be competitive to prompt optimization methods, even though some of which are not in the true few-shot learning setting @xcite . overall, gc is a lightweight and effective approach making llms better few-shot learners and saves the effort of cumbersome prompt engineering. ## background in the icl paradigm, we have a few training examples d @xmath0 ## distribution shift of in-context model in this section, we provide both theoretical and empirical evidence showing that the in-context model has a distribution shift on the label. ## generative calibration with the understanding in the previous section, if we accept that p(x|y) = q(x|y), the solution is quite straightforward, that is, adjusting the label marginal to the desired one yields the true label predictive distribution: we follow this convention and leave accurate estimation of data label marginal q(y) in future works 8 . 8 some methods to estimate the data label marginal include bbse @xcite or since our method involves multiple generations (for estimating p(y)), we denote it as generative calibration (gc). gc could be also explained via cost-sensitive learning theory @xcite , where the label marginal is one of the most widely-used costs @xcite . although previous state-of-the-art calibration methods @xcite @xcite share the same form as ours in equation (9), in contrast to our principled unbiased estimate, their estimates are too heuristic to be solid. for example, contextual calibration @xcite estimates the label marginal via heuristically constructed seemingly context-free texts such as "n/a": p(y) = p(y|"n/a"). however, these texts are never verified to be context-free as claimed, because llms might have a preference bias on them. also, llms barely see these texts in the pretraining corpus, which would pose an out-of-distribution (ood) @xcite problem. ## analysis following the finding that gc generally outperforms icl and other calibration methods, we conduct further analysis in this section. ## conclusion in this work, we for the first time theoretically and empirically identify that the in-context model mainly poses a label shift to the data distribution. we then propose generative calibration, a lightweight solution to mitigate this distribution shift. exhaustive experimental results on 12 tasks and 12 llms demonstrate the effectiveness of our approach, in its improvement and stability.