Rectifying Demonstration Shortcut in In-Context Learning (2024)

Abstract: Large language models (LLMs) are able to solve various tasks with only a few demonstrations utilizing their in-context learning (ICL) abilities.However, LLMs often rely on their pre-trained semantic priors of demonstrations rather than on the input-label relationships to proceed with ICL prediction. In this work, we term this phenomenon as the ‘Demonstration Shortcut’.While previous works have primarily focused on improving ICL prediction results for predefined tasks, we aim to rectify the Demonstration Shortcut, thereby enabling the LLM to effectively learn new input-label relationships from demonstrations.To achieve this, we introduce In-Context Calibration, a demonstration-aware calibration method.We evaluate the effectiveness of the proposed method in two settings: (1) the Original ICL Task using the standard label space and (2) the Task Learning setting, where the label space is replaced with semantically unrelated tokens.In both settings, In-Context Calibration demonstrates substantial improvements, with results generalized across three LLM families (OPT, GPT, and Llama2) under various configurations.

## introduction large language models (llms) have demonstrated their effectiveness on a wide range of tasks through in-context learning (icl), where models learn to perform a task from demonstrations @xcite . leveraging their pre-trained knowledge, llms can associate various words in the demonstration with specific semantics (e.g., associating 'extremely painful' with 'negative'), thereby performing new tasks using only a small set of input-label examples, without requiring parameter updates @xcite . however, llms often rely on the semantics from their pre-trained knowledge of given demonstrations, resulting in insufficient task learning for the patterns of the provided input-label pairs. @xcite @xcite . this issue intensifies as the model size decreases @xcite . @xcite suggest that smaller llms show promise in learning new mappings from demonstrations in some tasks, yet they still struggle to override semantic priors acquired during pre-training. therefore, it is necessary to develop a method that enables llms of various sizes to effectively mitigate semantic priors preferences and learn to perform unseen tasks from demonstrations. prior works have primarily focused on the instabilities of llms in icl prediction @xcite . to mitigate these instabilities, these studies introduced content-free tokens or utilized the entire test set to calibrate prediction probabilities @xcite @xcite . however, they lack consideration of the semantic priors of llms on the demonstration and do not verify whether their approach enhances llms to learn new tasks from the demonstrations. in this work, we investigate how the llms' pretrained knowledge on the demonstrations affects icl. we define the following phenomenon as a demonstration shortcut: the reliance of llms on their pre-trained semantic priors of demonstrations in icl prediction, rather than learning from the input-label relationships presented in these demonstrations. due to the demonstration shortcut, llms' icl predictions may be overly dependent on the semantics of the given demonstrations even when the label distribution is uniform and the order is identical (figure 1 ). to tackle this problem, we propose in-context calibration, a method designed to rectify the demonstration shortcut in icl. in-context calibration estimates the semantic prior of llms on each demonstration sample with the in-context examples. formally, for each example in the demonstration, we estimate its semantic prior relative to the remaining examples and calculate the expected semantic priors of the demonstrations. at test time, we use this term to rectify llms' dependency on semantic priors and enable the model to learn the intended input-label relationships from the demonstrations. we evaluate the effectiveness of in-context calibration on 27 classification datasets from two perspectives: (1) original icl task and (2) task learning settings. in the original icl task, we use the standard label space. in the task learning setting, the label space is replaced with semantically unrelated tokens. this requires llms to learn the novel input-label relationships to achieve high performance, as these relationships are never seen in pre-training. our proposed method not only demonstrated enhanced performance across various tasks but also showed improvement in task learning abilities. specifically, in-context calibration outperforms other icl methods in natural language inference (nli) tasks, which demand high task learning ability. we also demonstrate that in-context calibration enhances icl performance across various model types and sizes, effectively rectifying the 'demonstration shortcut' problem. ## demonstration shortcut in this section, we first introduce a new typology termed demonstration shortcut. this concept in a zero-shot setting, an llm predicts the test article to the world label. with the first demonstration set, the llm predicts the business label through icl. however, with the second demonstration set which has the same label order but different semantics in the examples the llm predicts the sports label. gpt-j is used for these experiments. see appendix a for a full description of the demonstrations. substantiating our intuition, we conducted additional experiments based on figure 1 . we constructed demo #3 to be characterized by world semantics across all examples, while demo #4 features across technology-related semantics. all demonstration sets were designed to have a uniform label distribution and an identical sequence, as depicted in figure 1 . we ensured the test set also followed a uniform label distribution (25 samples for each label) and reported the average label prediction probabilities for these examples in ## in-context calibration this section introduces a novel calibration method to rectify the demonstration shortcut. we first revisit existing calibration methods designed to improve icl predictions and analyze their limitations, particularly regarding demonstration shortcut. we then propose in-context calibration, our approach to rectifying the demonstration shortcut in icl. ## experimental setups we investigate the effectiveness of in-context calibration in two aspects: (1) the model's performance on the original icl task (using standard label space from the dataset) and (2) the task learning setting (label space is randomly mapped to semantically unrelated tokens), following the experimental settings of @xcite . since the newly introduced input-label mappings have never been parameterized during pre-training, the model must utilize its task learning abilities to handle the problem. in our main experiment, unless stated otherwise, we conduct task learning experi-ments using string numbers foot_1 , and to avoid any pretrained bias, each label is randomly assigned to a unique string number for every seed. ## conclusion in this paper, we introduced the term 'demonstration shortcut', which refers to the reliance of llms on their pre-trained semantic priors of demonstrations for making icl predictions. to rectify this demonstration shortcut and enable the model to learn the input-label relationships from the demonstrations, we proposed a novel method, in-context calibration, based on the provided demonstrations. this demonstration-aware calibration consistently yields the improved performance, regardless of model sizes or types, across various settings. with the introduction of in-context calibration, we anticipate more reliable applications of large language models.