Se^2 : Sequential Example Selection for In-Context Learning (2024)

Abstract: The remarkable capability of large language models(LLMs) for in-context learning(ICL) needs to be activated by demonstration examples. Prior work has extensively explored the selection of examples for ICL, predominantly following the “select then organize” paradigm, such approaches often neglect the internal relationships between examples and exist an inconsistency between the training and inference. In this paper, we formulate the problem as a Se quential Se lection problem and introduce Se^2 , a sequential-aware method that leverages the LLM’s feedback on varying context, aiding in capturing inter-relationships and sequential information among examples, significantly enriching the contextuality and relevance of ICL prompts. Meanwhile, we utilize beam search to seek and construct example sequences, enhancing both quality and diversity. Extensive experiments across 23 NLP tasks from 8 distinct categories illustrate that Se^2 markedly surpasses competitive baselines and achieves 42% relative improvement over random selection. Further in-depth analysis shows the effectiveness of proposed strategies, highlighting Se^2 ‘s exceptional stability and adaptability across various scenarios. Code available at https://github.com/microsoft/LMOps.

## introduction large language models (llms) @xcite @xcite have demonstrated remarkable capabilities in handling a wide range of problems and tasks through in-context learning (icl). icl allows these models to learn from a limited number of examples without the need for parameter updates. despite @xcite ; @xcite ; @xcite shows the icl superior potential for various scenarios, the performance of icl heavily relies on the careful example selection @xcite @xcite figure 1 : the "select then organize" paradigm and our sequential example selection. 2022; @xcite . recent work @xcite has also shown there are significant variances in performance across different sets of examples. thus, the example selection plays a vital role in the behavior of icl. given the diverse nature of nlp tasks, researchers have proposed various heuristic criteria for example selection, including entropy @xcite , influences @xcite , uncertainty @xcite , and so on. there are also some approaches focused on retrieving examples based on the similarity between the input and the examples @xcite . more recent efforts @xcite have explored selecting examples based on their contribution to output performance. the above methods select examples and concatenate them together, following the "select then organize" paradigm, as shown in figure 1 . however, these methods often overlook the impact of the relationship between examples. to address this, @xcite proposes a reinforcement learning (rl) approach to formalize the process. yet, this approach faces limitations in modeling and applicability due to the constraints of the rl framework. selecting an optimal example sequence from a large candidate pool is an np-hard problem. to overcome this challenge, we propose a novel method for example selection, called se 2 , which selects the ideal sequence by modeling the conditional probability of the example sequence given the current context. se 2 employs a sequentialaware model that utilizes feedback from an llm across a diverse set of prompting examples as training data. this allows se 2 to model the interrelationships and sequential information between examples. during inference, we adopt a beam search strategy to construct example sequences, which can enhance the quality and diversity of prompts. we empirically validated the effectiveness of se 2 at 23 popular tasks, including question answering, paraphrase detection, and so on. se 2 outperformed competitive baselines and achieved a 42% relative improvement over random selection. our quantitative evaluations demonstrate the advantages of se 2 , showing a significant performance boost with little variance from the sequential training pattern and improved example sequence quality through beam search. moreover, se 2 exhibits strong transferability, enhancing larger llms with feedback from smaller scoring llms. the case study also reveals se 2 's ability to identify example sequences with inherent logical relationships. overall, our contributions are summarized as follows: ## preliminary in-context learning (icl) is a pivotal capability of large language models (llms), enabling these models to undertake a variety of tasks by observing task prompts without updating parameters. in-context prompts are typically sequences that comprise multiple examples. formally, a k-shot prompt for icl consists of k examples. given a test sample (x test , y test ), llms predicts ŷ based on the in-context prompt and input x test : ## method in this work, we depart from the traditional "select then organize" paradigm and propose a novel sequential example selection method, namely se 2 , to construct in-context prompts. this section outlines our method, including example scoring, context sequence construction, training, and the inference pipeline. ## experiments we conducted extensive experiments on a wide range of nlp tasks. the results and analysis of the experiments demonstrate the effectiveness and advantages of our method. ## conclusion in this work, we explore a new sequential example selection paradigm for icl and propose se 2 , a sequential-aware method that can end-to-end select ideal example sequences for test inputs. se 2 leverage the feedback from llms across a broad spectrum of inputs and examples. this approach not only facilitates the modeling of sequential information and the intrinsic connections between examples but also empowers the construction of prompts through the beam search strategy. through extensive experimentation, se 2 demonstrated superior performance over established baselines, highlighting its ability to generate more effective prompts through beam search. our analysis revealed se 2 's effectiveness and robustness in example selection, contributing to its stability and adaptability across different tasks and llms. this work underscores the importance of sequential example selection in improving icl, offering valuable insights for future research in natural language processing.