Retrieved In-Context Principles from Previous Mistakes (2024)

Abstract: In-context learning (ICL) has been instrumental in adapting large language models (LLMs) to downstream tasks using correct input-output examples. Recent advances have attempted to improve model performance through principles derived from mistakes, yet these approaches suffer from lack of customization and inadequate error coverage. To address these limitations, we propose Retrieved In-Context Principles (RICP), a novel teacher-student framework. In RICP, the teacher model analyzes mistakes from the student model to generate reasons and insights for preventing similar mistakes. These mistakes are clustered based on their underlying reasons for developing task-level principles, enhancing the error coverage of principles. During inference, the most relevant mistakes for each question are retrieved to create question-level principles, improving the customization of the provided guidance. RICP is orthogonal to existing prompting methods and does not require intervention from the teacher model during inference. Experimental results across seven reasoning benchmarks reveal that RICP effectively enhances performance when applied to various prompting strategies.

## introduction in recent years, large language models (llms) have achieved superior performance on a wide range of reasoning tasks, which include mathematic reasoning @xcite @xcite , commonsense reasoning @xcite @xcite , symbolic reasoning @xcite @xcite , and so on. to enhance these capabilities further and to align model processing closer to human-like reasoning, recent research has increasingly focused on in-context learning (icl) figure 1 : inference pipeline of ricp. the teacher model analyzes mistakes from the student model, creating guiding principles. these principles help prevent the student model from making similar mistakes. @xcite @xcite , where models generate responses based on a few provided correct examples, effectively adapting to new tasks without extensive training. while icl has proven effective by primarily leveraging correct examples for task adaptation, learning from mistakes remains a fundamental aspect of human learning processes @xcite @xcite . recent studies have shown that utilizing mistakes can also enhance the performance of llms. these methods often start by extracting principles from mistakes identified by the models, and then use these principles to avoid future similar mistakes. for example, tpd @xcite ) uses a teacher model to generate instructions based on the feedback from the student. leap @xcite creates both low-level and high-level principles from the students' mistakes. self-rethinking @xcite encourages llms to rethink whether they have made similar previous mistakes, while @xcite maintains a rule collection to avoid previous mistakes. however, despite the advancements, these methods still face two main limitations. (1) firstly, lack of customization. these methods typically utilize a consistent set of principles across different questions. however, the fixed principle set may not suit all questions, potentially leading to confusion when inapplicable. as a result, the performance of llms may suffer due to the limited customization of these generalized principles. (2) secondly, inadequate error coverage. existing methods usually do not adjust the distribution of principles to encompass a broader spectrum of errors. the inadequate coverage not only narrows the scope of mistakes that the principles can address but also leads to an overemphasis on similar mistakes, resulting in unnecessary token consumption. consequently, this limitation not only diminishes the effectiveness of these methods but also increases the inference costs. to address the aforementioned limitations, we propose a novel approach named retrieve in-content principles (ricp). ricp is a teacherstudent framework where the teacher generates principles based on the student's observed mistakes, and the student applies these principles to prevent the recurrence of similar mistakes. specifically, ricp involves three stages: 1) insight generation: the student model is evaluated on the training set to collect mistakes. the teacher model then analyzes each mistake to generate a high-level reason and several specific insights aimed at avoiding similar errors. 2) principle formulation: we cluster mistakes based on their underlying reasons, with each cluster comprising questions that exhibit similar error patterns. mistakes sampled from these clusters are analyzed by the teacher model to formulate task-level principles, which are shared by all questions. for each question, the most similar mistakes are retrieved from each cluster based on question semantics. we apply clustering to the insights from these mistakes to reduce the redundancy and insights sampled from these clusters establish the question-level principles, which are specific to each question. 3) principle utilization: both the task-level and question-level principles are integrated into the student model's existing prompt to enhance its question-answering capabilities. the main advantages of our method are three folds: (1) firstly, ricp retrieves the most relevant insights for each question, enhancing the specificity and relevance of the question-level principles. this tailored approach notably improves the customization of the principles provided to the student model, leading to more precise and cus-tomized guidance. (2) secondly, during the principle formulation stage, we apply reason clustering to construct clusters representing different error patterns, which are then utilized to derive both tasklevel and question-level principles. this strategy broadens the error coverage of the principles. additionally, insight clustering reduces the redundancy of the question-level principles, enhancing the efficiency of our approach. (3) thirdly, ricp is orthogonal to existing prompting methods, enabling seamless integration that improves performance across various prompting strategies. importantly, during inference, ricp does not require intervention from the teacher model, which reduces computational overhead. we have tested ricp across seven benchmarks encompassing three reasoning tasks. the empirical results validate that ricp can significantly enhance the accuracy of reasoning tasks, confirming its effectiveness and utility in diverse applications. to summarize, our contributions are as follows: ## conclusion in this paper, we introduce ricp, a teacher-student framework designed to prevent the student model from making previous mistakes. ricp significantly enhances the customization and mistake coverage of principles by providing relevant insights for each question and applying hierarchical clustering to the reasons and insights. extensive experiments across seven benchmarks in three reasoning tasks with various llms demonstrate that ricp consistently enhances model performance.