Chain-of-Quizzes: Pedagogy-inspired Example Selection in In-Context-Learning (2024)

Abstract: In-context learning (ICL) has emerged as a powerful tool for enhancing large language models (LLMs) in addressing downstream tasks. In this paper, we explore the vital task of example selection in ICL by mimicking the human learning process. We propose a Chain-ofQuizzes (CoQ) framework inspired by educational theories such as Bruner’s Spiral Learning and Mastery Learning theory. Specifically, our framework employs the LLMs to answer the quiz (question in the example) to sift ‘good’ examples, combines these examples iteratively with the increasing complexity, and utilizes a final exam to gauge the combined example chains. Our extensive experiments on diverse reasoning datasets show the proposed approach outperforms baseline models. These findings underscore the framework’s potential for future research. The code and data will be made available here1.

heading: 1 Introduction
text: With the scaling up the model size and corpus size (Devlin et al., 2018; Radford et al., 2019; Brown et al., 2020; Chowdhery et al., 2023), large language models (LLMs) have brought innovation to many fields (Adeshola and Adepoju, 2023; Cheng et al., 2023; Wu et al., 2023b). One of the most important abilities of LLMs, in-context learning (ICL), is a paradigm that allows language models to learn downstream tasks given only a few demonstrative examples (Dong et al., 2022).
In this paper, we delve into the essential task of example selection within ICL. Since ICL assumes that “LLMs can deduce many things from a few cases”, like chain-of-thought that mimics the reasoning process of humans (Chu et al., 2023), we propose to rethink the selection of examples in
* Equal contribution. † Corresponding authors. 1https://github.com/anlaiJoe/Chain-of-Quizzes
analogy to the human learning process as shown in Figure 1. Specifically, we investigate two key questions: 1) What is a ‘good’ example for a learner (LLM)? and 2) How to efficiently combine these good examples?
Regarding the first question, in the human learning process, after acquiring several good examples, learners’ ability of question-answering can be improved. The example here is considered not as knowledge itself but rather as a quiz to help the learner become familiar with the question type and activate relevant knowledge, implying that it must be understood by the learner easily. Therefore, we consider an example as ‘good’ for an LLM if it can successfully respond to the question2 posed in the example, and we will only use good examples in the prompts.
For the second question, we propose a Chainof-Quizzes (CoQ) framework inspired by two educational theories: Bruner’s Spiral Learning theory and Mastery Learning theory3. Bruner’s Spiral
2For simplicity, we focus on QA tasks in the current work. 3Here we use simple words like ‘example’ and ‘question’
to represent complex pedagogical concepts.
10136
Learning theory proposes that learning should be built upon what learners have previously learned, and should gradually increase in complexity of examples (Takaya, 2008). The Mastery Learning theory suggests that the learners should master the current examples before answering the new question (Block et al., 1971). Specifically, the CoQ framework engages the LLM in answering the quiz (question in the example) to sift ‘good’ examples. It incrementally increases the complexity of the quiz based on the model’s performance when combining the examples into a chain, and employs a final exam to evaluate the effectiveness of each example chain. Our framework is further refined using majority voting to account for the inherent randomness in example selection.
Extensive experiments conducted across various datasets demonstrate that our approach yields superior results compared to baseline models. These initial findings highlight the framework’s significant potential for future research.

heading: 2 Related Work

heading: 2.1 In-Context Learning
text: With the development of deep learning, especially the LLMs, significant progress has been made in many tasks (Zhang et al., 2022a, 2023a,b; Wu et al., 2020, 2022, 2024, 2023a; Zhang et al., 2023c, 2024b,a). In-context learning (ICL) is a method where LLMs leverage provided context (e.g., examples) to perform tasks without explicit retraining (Brown et al., 2020; Min et al., 2021; Chen et al., 2022; Wei et al., 2023; Wu et al., 2023b). Recent work in ICL has predominantly concentrated on the design of prompt templates, such as Chain-of-Thought (CoT) and self-consistency (Wei et al., 2022; Zhang et al., 2023d; Wang et al., 2022b). Some studies have explored example selection, which is another important direction of ICL, the existing methods generally rely on retrieval mechanisms for each test instance to select relevant examples (Qin et al., 2023; Mavromatis et al., 2023; Li and Qiu, 2023). Zhang et al. (2022c) uses a grouping approach to ensure the diversity of examples, and Fu et al. (2022) leverages examples with more reasoning steps. (Ye et al., 2023) employ a Conditional Determinantal Point Process (DDP) for joint probability modeling of the demonstration examples. (Zhang et al., 2022b) proposes an identifying generalizable policies-based demonstration selecting strategy. (Rubin et al., 2022) utilizes
LLM to assess the quality of demonstrations. (Li and Qiu, 2023) uses a filter-then-search method to tackle the enumerating challenge. (Chang and Jia, 2023) adopts a scoring approach to address the issue of demonstration selection. In this work, mimicking the human learning process, we propose a framework that selects examples iteratively to boost the performance of ICL.

heading: 2.2 Curriculum learning
text: Curriculum learning (CL) is one of the important data selection strategies, which is inspired by the way humans learn, and involves gradually increasing the complexity of training data, allowing models to build upon simpler concepts before tackling more complex ones (Wang et al., 2021). The key of CL is to define the complexity of training data and plan the training order (Soviany et al., 2022). This strategy not only facilitates more efficient learning but also potentially improves model generalization. CL is similar to the example combination part of our framework, but it is used for model training while we focus on the example selection for ICL.

heading: 3 Method
text: In this section, we introduce details of the framework, whose illustration is shown in Figure 2.

heading: 3.1 Quiz Phase
text: In the first phase, our goal is to select multiple example chains from the initial data pool, which contains a large number of question-answer pairs, as well as to construct a challenging data pool to be used in the next phase. Each example chain comprises k sequential good examples, with each example being a question-answer pair. We add one example per round until we complete k rounds, where k is a hyperparameter.
In the first round, we randomly select a number of questions from the data pool. We then prompt the LLM to take a quiz by answering the selected questions one by one with the following prompt:
P1: [Please attempt to answer the question step by step: <question>]
After reviewing the answers, questions answered correctly are paired with their answers to form good examples for the next round. Incorrectly answered questions are added to the challenging pool.
In subsequent rounds, we sample new questions from the remaining data and conduct a new quiz, leveraging the ICL from previously correctly answered examples. Prompt for these rounds is:
P2: [Based on the following examples <examples>, please answer the question step by step: <question>]
Drawing on Bruner’s Spiral Learning theory, the complexity of new questions should be increased. Following the previous work (Lewis and Frank, 2016), we approximate the question’s complexity by its length and ensure that each new question is longer than those in the examples of the previous rounds. If answered correctly, this indicates that the question-answer pair is a good example within the context of the provided examples. Incorrectly answered questions are again added to the challenging pool.
After k rounds, we obtain several example chains, each containing k sequentially arranged examples.

heading: 3.2 Final Exam Phase
text: Here, the objective is to find optimal example chains, which can help the LLMs answer the questions that they failed before. Recall that based on the quiz phase, we have curated multiple candidate example chains as well as accumulated a challenging data pool containing questions previously unanswered by the LLM. Guided by the principles of Mastery Learning theory, we now conduct a final exam to assess the LLM’s proficiency over the example chains. The final exam process is straightforward: we randomly draw questions from the challenging data pool and require the LLM to answer them using the candidate example chain in the prompt. The prompt used is the same as P2 in Section 3.1. A correct answer indicates that the LLM successfully “masters” the example chain; otherwise, the candidate chain is dropped.
After completing both phases, we have now several validated example chains. Note that one can
repeat the phases several times until enough validated example chains are obtained.

heading: 3.3 Inference
text: During the inference, when presented with a new test question, we can thus obtain different answers using different example chains. A majority vote is then used to determine the final answer. This voting strategy also helps to mitigate the randomness inherent in our framework, ensuring more reliable and consistent results.

heading: 4 Experiments

heading: 4.1 Datasets
text: We conducted experiments across three distinct categories of QA datasets to assess the capability of our method. Specifically, for Mathematics, we utilized GSM8K (Cobbe et al., 2021), AddSub (Hosseini et al., 2014), AQuA (Ling et al., 2017), SingleEq (Koncel-Kedziorski et al., 2015), and SVAMP (Patel et al., 2021). For Commonsense Reasoning, we employed CSQA (Saha et al., 2018) and StrategyQA (Geva et al., 2021), and for Symbolic Reasoning, we utilized the Last Letter and Coin Flip datasets (Wang et al., 2022a). These diverse datasets enabled us to thoroughly validate the effectiveness of our method across various reasoning and computational tasks.

heading: 4.2 Experimental Settings
text: Our experiments were conducted using GPT-3.5Turbo-0613 as the underlying LLM.

heading: 4.2.1 Baselines
text: Zero-shot approach directly employs the LLM to answer questions without presenting any examples. Chain-of-thought (CoT) (Kojima et al., 2022) uses
the “Let’s think step by step” prompt to encourage the LLM’s reasoning ability. Plan-and-Solve Prompting (PS) (Wang et al., 2023) addresses the problem by first creating a plan to break down the task into smaller subtasks, and then executing these subtasks according to the plan. Few-shot approach provides the LLM with a fixed set of examples before it attempts to answer the questions; the examples are randomly chosen questionanswer pairs. Tree of Thought (ToT) (Yao et al., 2023) is a framework allow LLM to explore multiple reasoning paths and make decisions through a tree-like structure of intermediate thoughts. AutoCoT (Zhang et al., 2022c) aims to automatically construct diverse and effective examples by sampling questions and generating reasoning chains. Self-Consistency (Wang et al., 2022b) employs a strategy of first generating 40 reasoning paths and then identifying the most reliable answer by a majority voting.
In our approach, each example chain contains 5 examples, and we have 10 example chains for majority voting. We conduct the following ablation experiments on three main datasets: 1) w/o Quiz: replace the quiz by randomly selecting the same number of examples. 2) w/o Final Exam: remove the final exam phase. 3) Vary the number of examples in each set and the number of example chains. 4) Replace the underlying LLM with Gemini-Pro.

heading: 4.3 Experiment Results
text: We report the accuracy of various methods, with all results averaged over three runs.
From Table 1, we observe the following: 1)
Compared to Zero-shot, the performance of the Few-shot drops significantly when directly adding examples to the prompt (e.g., accuracy on GSM8K decreases from 69.4% to 23.1%). This may be because the examples in the Few-shot are questionanswer pairs, which forces the LLM to generate the answer directly without any reasoning. 2) Our method surpasses baselines on most datasets, particularly in Mathematics. This proves the effectiveness of our example selection framework. Compared to Self-Consistency, we use fewer reasoning paths and achieve better performance. 3) Our method did not yield optimal results on the StrategyQA and Coin Flip datasets. Upon analyzing these datasets, we find that there exists a unified problem-solving approach. For instance, in StrategyQA, the specific problem-solving approach is encompassed by the PS method, leading to its exceptional performance on this dataset.
From Table 2, it is also evident that our framework maintains its superiority even when the underlying Large Language Model (LLM) is changed (Gemini), demonstrating its generality.
From the ablation study presented in Figure 3, we can draw several conclusions: 1) The removal of the quiz phase results in a significant decline in performance, as evidenced by the decrease in AQuA’s accuracy from 70.8% to 57.0%. This suggests the efficacy of our iterative example selection methodology. 2) Eliminating the final exam phase also causes a minor reduction in performance, suggesting that the final exam contributes positively to
the refinement of example chains. 3) An increase in the number of example chains correlates with improved accuracy. However, to achieve an optimal balance between speed and accuracy, selecting 10 example chains is found to be the best choice. 4) The larger the number of examples in an example chain, the better the performance. This proves the effective interaction among the examples.

heading: 5 Conclusion
text: In this work, inspired by pedagogy theories, we propose a novel framework Chain-of-Quizzes (CoQ) to select examples for the in-context learning of LLM. Extensive results demonstrate the effectiveness of our approach. In the future, we will: 1) experiment with other question complexity measurements and 2) expand the CoQ to more tasks.

heading: 6 Limitations
text: In this section, we discuss the limitations of our work:
• The current implementation of our framework can be regarded as preliminary. It however presents an opportunity for further exploration, particularly in diversifying the settings. For example, the definition of “complex” can be extended in Section 3.1. • Our approach utilizes examples in the form of question-answer pairs. While this format has its merits and is quite commonly used nowadays, there exists potential for more effective usage. Incorporating reasoning steps within these examples could provide a deeper context for the LLMs, potentially leading to improved performance.

heading: Acknowledgments
text: This work was supported by the National Natural Science Foundation of China (62441605, 62376245, 62376243, 62037001, U20A20387), the Starry Night Science Fund of Zhejiang University Shanghai Institute for Advanced Study (SNZJU-SIAS-0010), the Key Research and Development Program of Zhejiang Province, China (No. 2024C03255), the Fundamental Research Funds for the Central Universities(No. 226-2022-00051).
Finally, we would like to thank the anonymous reviewers for their helpful feedback and suggestions.

heading: Future Work
text: In the future, we will: 1) experiment with other question complexity measurements and 2) expand the CoQ to more tasks