DDPrompt: Differential Diversity Prompting in Large Language Models (2024)

Abstract: Large Language Models (LLMs) have shown that their reasoning ability could be enhanced through approaches like Chain-of-Thought (CoT) prompting. However, these methods use single prompts for different types of questions and do not design appropriate prompts for questions with different characteristics. In this paper, we aim to explore a methodology that generates differentially diverse reasoning paths for different types of questions. To achieve this, we propose a novel prompting strategy called Differential Diversity Prompting (DDPrompt). Firstly, we generate the optimal prompts collection based on question characteristics. Then, we use this optimal prompt collection to generate multiple answers for a question and choose the final answer by voting. We evaluated DDPrompt on twelve reasoning benchmarks and significant improvement in the performance of LLMs on complex reasoning tasks (e.g., GSM8K 75% → 84%, Tracking Shuffled Objects (68.8% → 83.9%)).

heading: 1 Introduction
text: Large Language Models (LLMs) (Brown et al., 2020; Chowdhery et al., 2023) have shown remarkable abilities by learning from demonstrations while keeping their parameters frozen, which is called prompting. The design of prompting is crucial as it can significantly impact the performance of the LLMs on complex reasoning tasks (Chu et al., 2023), such as arithmetic reasoning (Cobbe et al., 2021; Patel et al., 2021), commonsense reasoning (Geva et al., 2021; Talmor et al., 2019), symbolic reasoning (Wei et al., 2022; Srivastava et al., 2022).
Recent studies (Chu et al., 2023) have explored various prompting strategies. For instance, Chainof-Thought (CoT) prompting (Wei et al., 2022)
*Corresponding author
provided step-by-step reasoning examples to facilitate LLMs decomposing complex reasoning tasks into intermediate steps. However, this method required careful manual design of demonstrations, which is time-consuming and labor-intensive. ZeroShot-CoT (Kojima et al., 2022) discovered that by adding a single trigger sentence, such as "Let’s think step by step", after the question to induce the LLMs in generating the reasoning paths, they could achieve competitive performance to standard CoT. Some research (Wang et al., 2023; Naik et al., 2023) have found that utilizing diverse prompts could effectively improve the reasoning ability of LLMs. For example, (Wang et al., 2023) introduced a self-consistency technique involving generating multiple reasoning paths using a decoding strategy different from standard CoT. (Naik et al., 2023) leveraged LLMs to automatically generate diverse prompts, which were then ensemble across multiple inference calls for each question.
In this paper, we aim to improve the performance of the LLMs by designing a prompting strategy that decreases manual labor and increases the diversity of prompts. One method we were considering is to utilize diversity trigger sentences, such as "Let’s think step by step", "Let’s think about this logically" mentioned in Zero-Shot-CoT, to facilitate LLMs generate diversity reasoning paths for each question. However, this naive approach is inefficient. As per human experience, choosing the appropriate methods for a question based on its characteristics is crucial. Inspired by this mind, we assume that different trigger sentences have varying effects on different types of questions. We choose appropriate trigger sentences to generate reasoning paths for a question. As shown in Figure 1, we conducted a preliminary experiment on the GSM8K (Cobbe et al., 2021). We noticed that the accuracy of different trigger sentences varied across different clusters. Therefore, we explore an approach to generate differentially diverse reasoning paths for different
types of questions. To achieve this, we propose a novel prompting strategy called the Differential Diversity Prompting (DDPrompt). This approach involves two stages. In the first stage, we generate an optimal trigger sentence set for each type of question. In the second stage, we utilize the optimal trigger sentence set to obtain the final answer for a question. By using this approach, we can provide differentially diverse reasoning paths for different types of questions and ensure an analysis from various perspectives.
We evaluate DDPrompt on twelve reasoning benchmarks from four categories of reasoning tasks, including arithmetic, commonsense, symbolic, and logical reasoning tasks. The result shows that DDPrompt could significantly improve the performance of LLMs compared to Zero-Shot-CoT. For instance, GSM8K (75% → 84%), AQUA-RAT (50% → 63%), Last Latter (64% → 89.8%), Tracking Shuffled Objects (68.8% → 83.9%).

heading: 2 Method
text: In this section, we provide a detailed explanation of the techniques used in DDPrompt. This method is distinct from the Zero-Shot-Cot (Kojima et al., 2022), which uses a uniform trigger for different questions, e.g., Let’s think step by step. Figure 2 shows the difference between ZeroShot-Cot and DDPrompt. DDPrompt involves two stages: Generating Optimal Trigger Sentence Set(GOTSS) and Inference.

heading: 2.1 GOTSS
text: In this section, we introduce how to generate the optimal trigger sentence set for different types of questions, which consist of two parts: (1) Question clustering, We partition the questions into a small number of clusters based on their similarity; (2) Generating Optimal collection. An optimal trigger sentence set is generated for each cluster by verifying the validity of different trigger sentences.

heading: 2.1.1 Question clustering
text: To classify the questions into different types, we first cluster the questions based on their similarity. Give a question collection Q. We obtain an embedding for each question q ∈ Q using SentenceBERT (Reimers and Gurevych, 2019). Then, the question embeddings are fed into the K-Means clustering. Finally, we get a collection of clusters C = {c1, c2, ..., cm}, where each cluster ci ∈ C contain several questions of the same type and m is the number of the cluster in C.

heading: 2.1.2 Generating Optimal Collection
text: Since different trigger sentences may perform differently depending on the type of question. For each cluster, we select a few best-performing trigger sentences to form the optimal trigger sentence set. Specifically, followed (Kojima et al., 2022), we first manually constructed a set of different trigger sentences T = {t1, t2, ..., tn}, where n is the number of trigger sentence in T. Second, we verify the effectiveness of n trigger sentences separately for each cluster ci ∈ C obtained in the previous parts. For each t ∈ T and each q ∈ ci, we input [q, t] into (Kojima et al., 2022) to obtain an answer a. We then compare a to the ground truth to determine the accuracy of t for ci. After that, we get the accuracy of n trigger sentences for ci. We then choose the highest accuracy k trigger sentences to form the optimal trigger sentence set for ci, where k < n. We perform the above operation for each c ∈ C, and finally get a collection of optimal trigger sentence set D = {d1, d2, ..., dm}, where di is the optimal trigger sentence set for ci.
During the GOTSS phase, we cluster the training dataset and then randomly select a subset of samples to generate the optimal trigger sentence set. In the case where only the test dataset is available, we randomly partition the test dataset into a training dataset and a test dataset, and then apply the same procedure to the specified training dataset.

heading: 2.2 Inference
text: In the previous stage, we generated an optimal trigger sentence set for each cluster. In this stage, we leverage these optimal trigger sentence sets to infer the answer to the question. As shown in Figure 2. First, give a question q, we obtain embedding of q using Sentence-BERT (Reimers and Gurevych, 2019). Then, we identify the cluster that is most similar to q by computing the cosine similarity between q and each cluster ci ∈ C. Subsequently, we select ci that is most similar to q and retrieve the optimal trigger sentence set di = {t1, t2, ..., tk} for ci. For each t ∈ di, we input [q, t] into (Kojima et al., 2022) to obtain an answer a. Finally, we get k answers for q and the final answer is determined by utilizing majority voting.

heading: 3 Experiments

heading: 3.1 Tasks and Datasets
text: In the experiment, we evaluate DDPrompt on twelve benchmarks from four categories of reasoning tasks: (1) Arithmetic (SingleEq (KoncelKedziorski et al., 2015), AddSub (Hosseini et al., 2014), MultiArith (Roy and Roth, 2015), AQUARAT (Ling et al., 2017), GSM8K (Cobbe et al., 2021), SVAMP (Patel et al., 2021)); (2) Commonsense (CSQA (Talmor et al., 2019), StrategyQA (Geva et al., 2021)); (3) Symbolic (Last Letter Concatenation, Coin Flip) (Wei et al., 2022); (4) Logical (Date Understanding, Tracking Shuffled Objects) (Srivastava et al., 2022).

heading: 3.2 Baselines
text: We compare DDPrompt to four baselines: ZeroShot (Kojima et al., 2022), Zero-Shot-CoT (Kojima et al., 2022), Few-Shot (Wei et al., 2022), and FewShot-CoT (Wei et al., 2022). Zero-Shot and ZeroShot-CoT utilize the same trigger sentence as stated in (Kojima et al., 2022). Few-Shot and Few-ShotCoT use the same demonstration examples as stated in (Wei et al., 2022)
In the experiment, we use the GPT3.5-turbo from OpenAI1 as LLM. We manually constructed n = 14 different trigger sentences and set k = 5.

heading: 3.3 Result
text: The accuracy of DDPrompt is compared with different baseline methods for twelve reasoning datasets in Table 1. DDPrompt shows significant improvement in performing reasoning tasks as compared to Zero-Shot-CoT. For instance, GSM8K (75% → 84%), AQUA-RAT (50% → 63%), Last Latter (64% → 89.8%), Tracking Shuffled Objects (68.8% → 83.9%). DDPrompt outperforms eight out of twelve reasoning tasks (SingleEq, MultiArith, AQUA-RAT, GSM8K, SVAMP, CSQA, Last Letter Concatenation, Tracking Shuffled Objects) compared to Few-Shot-CoT that has manual design rationales. Additionally, for the arithmetic reasoning tasks, AQUA-RAT, GSM8K, and SVAMP datasets involve multi-step reasoning, which is more complex than other arithmetic datasets (Chu
1https://openai.com/
et al., 2023). DDPrompt has proved to be more effective in improving performance on these complex datasets. It indicates that DDPrompt is better suited for solving intricate and challenging problems.
In the datasets used in this paper, the first three arithmetic datasets, i.e. SingleEq, AddSub, and MultiArith, contain relatively simple problem(Chu et al., 2023). Consequently, commendable results can be attained without necessitating multiperspective analysis, as shown in Table1. For these datasets, using Zero-Shot-CoT and Few-Shot-CoT produces satisfactory results, reducing the distinctiveness of our approach’s advantage. However, for the last three arithmetic datasets, especially AQUARAT and GSM8K, which contain more intricate problems(Chu et al., 2023), addressing these intricate problems requires generating multiple reasoning paths for solving the problem from diverse perspectives(Wang et al., 2023). This significantly improves the performance of our method on more complex arithmetic problems.

heading: 3.4 Ablation study
text: To evaluate the effectiveness of two design components of DDPrompt, which are called Random-K and Single: In the Random-K variation, K trigger sentences are randomly selected and compared with the K trigger sentences that have the highest accuracy to evaluate the effectiveness of Top-K. In the Single variation, only the Top 1 trigger sentence is selected as a contrast experiment to evaluate the effectiveness of diversity. We conduct an ablation
study by removing each component one at a time. Figure 3 shows an ablation study results with two variations of DDPrompt. We can conclude that both design components are effective and essential.

heading: 4 Related Works
text: Chain-of-Thought (CoT) (Wei et al., 2022) generated intermediate thought steps for problemsolving and significantly improved the reasoning ability of LLMs. Different from the CoT approach, least-to-most (Zhou et al., 2023) suggested solving complex problems by decomposing them into a series of simpler subproblems. These methods are tedious to manually construct the appropriate rationales for the different questions in the demonstration. Zero-Shot-CoT (Kojima et al., 2022) in-
volved adding a simple trigger sentence like "Let’s think step by step" after the question to facilitate LLMs generating a step-by-step reasoning path. Auto-CoT (Zhang et al., 2022) proposed selecting demonstrations from different cluster methods and exploiting the benefits of diversity in demonstrations. (Wang et al., 2023) proposed a selfconsistency method that replaced the greedy decoding method used in CoT with a temperature sample to obtain a set of diverse reasoning paths. Li et al. (Li et al., 2023) proposed sampling from varying prompts and then employed a verifier to verify the quality of each reasoning path.

heading: 5 Conclusion
text: In this paper, we introduce DDPrompt, which is designed to generate differentially diverse reasoning paths for different types of questions. DDPrompt consists of two stages: the GOTSS stage, which generates the optimal trigger sentence set for each type of question; the Inference stage, which uses this optimal trigger sentence set to generate multiple answers for a question and choose the final answer by majority voting. We evaluated DDPrompt’s performance on twelve reasoning benchmarks and observed a significant improvement in the performance of LLMs.

heading: 6 Limitations
text: Our proposed DDPrompt is capable of generating differentially diverse reasoning paths for different types of questions. The inference stage requires multiple trigger sentences and questions to be fed into the LLM to generate multiple answers. As a result, our method is much slower in reasoning than other prompting methods. Additionally, another limitation is that we tested DDPrompt using only GPT3.5-turbo and have not yet evaluated it on other LLMs. Therefore, we will evaluate DDPrompt on other LLMs in the future.

heading: Acknowledgements
text: This work is supported by the National Natural Science Foundation of China (No.62206004, No.62272001, No.62072419, No.62106004) and Hefei Key Common Technology Project (GJ2022GX15).

heading: A Appendix
text: A.1 Compared to other methods DDPrompt automatically generates diverse reasoning paths for different types of problems, and clustering techniques constitute a component of our methodology. It is similar to selfconsistency(SC)(Wang et al., 2023) and AutoCoT(Zhang et al., 2022). Furthermore, we conducted complementary experiments on SC and Auto-CoT employing GPT3.5-turbo on the AQUARAT and GSM8K datasets. Notably, SC is a fewshot method that requires manually constructing reasoning paths. For a fair comparison, we compare DDPrompt with zero-shot setting SC. The results are presented in Table 2, and it reveals that DDPrompt exhibits superior performance compared to SC and Auto-CoT across both the AQUARAT and GSM8K datasets.
A.2 All trigger sentences Table3 shows all trigger sentences used in this paper.

heading: Future Work
text: Therefore, we will evaluate DDPrompt on other LLMs in the future