Dynamic Voting for Efficient Reasoning in Large Language Models (2023)

Abstract: Multi-path voting methods like Self-consistency have been used to mitigate reasoning errors in large language models caused by factual errors and illusion generation. However, these methods require excessive computing resources as they generate numerous reasoning paths for each problem. And our experiments show that on the arithmetic reasoning task, SVAMP, half of the problems fail to obtain noticeable accuracy gains when voting with more than three paths. In this paper, we propose a novel multi-path voting technique called Dynamic Voting, which effectively reduces the number of reasoning paths during multi-path voting while preserving accuracies by applying early exiting for problems that large language models can confidently solve. Experimental evaluations on arithmetic, commonsense, and symbolic reasoning tasks under few-shot and zero-shot settings demonstrate that Dynamic Voting achieves comparable accuracies employing significantly fewer reasoning paths. Notably, one of our Dynamic Voting strategies outperforms Self-consistency using only 24.7% of the number of paths on the LetterConcat task in the few-shot setting. Furthermore, Dynamic Voting showcases strong robustness in threshold selection. It also demonstrates excellent generalizability when combined with other voting techniques, different models, and diverse prompts.

## introduction prominent large language models (llms) like @xcite , chinchilla @xcite , and palm @xcite demonstrate exceptional performance in natural language processing tasks. despite the success, their performance on reasoning-heavy tasks, such as mathematics, common sense, and logical reasoning tasks, remains limited when solely increasing * corresponding author. the model size @xcite . to address this issue, @xcite propose chain of thoughts (cot) to guide llms to reason step by step, which however suffers from factual errors and hallucinations in reasoning paths @xcite @xcite @xcite . to mitigate the errors of cot, researchers propose multi-path voting methods, including self-consistency @xcite , di-verse @xcite , and rationale-augmented ensembles @xcite , which generate multiple reasoning paths and aggregate the paths with voting. however, multiple reasoning paths lead to significant increases in computational resources. for instance, self-consistency generates 40 reasoning paths per question, while diverse employs 100, resulting in tens of times more computational effort. this paper aims to address the following problem: how to achieve comparable accuracy in multipath voting using significantly fewer computational resources? one intuition is that simple problems can be solved with fewer reasoning paths and tilting computational resources to complex problems will effectively improve the overall performance of reasoning. in our empirical investigation on an arithmetic reasoning task, svamp @xcite , 57.5% of the problems demonstrate a remarkable level of accuracy, reaching 95.7%, with only three unanimous reasoning paths. and augmenting the reasoning paths on these problems does not significantly improve the accuracy. this shows that generating more paths on half of the problems in svamp is a waste of computational resources with negligible benefit. a more detailed analysis of resource wastage can be found in appendix a. inspired by early exiting @xcite , which prioritizes computational resources to critical features to minimize computational consumption, we propose dynamic voting to reduce the number of reasoning paths while maintaining comparable accuracies in reasoning with llms. the core idea behind dynamic voting is to apply early exiting for problems that the llms can confidently solve with a handful of paths, thus avoiding unnecessary generation of numerous paths. a key challenge in dynamic voting lies in the identification of problems that the llms can confidently solve. self-consistency establishes a robust association between voting consistency and accuracy, which aligns with our finding that a few unanimous paths on svamp lead to high accuracy. thus, we adopt voting consistency as the confidence of the llms in solving problems and validate it through a range of experiments. dynamic voting is a simple and effective method that involves multiple rounds of voting with corresponding consistency thresholds. initially, the llm generates a few paths and votes on them. if the voting consistency threshold is reached, the voting concludes and the current voting result is outputted. otherwise, the llm generates another reasoning path and all the generated paths are used for the subsequent round of voting. this iterative process continues until the threshold is reached or the number of paths reaches a preset maximum. figure 1 illustrates an example of the process. in practice, we introduce two dynamic voting strategies: confidence-based dynamic voting (cdv) and percentage-based dynamic voting (pdv). cdv employs a fixed threshold in each round, offering a straightforward approach. however, it is limited by the need for prior knowledge regarding the probability of correctly solving the problem. for example, when the probability of correct reasoning is 0.5, setting a threshold of 0.9 renders early exiting unlikely. to overcome this limitation, pdv applies early exiting to a fixed proportion of questions that obtain the highest consistency in the current voting round, offering greater flexibility and adaptability in determining when to terminate the voting process. we evaluate these strategies using the openai gpt-3.5 model foot_0 in fewshot and zero-shot settings. following @xcite and @xcite , we conduct experiments on arithmetic reasoning (gsm8k @xcite , svamp @xcite ), commonsense reasoning (csqa @xcite , strategyqa @xcite ), and symbolic reasoning (letterconcat @xcite ) tasks. dynamic voting achieves comparable accuracies using significantly fewer reasoning paths than selfconsistency foot_1 . across the five evaluated datasets, dynamic voting achieves a comparable average accuracy to self-consistency while employing less than 45% of the reasoning paths. specifically, in the few-shot setting on letterconcat, cdv outperforms self-consistency using 24.7% of the reasoning paths. moreover, dynamic voting demonstrates substantial accuracy improvements under reduced path constraints. when both are limited to 25% reasoning path usage foot_2 , dynamic voting achieves an average accuracy gain of 1.6 and 4.7 over self-consistency in the few-shot and zero-shot settings, respectively. remarkably, pdv improves by 12.7 on letterconcat in the zero-shot setting. furthermore, dynamic voting attains higher accuracies when employing comparable paths. on gsm8k, cdv and pdv achieve improvements of 1.9 and 1.0, respectively, compared to the highest accuracy of self-consistency in the zero-shot setting with less than 100% reasoning path usage. in addition to its resource efficiency, dynamic voting exhibits several excellent features. dynamic voting demonstrates stronger robustness of thresholds than self-consistency, providing flexibility in setting appropriate threshold levels. moreover, dynamic voting performs well when combined with other voting techniques, different llms, and diverse prompts. ## dynamic voting this section provides a comprehensive exposition of our proposed method, dynamic voting. we present an overview of the method in section 3.1. we then introduce two dynamic voting strategies, confidence-based dynamic voting and percentagebased dynamic voting in section 3.2 and 3.3. ## conclusion to address the problem of computational resource wastage caused by multi-path voting using large language models, this paper proposes dynamic voting and designs confidence-based and percentagebased dynamic voting methods, which achieve comparable performance to self-consistency using significantly fewer reasoning paths. we demonstrate the effectiveness and efficiency of dynamic voting on arithmetic reasoning (gsm8k, svamp), commensense reasoning (csqa, staat-egyqa), and symbolic reasoning (letterconcat) tasks using the gpt-3.5 model in few-shot and zero-shot settings. dynamic voting achieves comparable accuracies to self-consistency on all the five evaluated tasks with less than half of the reasoning paths. moreover, dynamic voting performs more robustly for the selection of thresholds than selfconsistency and demonstrates its generalizability combined with other voting techniques, different llms, and varied prompts. our study provides valuable insights for developing efficient and effective reasoning methods with large language models. future research can explore more sophisticated dynamic voting methods or explore the application of dynamic voting in other learning scenarios.