Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generation (2024)

Abstract: Self-consistency (SC), leveraging multiple samples from LLMs, shows significant gains on various reasoning tasks but struggles with free-form generation due to the difficulty of aggregating answers. Its variants, UCS and USC, rely on sample selection or voting mechanisms to improve output quality. These methods, however, face limitations due to their inability to fully utilize the nuanced consensus knowledge present within multiple candidate samples, often resulting in suboptimal outputs. We propose Fine-Grained Self-Consistency (FSC) to addresses these limitations by extracting and integrating segment-level commonalities from candidate samples, enhancing the performance of LLMs both in open-ended and reasoning tasks. Based on this, we present two additional strategies: candidate filtering, which enhances overall quality by identifying highly similar candidate sets, and merging, which reduces input token requirements by combining similar samples. The effectiveness of FSC is demonstrated through extensive experiments on various tasks, including summarization, code generation, and mathematical reasoning, using GPT-3.5-turbo and GPT-4. The results indicate significant improvements over baseline methods, showcasing the potential of FSC to optimize output quality by effectively synthesizing fine-grained consensus knowledge from multiple samples.

## introduction the remarkable success of large-scale language models (llms) have transformed the landscape of natural language processing, showcasing significant improvements across a wide range of tasks, from reasoning tasks @xcite with distinct answers like arithmetic and commonsense reasoning to free-form generation tasks like code generation @xcite and summarization @xcite . however, llms may still generate suboptimal samples in challenging tasks. efforts to improve output quality involve selecting the best response from multiple samples based on specific criteria. this includes using trained models for reranking outputs @xcite and employing llms to evaluate the responses @xcite . however, both approaches require additional models and overlook the knowledge present among the candidates. @xcite introduce selfconsistency (sc) to improve performances without additional models, which mitigates noise from individual sampling by employing a voting mechanism across multiple samples. unfortunately, sc is limited to tasks where the final answer can be aggregated through precise matching. how to aggregate the answers for free-form problems remains unclear. recently, some works seek to evolve the idea of self-consistency into open-ended generative tasks. ucs @xcite calculates the overlap of unigrams between candidates and then selects the final answer with highest value. alternatively, usc @xcite leverages the capabilities of llms instead of rule-based criteria to choose the most consistent one. however, both approaches still rely on selection or voting mechanisms, which do not align well with the nature of free-form generation tasks, i.e., the final quality is determined by the entirety of the output content, rather than specific individual tokens. therefore, sample-level selection methods only yield suboptimal outputs, primarily due to two reasons: (1) they are unable to incorporate consensus knowledge from unselected samples, which, despite their lower overall quality, may contain locally valuable information to enhance the quality of selected samples. (2) they cannot eliminate low-quality segments from selected samples to further improve overall quality. in a word, such coarse-grained selection methods suppress or overlook the role of fine-grained consensus within the candidate samples. figure 1 and distribution usc ucs random 5 / 0 0.0 % 0.0 % 0.0 % 4 / 1 13.9 % 13.9 % 20.0 % to address this issue and better leverage the consensus knowledge among multiple samples for open-ended problems, we propose fine-grained self-consistency (fsc). specifically, after generating multiple candidate samples, fsc extracts the segment-level common elements within them by taking full advantage of llm's text comprehension and contrasting capabilities. subsequently, it synthesizes these consensus elements to produce an optimized output, effectively integrating the essence from the candidate samples. based on this, we further propose two strategies: candidates filtering and merging. the former employs automated metrics to identify candidate sets that exhibit high similarity, thereby enhancing the overall quality of the candidates. the latter strategy merges samples that are highly similar, effectively reducing the quantity of input tokens required. a wide range of tasks, including summarization, code generation, and formal mathematical reasoning, are evaluated on gpt-3.5-turbo and gpt-4 for proposed fsc. the results show that proposed method outperform baselines in a large margin. additional experiments indicate that filter strategy can further enhance performance by selecting better candidates and merge strategy can reduce cost while maintaining the performance. to summarize, this work includes the following contributions: ## method the core idea behind fine-grained self-consistency is to measure and extract the commonality of each response generated by llm at the segment level, and then fuse to generate the final response with superior commonality. figure 2 illustrates the idea of select-based self-consistency and the proposed fine-grained self-consistency (fsc). specifically, select-based self-consistency ranks the responses generated by llm according to specific consistency criteria @xcite , and the top-ranked response is selected as the output. however, these methods do not assess the consistency at a more fine-grained level within the response. on the other hand, fsc first measures the commonality of each segment of the responses generated by llm, and extracts the corresponding common part, then fuses the common part and generates the final output, achieving a more refined consistency sample output. we present the overall workflow of fine-grained self-consistency (fsc) in figure 3 , including fsc with two plugins, candidates filtering and merge. ## conclusion in this work, we propose fine-grained self-consistency (fsc), which fully leverages the segment-level consensus knowledge extracted within multiple samples to overcome the coarsegrained limitations faced by selection based selfconsistency methods. to improve performance and minimize costs, we further propose two strategies called candidates filtering and merge. extensive experiments demonstrate that fsc notably boosts the performance on diverse range of tasks, exhibits superior robustness against noise in input responses, and can be generalized to those tasks where answer extraction is feasible through voting. additional experiments confirm that the proposed candidates filtering and merge strategies can further enhance the performance of fsc while reducing the required computational cost.