Title: Path-Consistency with Prefix Enhancement for Efficient Inference in LLM
Primary area: optimization
Keywords: Large Language Model, Efficient Inference, Reasoning, Chain-of-Thought Prompting, Self-Consistency

Abstract: To enhance the reasoning capabilities of large language models (LLMs), self-consistency has become a popular approach, combining multiple samplings with majority voting. However, current methods are computationally expensive and time-consuming due to the need for numerous samplings. To address this, this paper introduces path-consistency, which leverages the confidence of earlier-generated answers to identify the most promising prefix and guide the generation of subsequent branches. By dynamically guiding the generation of subsequent branches based on this prefix, path-consistency mitigates both the errors and redundancies from random or less useful sampling in self-consistency. This approach reduces errors and redundancies from random sampling, significantly accelerating inference by minimizing token consumption. Our extensive empirical results demonstrate that path-consistency improves inference latency by up to 40.5\%, while maintaining task accuracy across various tasks, including mathematical reasoning, commonsense reasoning, and symbolic reasoning.

TL;DR: We propose path-consistency which achieves optimization of self-consistency, extracting information from early branches of self-consistency in the form of "prefix", guiding the generation of subsequent branches more efficiently.