E o T : Evolution of Thoughts for Complex Reasoning Tasks (2025)

Abstract: Knowledge-based complex reasoning remains a significant challenge for large language models (LLMs) with in-context learning. To tackle this issue, previous studies focus on ensuring behavior fidelity, factuality, or reliability in generated reasoning processes that guide LLMs to produce solutions. However, these studies often neglect the simultaneous optimization on all these three aspects for each thought. The main challenges are the lack of comprehensive assessment mechanisms and the difficulty of efficient thought-level optimization. This paper introduces the Evolution of Thoughts (EoT) framework, which enhances the factuality, fidelity, and reliability of each thought in the reasoning process through a few LLM inferences. We propose a thought assessment method that is sensitive to knowledge and LLM behaviors, using three scorers to evaluate each thought by considering domain context, semantic alignment, and behavior impact. Additionally, we establish a self-reflective evolution mechanism to facilitate each reasoning process generation in a single-forward inference. Extensive experiments demonstrate that, for knowledge-based complex tasks, EoT improves the factuality and fidelity of reasoning processes by approximately 16.5% and 48.8%, respectively, while enhancing LLM reasoning capability by about 6.2%, outperforming advanced approaches.

## introduction large language models (llms), exemplified by the gpt @xcite and @xcite series, have achieved remarkable success in various natural language processing (nlp) tasks. these models often employ in-context learning (icl) schemes, enabling them to learn from contextual examples without updating billions of parameters @xcite . however, complex reasoning tasks requiring the comprehension of long-context knowledge and the generation of intricate solutions remain challenging for llms with icl prompting @xcite . many studies have shown that guiding llms through step-by-step thought prompts significantly enhances their reasoning capabilities @xcite . these prompts inspire llms to create reasoning processes that explain their behaviors and aid in task resolution. each reasoning process, such as the chain of thoughts (cot) and its variants @xcite , comprises a sequence of coherent text units known as thoughts, which serve as intermediate reasoning steps. in these mechanisms, the reasoning capability of llms is often reflected in the reliability of reasoning processes, which assesses the confidence or correctness of the solutions produced by llms under the guidance of reasoning processes @xcite . therefore, many efforts have been dedicated to exploring logical reasoning processes with high reliability @xcite . existing studies on optimizing reasoning processes can be categorized into three main areas. firstly, some studies focus on enhancing behavior fidelity @xcite . previous research @xcite indicates that llms often provide reasoning processes that differ significantly from their actual reasoning behaviors, which compromises their reasoning capabilities. this discrepancy arises from the lack of awareness regarding the internal knowledge state in black-box llms. to address this issue, xllm @xcite revises reasoning processes through evolutionary iterations to improve their behavior fidelity. secondly, some studies address non-factual errors in reasoning processes to mitigate the hallucination phenomenon in llms. for example, ye et al. @xcite bot @xcite t ot it e ✓ thought got @xcite got it e ✓ thought cot-dec @xcite sqat sf i ✓ crp ye et al. @xcite cot sf i ✓ ✓ crp xllm @xcite cot sf i ✓ crp eot (ours) cot sf i ✓ ✓ ✓ thought 1 cot , t ot , got and sqat stand for chain of thoughts, tree of thoughts, graph of thoughts, and sub question-answer as thoughts, respectively. reli., fact. and fide. stand for the factors of reliability, factuality and fidelity respectively. 2 it e, sf i and crp stand for iterations of each thought exploration, single forward inference, and complete reasoning process, respectively. and durrett, 2022) extract the most factual reasoning process in each generation. thirdly, many studies directly explore reasoning processes with high reliability @xcite . for instance, bot @xcite uses an iterative method to explore many trees of thoughts, and accumulates trial-and-error experiences to derive a reasoning process yielding a reliable answer. nevertheless, enhancing the reasoning process for complex tasks presents three major challenges. multi-objective optimization regarding factuality, fidelity, and reliability has been a long-standing issue, as highlighted by previous research @xcite . secondly, the complexity of long-textual tasks exacerbates the difficulty in optimizing reasoning thoughts. these tasks require detailed knowledge across domains and expect complex solutions, it necessitates a reasoning process with multiple thought steps, involving intricate logic and precise extraction of relevant facts. this complexity hinders the assessment and improvement of the effectiveness of each thought. thirdly, achieving thought-level optimization for reasoning processes while maintaining efficiency is challenging. some studies @xcite improve thought quality by generating individual reasoning thoughts through extensive explorations, which incurs significant overhead. conversely, studies like @xcite strive to produce an improved reasoning process using a single llm inference, it enhances time efficiency but compromises the fine-grained optimization for each thought step. to address these challenges, we propose eot, a framework that evolves reasoning processes to achieve multi-objective optimization in factuality, fidelity, and reliability. firstly, eot includes an assessment mechanism designed for complex reasoning tasks, which evaluates each thought in reasoning processes from all three perspectives. secondly, we introduce a prompting mechanism to facilitate the creation of evolved reasoning processes with a single-forward llm inference. this enables llms to comprehend thought assessment outcomes and ensure collaborative optimization for each thought within a few rounds of self-reflective evolution. as highlighted in we conducted extensive experiments on two datasets, including one with 40 production operational maintenance tasks. compared to five advanced frameworks, eot improves reliability, factuality, and fidelity of reasoning processes by about 6.2%, 16.5%, and 48.8% respectively. the contributions of this work are summarized as follows: ## problem setup this section formulates reasoning processes, introduces three key factors that impact llms' reason-ing capabilities, and defines the evolution problem. ## experiments 4.1 setup datasets to evaluate the effectiveness of eot, we conduct experiments on two datasets: 1) op-sqa: a question-answering dataset with 40 complex operational maintenance (om) tasks covering cloud computing, code management, application upgrades, and more. 2) longbench @xcite : a benchmark comprising problems from various open-source datasets such as hotpotqa, multifieldqa, wikimqa. we select 60 representative questions from longbench. since eot is designed to improve the capability of llms on knowledge-intensive reasoning tasks, we carefully select complex reasoning tasks for evaluation. each task includes a long domain knowledge context with thousands or tens of thousands of tokens, a question description, and a reference answer, necessitating intricate solutions. to highlight the complexity of these 100 representative reasoning tasks, we provide statistics on the context length of selected questions, including the mean, the 95th percentile (p95) and the maximum (max) value. testbed and parameter settings we utilize two widely used llms, qwen2-72b @xcite and gpt-4 turbo @xcite , each serving as the generator of both reasoning processes and solutions. for the assessor, eot uses gpt-4 turbo to implement the nli model ϕ, due to its advanced nli accuracy as evaluated in appendix e.2, and compute window-based bleurt scores in eq. ( 9 ) using v100 gpus. metrics to assess llms' reasoning capabilities, we compute four metrics by comparing answers generated by the reasoning process ŷ with the reference answer a: 1) reliability defined in eq. ( 10 ), 2) bleurt, 3) rouge-l @xcite , and 4) nli results from gpt-4 turbo, denoted as entail(%), which measures the percentage of statements in ŷ entailed by a. additionally, following prior studies @xcite , we use s f ac and s f id from eq.( 7 ) and eq.( 11 ) to assess performance in factuality and behavior fidelity, respectively. scores of a reasoning process are averaged from those of thoughts. higher values indicate better performance for each metric. baselines we compare eot with five advanced frameworks for evolving reasoning processes: 1) bot @xcite , cot-dec @xcite , and its variant factor-dec, which emphasize reliability optimization; 2) xllm @xcite designed for fidelity optimization; and 3) calibrator @xcite targeting factuality and fidelity optimization. according to the schema of reasoning process generation, these frameworks can be categorized into three groups. (1) exploration of structured thoughts: bot guides llms to explore ensemble of trees of thoughts (tots) and acquire trial-anderror reasoning experiences for each explored tot. in bot, a reasoning thought is generated by the llm as a node of tot in each inference, and the complete reasoning process is formed by a series of such llm inferences. (2) revision of the complete reasoning process: a) xllm provides feedback on the fidelity assessment for the complete reasoning process and guides llms to optimize reasoning processes through iterative improvements; b) calibrator focuses on revising reasoning processes to improve factuality and further enhance the reasoning capability of llms through iterations. in these studies, llms generate a complete reasoning process in textual paragraphs via a single-forward inference. (3) question decomposition: a) cot-dec prompts llms to decompose the reasoning process to a sequence of subquestion-subanswer pairs produced in single-forward inference in each iteration. it seeks decomposed results that achieve the best reliability performance as the final reasoning process after multiple iterations; b) factor-dec is a variant of cot-dec. in each evolution iteration, factor-dec guides llms to decompose a reasoning process into a subqeustion sequence and prompts llms to answer these subquestions through step-by-step inferences. in other words, each iteration of factordec involves multiple steps of question-answering in generating a reasoning process. ## conclusion through in-depth analysis, we outline three key factors in reasoning processes to enhance llms' capabilities, namely factuality, fidelity and reliability. we propose eot to evolve llm-generated reasoning processes across these dimensions. an assessor is designed to quantify performance on these aspects for each thought in reasoning processes. additionally, we propose a prompting mechanism that efficiently guides llms to comprehend assessments and trigger self-reflection to achieve thoughtlevel and multi-objective optimization of reasoning processes via single-forward inference. eot offers two key advantages. first, it considers comprehensive factors to improve reasoning capability. second, it ensures thought-level evolution with high time efficiency. in the future, we will address unsupervised evolution of reasoning processes to further enhance reasoning capabilities of llms for out-ofdomain tasks without reference solutions.