Advancing Reasoning with Off-the-Shelf LLM s: A Semantic Structure Perspective (2025)

Abstract: Large Language Models (LLMs) have shown strong capabilities in zero-shot reasoning and generalization to new tasks. However, the zero-shot performance of general LLMs on complex tasks, such as multi-hop reasoning, remains suboptimal, while reasoning LLMs suffer from hallucinations and unfaithfulness. In this paper, to handle these limitations, we introduce a novel structure analysis method that helps LLMs better understand the question structure and guide the problem-solving process. We demonstrate that existing reasoning strategies, such as Chain-of-Thought and ReAct, significantly benefit from the LLM’s inherent understanding of semantic structure. We further ground our method in the theory of probabilistic graphical models to support its effectiveness. To enhance the reasoning process, we augment the structure analysis with refinement and retrieval capabilities, forming a multi-agent reasoning system called Structure-oriented Autonomous Reasoning Agents (SARA). Extensive experiments show that SARA significantly improves zero-shot performance on knowledge-intensive and mathematical tasks. Remarkably, our approach makes a general LLM competitive with dedicated reasoning models in several benchmarks and demonstrates strong robustness against corrupted reasoning paths.

## introduction large language models (llms) have shown remarkable potential in various reasoning tasks @xcite @xcite @xcite , making llmbased reasoning a fascinating area of research in artificial intelligence. besides the literature which exhibits llms' strong reasoning abilities when provided with task-specific exemplars @xcite @xcite , more recent studies in zero-shot reasoning methods @xcite demonstrate their unique advantages, and reasoning llms @xcite are specially trained to perform complex reasoning tasks and exhibit impressive zeroshot reasoning performance. despite the promising potential of zero-shot reasoning, significant challenges persist in general llms and reasoning llms. for general llms, a primary concern is its inferior performance on complex tasks (e.g., answering multi-hop questions) compared to human or few-shot methods @xcite . among incorrect responses, it is often observed that zero-shot methods cannot demonstrate human-like thinking processes, such as comprehensively understanding the problem statements. on the other hand, while reasoning llms achieve stronger performance on complex tasks, they also exhibit more frequent hallucinations in intermediate steps @xcite , and are prone to unfaithful reasoning @xcite , overthinking @xcite , and usually with higher computation cost. to explore an effective and efficient way to enhance the llm's reasoning capability, we find that human cognition literature offers valuable insights. @xcite @xcite shows that skilled problem-solvers excel at reasoning through new problems without examples or external guidance. those solvers analyze a problem's structure, leveraging linguistic and logical patterns to gain a comprehensive understanding @xcite . this process identifies critical components @xcite , their relationships, and related subquestions, while outlining key steps along the reasoning path. these key steps help consolidate the reasoning trajectories, thereby improving reasoning performance effectively and efficiently. inspired by the human analytic thinking process, we introduce a structure-oriented analysis to activate an understand-then-reason process and then improve llm's zero-shot reasoning capability, i.e., llms are guided to understand the structure of problem statements and generate a comprehensive understanding before performing the reasoning process. the proposed method is based on the syntax and semantic structures in the statement, leveraging the inherent ability of llms to parse linguistic patterns @xcite . with the help of grammar structures, llms can accurately identify critical components in the problem statement and relationships among them and further discover related sub-questions. from this perspective, this analytic thinking process mimics human thinking behavior and thus helps explore correct reasoning paths toward solutions. we empirically demonstrate that simply adding this analysis on top of existing methods such as chainof-thought (cot) @xcite and react @xcite can significantly enhance the reasoning performance (section 3.1). in addition, our theoretical analysis (section 3.2) also suggests that extracting correct information from problem statements can effectively reduce reasoning errors, further indicating the potential of our structure-oriented analysis in improving llms' inherent reasoning capabilities. despite the effectiveness of structure-oriented analysis in direct prompting, we notice two typical errors: later reasoning steps deviating from the structure-oriented analysis, and factual errors resulting in incorrect answers even on the correct reasoning path. to handle challenges, we utilize two mechanisms, reflection @xcite and retrieval @xcite , to build a multi-agent reasoning system, structure-oriented autonomous reasoning agents (sara). the additional mechanisms let the reasoning process better follow the analysis and utilize external knowledge when necessary (section 5), achieving a comparable or even better performance than few-shot methods on both knowledgeintensive reasoning and math reasoning tasks for both general llms and reasoning llms. experiments also demonstrate sara's cost-effectiveness compared to the baseline methods. furthermore, we also observe enhanced robustness against backdoor attacks @xcite and injection attacks @xcite . to summarize, the main scientific contribution of this paper is our observation that the zero-shot reasoning ability of llms is not fully explored. supported by both empirical evidence and theoretical validation, the structure-oriented analysis proposed in this paper significantly enhances the zero-shot reasoning capability of llms. furthermore, we enhance the structure analysis with additional mechanisms, forming a multi-agent reasoning system to further improve the performance. llms for reasoning. in recent literature, there is growing interest in enhancing the reasoning capabilities of large language models (llms). chainof-thought (cot) prompting, introduced by @xcite , encourages models to generate intermediate reasoning steps, significantly improving performance on multi-step tasks. building on this, @xcite proposed zero-shot cot, prompting models to "think step by step" without task-specific examples, while @xcite introduced self-consistency to evaluate multiple reasoning paths and select the most consistent one. other methods can also be found in @xcite . besides, sub-problem decomposition is also a widely used approach: for example, @xcite uses few-shot prompting to decompose questions into sub-questions. other related works can be found in @xcite @xcite @xcite @xcite @xcite . however, most of the above methods require task-specific prompting or examples and the zero-shot methods show clear gaps in reasoning performance with few-shot methods. llm agents for problem-solving. besides llms' inherent reasoning capabilities, llm agents are increasingly employed to enhance performance on complex problems by incorporating external feedback, tools, and knowledge. for example, react @xcite enables models to interleave reasoning traces with task-specific actions, allowing them to gather additional information from external sources. some other related works can also be found in @xcite @xcite @xcite @xcite . 2539 when skillful human solvers encounter complex questions, a common routine is to first identify the critical components and related sub-questions for a comprehensive understanding of the problem @xcite . this skill can provide a global view of the problem-solving progress, reduce distractions from irrelevant information, and guide for correct reasoning paths @xcite . inspired by these skills, we introduce structure-oriented analysis, which leverages llms to explicitly extract syntactic and semantic elements from problem statements to guide the reasoning process. ## general agentic autonomous reasoning although section 3.1 demonstrates the effectiveness of our structure-oriented analysis as a direct prompting strategy, there is still a large room for improvement. in particular, we identify the following two typical errors and provide concrete wrong answers falling in those types in appendix j. type a error : while theorem 3.1 shows the potential benefit of utilizing the key information of structure-oriented analysis, the reasoning process can still deviate to incorrect states. type b error : even with a correct reasoning path and an appropriate hidden variable (e.g., θ 4 ), sampling can still introduce incorrect answers (e.g., hallucinating the name of the song). to handle the above challenges, we augment structure-oriented analysis with two mechanisms: 1) reflection (for type a error) to encourage aligning with structure-oriented analysis and maintain trajectory consistency. 2) retrieval (for type b error) with external knowledge to mitigate hallucination of llms. we integrate these mechanisms with structure-oriented analysis into a multi-agent system, named as structure-oriented autonomous reasoning agents (sara) to build a general and flexible solution for different reasoning tasks. ## conclusion in this paper, inspired by human cognition, we introduce structure-oriented analysis to encourage llms to understand the query in a more formulated way. utilizing the analysis, llms can better identify key steps when performing reasoning tasks, improving the performance. furthermore, built upon the structure-oriented analysis, we further establish a multi-agent reasoning system to the llm's reasoning process. experiments have demonstrated the effectiveness of the proposed framework in knowledge-intensive tasks, math reasoning tasks, and is also effective for reasoning models.