Tree of Agents: Improving Long-Context Capabilities of Large Language Models through Multi-Perspective Reasoning (2025)

Abstract: Large language models (LLMs) face persistent challenges when handling long-context tasks, most notably the “lost in the middle” issue, where information located in the middle of a long input tends to be underutilized. Some existing methods that reduce input have the risk of discarding key information, while others that extend context windows often lead to attention dispersion. To address these limitations, we propose Tree of Agents (TOA) , a multi-agent reasoning framework that segments the input into chunks processed by independent agents. Each agent generates its local cognition, then agents dynamically exchange information for collaborative reasoning along tree-structured paths. TOA enables agents to probe different reasoning orders for multi-perspective understanding, effectively mitigating position bias and reducing hallucinations. To improve processing efficiency, we incorporate prefix-hash caching and adaptive pruning strategies, achieving significant performance improvements with comparable API overhead. Experiments show that TOA, powered by compact LLaMA3.1-8B, significantly outperforms multiple baselines and demonstrates comparable performance to the latest and much larger commercial models, such as Gemini1.5-pro, on various long-context tasks. Code is available at https://github.com/Aireduce952/Tree-of-Agents .

## introduction the capabilities of large language models (llms) have seen rapid improvements in recent years. the advent of models such as @xcite and @xcite marks further progression in model inference capacity. although llms perform well in many scenarios, they still fall short in tasks involving long contexts, such as financial report comprehension @xcite , novel quizzes (wang † corresponding authors. et @xcite and legal contract analysis @xcite . there are three primary challenges in longcontext tasks: (1) position biases processing further impair performance, often referred to as the "lost in the middle" issue @xcite . (2) as input length increases, the presence of extraneous or redundant information grows, potentially diluting the model's focus and degrading output quality @xcite . (3) the computational demands for both training and inference escalate dramatically with longer sequences @xcite . to address these issues, existing approaches can be categorized into three main directions, as shown in category example work pnp no train focus cost inter. model modification position encoding @xcite ) ✘ ✘ ✘ high low attention mechanism @xcite ✘ ✘ ✘ high low training strategies @xcite ✘ ✘ ✘ high low input reduction rag @xcite ) ✘ ✘ ✔ medium medium prompt compression @xcite ✘ ✔ ✔ medium low multi-agent reasoning longagent @xcite ) ✘ ✘ ✔ high high coa @xcite by enhancing key information density or reducing input length through retrieval augmented generation @xcite or prompt compression @xcite . the third direction addresses long text processing by employing multi-agent reasoning @xcite to break down the input into manageable chunks, which are subsequently integrated to arrive at the final result. nonetheless, processing long contexts efficiently remains a challenging problem @xcite . extending the context window for a model may hurt its performance on short texts @xcite . retrieval-based method relies on the quality of external retrieval tools and may introduce noise @xcite . prompt compression may lose key information, especially when information is unevenly distributed in long texts. such methods are more suitable for qa or information extraction tasks and have limited performance for tasks that require global understanding @xcite . multi-agent reasoning requires a well-designed agent communication mechanism to get the right results. as illustrated in figure 1 , coa lets agents process text chunks sequentially and use a manager to integrate the results @xcite , and longagent uses a leader agent to coordinate multi-turn discussions with others @xcite . however, these approaches generally do not support multi-perspective understanding of the document. this study proposes a novel multi-perspective reasoning framework -tree of agents (toa), inspired by cognitive science theories and empirical findings on problem-solving. as shown in figure 1 , toa splits a long text into chunks and employs multiple agents to explore different orders of chunks along multi-paths of a tree structure. this design draws from kahneman's dual-process theory @xcite , which advocates for a multi-perspective approach, and newell and simon's theory of problem-solving as an iterative exploration process @xcite . unlike sequential processing, which often struggles in tasks requiring strategic exploration or early decisions, our method disrupts the natural reading order to enable agents to re-assess and refine their understanding through cross-validation. agents in toa collaborate by exchanging local cognition and applying the mechanism for consensus formation. the multi-perspective reasoning significantly improves performance on various tasks but increases computational cost. to mitigate this issue, we introduce two efficiency-enhancing strategies: 1) a prefix-hash-based cache mechanism that reduces redundant cognition generations; 2) an adaptive pruning strategy that terminates useless reasoning paths early. we conduct extensive experiments with eight baselines on two long-context reasoning datasets and one benchmark, demonstrating the effectiveness of our approach. the main contributions are: ## related work this section summarizes key directions in longcontext modeling for llms. ## methodology we first formulate the problem that toa aims to solve. given a query q and a document @xmath0, t 2 , ..., t m } with m tokens, where m is much longer than the context window length of the llm. our goal is to build a system that can respond to the query correctly according to the document. toa follows a three-phase reasoning process, as shown in figure 2 . ## experiment setup in this section, we describe the preliminary preparations for our experiments (see appendix a and b for complete setup for datasets and baselines). ## conclusion this study tackles the challenge of long-context reasoning in llms. we proposed toa to prompt multi-perspective understanding of long text by using multiple agents to probe the segmented text chunks in different orders. to enhance computational efficiency, we introduce prefix-hash caching and adaptive pruning techniques. empirical evaluations show that toa, despite being built on the lightweight llama3.1-8b, achieves performance on par with significantly larger commercial models on various long-context reasoning tasks.