Smurfs: Multi-Agent System using Context-Efficient DFSDT for Tool Planning (2025)

Abstract: Teaching large language models (LLMs) to use tools for solving complex problems can grant them human-like reasoning abilities. ReAct and its variants are popular frameworks for tool use in both single-agent and multi-agent systems. To address issues like error propagation and limited exploration in ReAct, the Deep First Search Decision Tree (DFSDT) was proposed, but it faces challenges such as rollback instability, redundant context, and premature termination in single-agent settings. We introduce “Smurfs,” a novel multi-agent system (MAS) that enhances DFSDT with a modular, context-efficient, and training-free design. Smurfs surpasses baseline methods in both the open-ended StableToolBench and the closed-ended HotpotQA tasks, reducing token usage by 60.9% compared to DFSDT and enabling Mistral-7b to perform on par with GPT-4-DFSDT. Extensive ablation studies confirm the effectiveness of Smurfs’ core components, offering valuable insights for the construction and interpretation of MAS, and paving the way for future exploration. We release the code at https://github.com/FreedomIntelligence/Smurfs.

## introduction the ability to manipulate tools for complex tasks has long been considered a distinctive characteristic of humans @xcite . can we extend this ability to today's large language models (llms), enabling them to utilize multiple tools to perform complex tasks beyond their inherent capabilities? if llms can use external tools to access knowledge or execute tasks beyond their fixed language modeling capabilities, we can shift the focus of llm training towards enhancing their reasoning and tool-use skills. this in previous multi-agent systems for tool planning, methods like chain-of-thought (cot) @xcite , react @xcite , and the more advanced dfsdt @xcite have been proposed to enhance llms' ability to handle complex multi-step tasks. however, these approaches face notable limitations. react has trouble in handling error propagation and limited exploration. dfsdt incorporates a rollback mechanism and depth-first search approach to address limitations of react, but it suffers from instability when the base model struggles with long-context reasoning. it also introduces inefficiencies due to redundant context handling and risks premature termination when solving multi-step problems. these challenges highlight the need for further innovation to better manage context and reasoning complexity in multi-tool planning systems. in this paper, we introduce 'smurfs', an innovative multi-agent system (mas) framework inspired by the collaborative and versatile nature of its namesake cartoon characters. the proposed framework leverages enhanced dfsdt to perform complex tool planning tasks without the need for additional training. the effectiveness of smurfs is demonstrated through both open-ended and closedended tool planning benchmark experiments @xcite , where it consis-i need recommendations for top paris hotels with breakfast included. please suggest ten, with ratings and reviews. ## related work to augment llms to do multi-tool planning for solving complex problems, previous work has seen numerous attempts. chain-of-thought @xcite is the first to propose the method of thought and answer chain reasoning. react @xcite further introduce the thought-action-observation format for tool chain reasoning, leading to the development of various multi-tool planning methods @xcite @xcite . the latest work, deep first search decision tree (dfsdt) @xcite , is proposed to address the inherent limitations of cot and react: (1) error propagation: error occurs at early stage of planning will result in wrong answer in the end, but it can only be identified until reaching the end of the planning chain (2) limited exploration: single solution chain can't explore the planning space completely. dfsdt is powerful in addressing multi-tool planning problems. its core concept involves employing a depth-first search (dfs) approach for multi-tool planning and backtracks whenever an llm think the solving process has entered a wrong state (for more details, see appendix b). when a tool fails or is deemed inadequate for solving the current problem, dfsdt backtracks to the previous solution state and attempts to resolve the issue using another solution plan. however, three limitations are identified with the mechanism of dfsdt: ## smurfs: mas with context efficient dfsdt the smurfs, the beloved cartoon characters, symbolize unity and resourcefulness, and are good at using tools to overcome any challenge they encounter. ## experiments to evaluate both the effectiveness and efficiency of the smurfs framework, in thie section, we carried out two multi-tool planning tasks: (1) an openended task, stabletoolbench @xcite , and (2) a closed-ended task, hotpotqa @xcite . in addition to these main experiments designed to assess the entire framework, we conducted an ablation studies followed by a case study to test the capabilities of each component within the multi-agent framework and investigate the underlying reasons for its effectiveness. ## conclusion in this paper, we propose smurfs, an innovative mas framework designed to enhance the toolplanning capabilities of llms without requiring additional training. through extensive experiments on both open-ended and closed-ended tool planning benchmarks, smurfs demonstrate its effectiveness by consistently outperforming baseline methods. ablation study further provides deeper insights into the impact of each agent in our framework. based on the ablation study, we propose a hypothesis that can be further tested through more comprehensive experiments in future research: weaker models benefit more from complex multi-agent systems and context segmentation, while stronger models perform better with comprehensive context and simpler agent systems. the findings not only advance the state-of-the-art in multi-tool planning systems but also highlight the potential of modular, training-free frameworks for llms in various practical applications. looking forward, future research could focus on exploring smurfs' use in new domains, such as facilitating the synthesis of high-quality multitool planning data and enhancing the base model's reasoning and tool-use abilities, further advancing the field of adaptive ai systems.