Tree-of-Code: A Self-Growing Tree Framework for End-to-End Code Generation and Execution in Complex Tasks (2025)

Abstract: Solving complex reasoning tasks is a key real-world application of agents. Thanks to the pretraining of Large Language Models (LLMs) on code data, recent approaches like CodeAct successfully use code as LLM agents’ action, achieving good results. However, CodeAct greedily generates the next action’s code block by relying on fragmented thoughts, resulting in inconsistency and accumulative hallucination. Moreover, CodeAct lacks action-related ground-truth (GT), making its supervision signals and termination conditions questionable in multi-turn interactions. To address these issues, we propose Tree-of-Code (ToC), a self-growing framework that generates nodes through self-supervision, incorporating prompt and model exploration in a GT-free setting. Each node employs CodeProgram, an end-to-end code generation paradigm that aligns executable code logic with global reasoning. This approach uses task-level execution success as both node validity and stop-growing flags, bypassing process supervision to enable online applications. Experiments on two datasets with ten popular zero-shot LLMs show that ToC boosts accuracy by nearly 20% over CodeAct with fewer than 1/4 turns. To further investigate the trade-off between efficacy and efficiency, ablation studies on different ToC tree sizes and exploration mechanisms validate ToC’s superiority.

## introduction large language models (llms) significantly improve agents' ability to leverage external tools. @xcite @xcite . effectively and efficiently handling complex realworld problems @xcite , especially those requiring multiple tools and calls @xcite , has become a key focus across industry and academia. currently, the widely used paradigm, react, @xcite , combines reasoning with action strategies, allowing for actions to be performed incrementally and adjusted based on environmental feedback. the application of code generation techniques to complex task planning and execution has garnered significant attention @xcite @xcite , particularly with the emergence of codeact @xcite approaches. codeact moves the interaction unit from react's individual tool calls to generating code blocks with local reasoning while leveraging code logic and libraries. rather than json @xcite or text @xcite , it treats code as action, utilizing llm's pre-trained coding skills for efficient handling of complex tasks. however, codeact treats each turn as an individual action rather than addressing the entire program, following a step-by-step generation process. while this approach may seem explorative, it has four critical limitations: (i). codeact assumes that the ground truth (gt) is known and uses gt matching as a termination criterion, which is unrealistic and unfeasible. (ii). fragmented thinking is inefficient. for simple problems, stalled thinking is not only unnecessary but also disrupts the logical chains in the code @xcite . moreover, as the number of turns increases, repeatedly integrating prior thoughts causes context overload, heightening model hallucinations @xcite , increasing computational cost. (iii). codeact lacks exploration of diverse reasoning paths. while it supports multi-turn interactions, it follows a single reasoning process. in contrast, solving complex problems often has multiple solutions @xcite , where different approaches can branch from different points, making it difficult to set a standard answer for each turn. @xcite . generated trajectory data is hard to reuse. when using these trajectories for supervised finetuning (sft), they cannot be directly combined into a single program response @xcite . reinforcement learning is also challenging due to the lack of process supervision @xcite , leading to fundamental issues. since defining and obtaining supervision signals for intermediate states is challenging, we propose using task-level feedback directly, treating task completion as a single step. we introduce codeprogram, an end-to-end code reasoning and generation paradigm, as a 'turn,' where the only environmental supervision is execution success. to incorporate reflection and exploration, we design an outcomedriven refinement framework, tree-of-code (toc), that enables multi-turn interactions with diverse solutions exploring the model and prompt pools as tree branches, where task-level codeprograms serve as the nodes. the final output is determined by voting on the collected nodes, selected based on their successful execution. it's important to note that, in this paper, a 'turn' refers to a single action of code generation. for our codeprogram, a turn involves completing the entire program, rather than just a single task step (as in codeact). although toc's name and structure are similar to "tree-of-thoughts" (tot) @xcite , their meanings fundamentally differ. our concept might be closer to a code "random forest" @xcite . while tot enhances "chain-of-thought" (cot) @xcite by exploring different thoughts within the same solution, toc explores multiple distinct program solutions. in other words, each node in toc represents a complete solution, and the tree as a whole captures different iterative optimizations (depth) across a variety of complete solutions (breadth). the core contributions of this paper are summarized as follows: tool agent 1 st turn 2 nd turn try different actions step-by-step ... ... n-1 th turn a conceptual task for illustration: draw a circular smiley face with two black round eyes and a black wide and slightly arched curved line as the mouth on a green background. 🟰 ❓ 🟰 ❓ 🟰 ❓ 🟰 ❓ n th turn program agent ... diverse completed programs explore and filter completed programs successfully executed final program one process done correctly : wrong : correct : / : an action : a completed task program filter: next action: : successfully executed but uncorrectlly execution failed : action: ## design motivation in industry, complex tasks requiring multiple tools and function calls, are typically driven by openended user queries. this creates two key challenges: (1) for zero-shot queries, it is unrealistic to pre-obtain task-level ground-truth (gt), which is required for sft @xcite or reinforced fine-tuning (reft) @xcite . moreover, without gt, the termination criteria become unclear. (2) multi-turn interactions lack a standard trajectory, making it difficult to define the process supervised signals @xcite . current methods often rely on 'llm-as-judge' to evaluate whether the user's needs are met at each step @xcite . however, it would require an api call after every step to check progress, ultimately increasing both time and token costs. besides, the evaluation without objective signals demands strong analytical and reasoning skills from llms. existing methods deliberately avoid these challenges by assuming gt is known @xcite , matching task-level gt with actionrelated outcomes at each step, like the tool agent in figure 1 : the interaction turn stops only if they match, or continues until the step limit is reached. since intermediate states are absent, if possible, why not treat each complete end-to-end execution as an atomic state? by iteratively exploring feasible solutions through parallel executions, we first collect a batch of solutions, and then determine the optimal one, as shown by the program agent in figure 1 . this idea inspires our node outcome-driven reflection system specifically designed for multitool interaction in real-world environments. our thought 2 : i apologize for the confusion. it seems there is an issue with this tool. let me try a different approach. ...... codeprogram import math ...... def haversine_distance(lat1, lon1, lat2, lon2): @xmath0...... @xmath1, @xmath2: ...... task query valid collections collect all executed-successfully nodes ans majority vote from valid collections accepted awaited discarded reflect and expand nodes valid: invalid: node execution success node execution failure thoughts based on the previous implementations, we have successfully find nearby restaurants' location..... 1. add more informative output, including...... 2. round the distance to three decimal places..... 3. add error handling for the case...... execution n: ['restaurant b','restaurant d'] execution coordinates for san francisco: (37.7749, 122.4194) found 2 restaurants within 1km of san francisco: 1. restaurant b (0.000 km away) 2. restaurant d (0.000 km away) layer 1 -layer 2 -layer3 -ans node incorrect continue next action node correct stop and output answer vote and output answer key contribution is a self-growing framework enabling llm agents to autonomously interact with code through zero-shot learning without gt supervision, whose implementation details will be subsequently presented. ## tree-of-code method following the design motivation, we need to collect all valid solutions and identify the one closest to the gt. by treating each tree node as a complete task-level solution and exploring different nodes for breadth while deepening through iterative refinement, we propose toc (tree-of-code), an execution-based, self-growing, and self-filtering tree for handling real-world complex tasks. ## conclusion this paper introduced the tree-of-code (toc) method, which enables self-growing, end-to-end thought-code generation based on successful execution, addressing complex multi-tool online tasks. with efficient model integration and prompt exploration, toc outperformed baselines on two complex task datasets, improving both efficiency and task-solving performance.