TaskCraft: Automated Generation of Agentic Tasks
Authors: Dingfeng Shi
Jingyi Cao
Qianben Chen
Weichen Sun
Weizhen Li
Hongxuan Lu
Fangchen Dong
Tianrui Qin
King Zhu
Minghao Liu
Yuchen Eleanor Jiang
Jian Yang
Ge Zhang
Jiaheng Liu
Changwang Zhang
Jun Wang
Wangchunshu Zhou
Venue: ICLR 2026 Poster (2026)
Keywords: agent
generation
LLM
agentic task
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: TaskCraft is an automated workflow for creating difficulty-scalable, multi-tool, and verifiable agentic tasks with execution trajectories.

Abstract: Agentic tasks, which require multistep problem solving with tool use and adaptive reasoning, are becoming increasingly central to the advancement of NLP and AI. Although benchmarks such as GAIA and BrowseComp have advanced agent evaluation, their scalability remains limited by the high cost of human annotation. We introduce TaskCraft, the first automated workflow for generating scalable, multitool, and verifiable agentic tasks of difficulty. TaskCraft progressively complexifies atomic tasks through depth-based and width-based extensions, with incremental validation via rejection sampling and LLM-based linguistic analysis, ensuring both scalability and efficiency. The generated tasks enable trajectory sampling within state-of-the-art workflows, supporting end-to-end SFT and RL training. Experimental results on multiple LLMs show that TaskCraft data substantially improves multi-hop reasoning and agentic capabilities. Further scaling with TaskCraft tasks and applying RL training yields additional gains, achieving state-of-the-art performance on four agentic benchmarks. The resulting dataset comprises 41k tool-intensive tasks across varied difficulty levels, including 12.6k tool-interaction trajectories and 5k multihop decompositions.

Published as a conference paper at ICLR 2025
TASKCRAFT: AGENTIC TASKS
AUTOMATED
GENERATION OF
Dingfeng Shi'; JJinyi Cao'; Qianben Chen', Weichen Sun', Weizhen Li', Hongxuan Lu', Fangchen Dong', Tianrui Qin', King Zhu'
Minghao Liu Yuchen Eleanor Jiang',
Jian Yang?., Ge Zhang?. Jiaheng Liu Changwang Zhang', Jun Wang', Wangchunshu Zhou
OPPO M-A-P $2077AI Beihang University BByteDance Nanjing University
(shidingfeng, zhouwangchunshu @oppo.
ABSTRACT
Agentic tasks, which require multistep problem solving with tool use and adap- tive reasoning, are becoming increasingly central the advancement of NLP and AL. Although benchmarks such GAIA and BrowseComp have advanced agent evaluation, their scalability remains limited by the high cost of human an- notation. We introduce TASKCRAFT, the first automated workflow for generat- ing scalable, multitool, and verifiable agentic tasks of difficulty. TaskCraft pro- gressively complexifies atomic tasks through depth-based and width-based ex- tensions, with incremental validation via rejection sampling and LLM-based lin- guistic analysis, ensuring both scalability and efficiency. The generated tasks enable trajectory sampling within state-of-the-an workflows, supporting end-to- end SFT and RL training. Experimental results multiple LLMs show that TaskCraft data substantially improves multi-hop reasoning and agentic capabil- ities. Further scaling with TaskCraft tasks and applying RL training yields addi- tional gains, achieving state-of-the-ed performance on four agentic benchmarks. The resulting dataset comprises 4lk tool-intensive tasks across varied difficulty levels, including 12.6k tool-interaction trajectories and 5k multihop decomposi- tions. https: thub. conr/OPO-PPersonallllasCCCC
INTRODUCTION
Agentic tasks, defined as autonomous multi-step problem solving that requires tool use and adap- tive reasoning, becoming increasingly central to AI and NLP. Recent progress in language agents Significant-Gravitat 2023); Wu et al. (2023); Li et al. (2023); Zhou et al. (0023 b 2024) empowered agentic workflows to address increasingly complex tasks. For example, ReAct Yao (TAO) paradigm, enabling workflows to solve problems through iterative reasoning and repeated interaction with the environment.
et al. (2023) adopts the Aangght-Attoon-Obsraallll
To assess advanced agent capabilities, benchmarks such as GAIA Mialon et al. (2023). BrowseC- omp Wei et al. (2025). and Humanity's Last Exam (HLE) Phan al. (2025) have been introduced. GAIÀ evaluates reasoning, tool use, and web browsing through 466 real-world questions. BrowseC- omp comprises 1,266 tasks that test an agent's ability to retrieve and integrate complex online formation. HLE includes 2,500 multimodal questions across more than 100 disciplines to measure advanced reasoning and domain knowledge. Although these datasets have advanced agent evalua- tion, their scalability is constrained by the high cost of manual annotation. For instance, constructing HLE required 1,000 experts to label only 2,500 examples, making large-scale expansion impractical.
Previous work has explored the use of large language models to automatically generate queries, addressing the searcity and annotation cost of human-labeled data. These queries can then support reasoning trajectory sampling for supervised fine-tuning (SFT) and Reinforcement Learning (RL). A representative example is the Self-Instruc framework Wang et al. (2022), which demonstrated that
"Equal contribution
1 "ooresponniig author

---

Published as a conference paper at ICLR 2025
LLMs generate high-quality, diverse instruction data for multiturn dialogues. However, these methods are primarily designed for static instruction-follee following scenarios and fall short in modeling agentic tasks that require interaction with external tools and environments. Consequently, such data are insufficient for training or evaluating agents that operate dynamic, real-world settings.
In this work, introduce TASKCRAFT, the first agentic workflow for the automated generation of agentic tasks (queries), with a particular focus on tasks that require chain-of-tool execution, Our approach provides the following advantages:
Scalability. The workflow supports adaptive difficulty, multi-tool integration, and the gen- eration of tasks beyond the capabilities of the dusk-generation agent, along with their cor- responding trajectories.
Efficient Verification. During each task extension, only incremental components undergo agentic validation, eliminating the need for full verification of the extended task.
Our approach begins by generating atomic tasks solvable with single-tool invocations, then pro- gressively increases their complexity through depth-based and width-based extensions. To ensure task quality, we apply rejection sampling retain cases where agents with external tools succeed but L.MSs fail, validating genuine tool necessity. LLM--based linguistic analysis accelerates valida- tion by rapidly examining the incremental modifications introduced during task complexification, without requiring full execution of the entire task.
Based on this method, we constructed a dataset of about 41k candidate tasks spanning different dif- ficulty levels. It further contains roughly 12.6k tool-interaction trajectories and around 5k instances of multi-hop sub-task decomposition.
To evaluate the effectiveness of our generated tasks, we build on the training data used in Tool- Integrated Reasoning (TIR) models Schick et al. (2023); Shen et al. (2023); Wu et al. (2025a) and augment it with TaskCraft-generaan
tasks for SFT and RL trajectory sampling, Incorporating these tasks consistently improves TIR model performance across multiple benchmarks. On GAIA, MHQA data yields 38.8%, which rises to 60.2% (+21.4) with 2.5k TaskCraft tasks and further to 60.8% (+22.0) with 8k TaskCraft tasks for RL, achieving state-of-the-art (SOTA) results among TIR models and demonstrating the effectiveness of our approach.
NOTATIONS AND PRELIMINARY
Task
"Steck with highest price incretse loday de
FINIC
LLM Parsing
Input Indox
LLA) Recsoning
Liagge Tool
Execution
Beed Connect e ENTCCesBBAet TSLAC-L45%)
Implich "Highest stock price increans"
Reasoning
Figure 1: Execution flow of a single tool invocation.
2

---

Published as a conference paper at ICLR 2025
To design tasks for agentic reasoning, abstract the execution process of an agentic task. As shown in figure | given a task 9. tool execution involves two stages: locating the input index ir (e.g., stock data website) and operating the tool I on it (e.g., browser accessing the website). Executing with in yields the context C' (e.g., stock price data), from which the L.L applies the relation R specified in the task (e.g., identifying the highest stock price) to derive the final answer a.
An agentic task can thus minimally defined by an input index ir and a relation R over the tool- execution context. Since R depends on the retrieved context C, the tool must be executed before the answer can be derived.
by: "Ampl2225Ammalleppoor
Tool List
Fisancial Highlight
In the 2025 fiscal year. Apple's
Web Teok
reached $383.3 billion.
201 Fesk
ueing
peevious year.
"In report Apple2025Ammal
Net profit was 594.7 billion. with a gress morgin 44 3%,
Conscal
A: "vosal
Report, what
in 2025?"
Openning eash flow from openations
Muasc
What's the enswers context content ie the PDF?
Capple202 wwnnuannnnnnaaaa Report..p.. pdd
billion.
What the relationship?
the tasi?
Atomic task generation. From an unlabeled corpus, we extract ir and derive textual content C' via tool execution. LLM identifies candidate answers from C, infers their relationship R, and constructs question conditioned on iT and R.
Figure 2:
Atomic Task
An atomic task is resolved with a single target tool invocation. To simplify, we disre- gard search and file system operations, assuming detailed input index ir enables retrieval through finite navigation.
Given answer a, the most direct approach to construct atomic task involves prompting LLM to generate the corresponding question. However, questions produced in this manner often suffer from low tool invocation rates, unpredictable difficulty levels, unregulated tool requirements, and inconsistent verification complexity (see section 4.4 for more details).
To address these issues, we assume ideal search engine that retrieves precise data based on it (e.g., paper titles, song titles). Under this assumption, we define a task as (q, a) = (fg R), a), where fg is a sampling function guiding the LLM generate natural language form by using ir and R.
3 AUTOMATED TASK GENERATION WORKFLOW
In this section, we describe our task construction workflow, which proceeds in three stages: (1) generating atomic tasks as the foundation, (2) progressively extending them to increase complexity, and (3) verifying their validity through efficient checks.
3.1 ATOMIC TASK GENERATION
As figure 2 shown, we begin by compiling corpus of unlabeled data aligned with the tool's input requirements. From this corpus, we extract ir and derive textual content C' via tool execution. For example, browsing, PDF, and image comprehension tools yield webpage titles, PDF names, and image paths, from which extract textual content C for answer sampling. We prompt an LLM to identify key candidate answers a from and infer their relationship with C, ultimately constructing question & conditioned on ir and R.
3

---

Published as a conference paper at ICLR 2025
The merged task
Who director the
step4 o² 0'=00P00
science fiction film, which was released Novermber
2014?
Interstella's
What science liction Gil was relessed Nov 20147
direclor?
step2
step3
Interstellar
T: science ficiion film
science fiction
The direstor Interstellar
step1 search agent
RC. science fiction film peleased Nov 2014
& science fietion film releused Nov 7, 2014
Christogher
Nolan
Interstellar
of Interstellar
Atomic task (first hop)
The second-hog task
(a) Depth-based based extension
LLM Marging
Nh, Whhat was Apple
42 Apple P.E period?
?
dy What were Apple Inc. Is QT 2025 EPS and P.E ratio the same pariod!
Line is Q! 2025 EPS?
Execute Tool
0y: 39.65
0; $2.40, 39.65
(b) Width-based extension
Figure 3: Strategy for task extension
3.2 TASK EXTENSION
In order to increase task difficulty in a scalable way. we adopted two extended task strategies: the basth extension and the width-bast baaeh extension. (see Appendix prompts details)
Depth-based extension. We aim construct tasks requiring multiple sequential tool executions, where each step depends on the output of the previous one. To achieve this, a new sub-task must be derived from known task q" The tool input index in at each stage exhibits strong extensibility due to (1) its frequent association with proper nouns, which are less likely to be memorized by LLMs, and (2) its natural suitability for recursive definition. Specifically, n-hop task (q", a)), consisting of a question g" and its corresponding answer a, is formulated as follows:
(q" a) ("), a)),
(1)
To extend a n-hop task q" into a (n+ hop task t we first find intermediate sub-task:
(g (4) (f, (1 (("+1), 14)
(2)
Here, 17+11 (e.g., song title) represents new index derived from iž (e.g., a fragment of the song's lyrics) through reversible operations. To achieve this, a search agent identifies the title of superset of iz. the web within the file system and uses it ex+1, We instruct the agent to search for superset to reduce the risk of cyclic generation.
During the search, the agent retrieves the supersets text content C (e.g., the complete lyrics of the song) with search tools. An LLM then analyzes C' to infer its relationship R2-1 to it (e.g.. that the fragment corresponds the third line of the lyrics).
Using this intermediate task, we can define the recursive formulation to obtain the (n+1)-hop task:
t1 a) (Jm(q", gn (77,)))
where fm is a function that guides the LLM to generate q in natural language by substituting i7 in q² with g=+1.
4

---

Published as a conference paper at ICLR 2025
Width-based extension. The goal of the width-based extension is generate new task that needs to be decoupled into multiple sub-tasks to be completed. For simplicity, for two sub-tasks 91 + d1 and 92 az the combined task Swidth can be represented as
(uoidth q1 92) - ai 02.
where the + indicates using LLM to merge and rephrase two question strings.
Trajectory generation. Two strategies exist for generating execution trajectories in this task: (1) For simple tasks, such as atomic tasks, existing agents can directly infer and capture the trajectory, including tool selection, parameters, return results, and plans. (2) For complex tasks, such as depth- wise extension tasks, the sub-task trajectory is recorded while iteratively expanding and validating new atomic tasks.
3.3 TASK VERIFICATION
To ensure that the generated tasks demand agentic reasoning and that each expansion is effective, verification is performed after every step. Within this workflow, task verification can be carried out in two phases:
Atomic fask verification; An atomic task is defined simple agent task solvable via single tool call. During verification, we relax this defimition slightly: for each candidate task, we evaluate the task agent's output within a limited number of tool-use steps (e.g., three) and compare with an infer-LLM separately. A judge-LLM verifies whether only the agent's 's output contains the golden answer, retaining only validated tasks. (see Appendix B for more details)
Task extension verification: This process is conducted purely through linguistic analysis without agent involvement. During depth-wise extension, we first employ a judge-LLM to validate: (1) whether the obtained 17 and its relation R=+1 constitute proper superset of 2 with logically sound relationships, and (2) whether the final input index i in g" is appropriately replaced by gu+1 in the expanded task q" q=+11 Furthermore, an infer-LLM derives the merged task, while the judge- LLM filters out tasks where the correct result is easily inferred, preventing information leakage that could render the task trivially solvable after merging. (see Appendix D for more details).
This framework ensures efficiency by applying agent reasoning only in atomic task verification creation, while relying on LLM-based verification elsewhere for faster execution. also enables complex task generation beyond agent capabilities, with reverse reasoning providing supervisory signals to enhance agent learning or reinforcement learning.
EXPERIMENTS
4.1
CORPUS CONSTRUCTION
Exilinn
1556
Acadomic
Table 1: Human evaluation for the generated tasks.
Govemmmett 1556
Web Took
75%
Linguistic fluency
91.7%
Accuracy
95.0%
Anage
10%
Report
44
Atomic
Single answer
83.3%
Paper
Information leakage
11.7%
Paper
Depth-hased extension
Extended validity
82.3%
policca
Non-superset
8.5%
TN
Figure 4: Corpus source distribution.
5

---

Published as a conference paper at ICLR 2025
Agent Execution Step
Tisk I: OpenAT's 2025 release, there's security doe for now gen models. It mentions SWE Lancer Dissonn benchmerk.
Answer:
What's the highest value? Which model & config resch
In April 2025
OpenAl securit
Step 1- OpenAT's April 2025 relesse, what's the cvalustion doe new - gan models?
does,
A: OpenAl of and o4-mini Systess Card
Laneer Diamond
(SWF Marager
Step What
SWE-Lance Dismond benchmmrkk the OpesAl o4-mina System include PDF context some inages.
Por
lask) eval,
modal's pessigt raee was just
Step 3: Q In Fig. 21, what's the highes (passgi 1) secumey! Which models & confige nchieve it?
Top passig] accuracy: (ochieved by deep and browsing launch candidates).
Image Read
0. 0.14/4%% much
expected.
Answer: la OpenAl's 2025 & of mini System Card, the Lancer Diamand benchmark shows highest Pasege3 of 50% by 03, deep research, and oi-mini Browsing lamnch contigs.
Agent Esecution
LUM
Task For the classic Disncy animated aeries festuring alien experiment chameter and Howaii, ormiened around the family whet livo-action spin-off officially relessef?
Answer:
thane allin
live Stith! sec to premiere on Dieneyt 2014.
Step 1: What the classie Disney anamaned seriss about an aliss experiment set Hawai??
A: lilo & Stitch
by that year, Disney hada't announeed release date.
Step 2: :
live action spin movie of "Lilo be officially released! A: The live actioe "Lilo & Sanch' mevio is scheduled be released on May 23, 2025.
Answer: The official date the Disney live-acton adapiationn of Sunch 23, 2025
Figure 5: Generated case examples requiring multiple tool calls for completion.
We collect seed documents across modalities to generate tool-specific atomic tasks, extracting key insights for relevance. For instance, our PDF processor constructs atomic tasks by combining titles with core findings, enhancing the need for agent-based PDF tool invocation. To support atomic task generation, we constructed a dataset comprising webpages, PDF files, and images. Webpage data constitutes the largest proportion (75%), sourced from up-to-date news across multiple domains. Image data accounts for 15%, primarily derived from financial reports and research papers, with filtering to retain images containing information beyond text. PDF data makes up 10%, originating from English financial documents and academic publications.
Human Evaluation. To verify the validity of the results, randomly sampled 60 atomic tasks and 48 depth-based extension tasks using human evaluation and scored them. As shown in Table these results highlight the overall effectiveness and controllability of task generation.
4.2 ENHANCING TASK GENERATION EFFICIENCY VIA PROMPT LEARNING
As noted in section 3.3 both atomic task generation and task extension require validation, with any failure leading to rejection. Reducing rejections and improving efficiency requires refining four prompt designs that substantially affect the rejection rate:
How to extract candidate answers from the corpus for atomic task generation (section 3.1
In depth-wise extension:
how the agent identifies the next input index (7 relevant to the current document and avoid eyclic reference;
how to prompt the LLM to generate a relation R=+1 so that the answer can be uniquely derived from the context;
How to integrate extended tasks and increase the complexity of existing questions,
i.e. 2+1 a) (fm(g", gg+++........ a), while maintaining clarity and coherence.
We adopt bootstrap few-shot learning Khattab et al. (0224) to optimize the four prompts. For atomic task generation, the prompt is augmented with 20 randomly sampled examples. Multiple candidate prompts are evaluated, and the one yielding the highest pass rate is selected. For task extension, focus on depth-wise augmentation and apply the same strategy using 10 sampled examples, refining the prompts to maximize the number of reasoning hops.

---

Published as a conference paper at ICLR 2025
To enhance the LLM's capability in identifying intermediate objectives, employ bootstrap few- shot learning Khattab et al. (0244) to systematically optimize four prompts corresponding to key challenges. Each prompt for atomic task generation is enhanced by appending 20 randomly sampled examples. Various prompt configurations are evaluated iteratively based on pass rates to select optimal examples. For depth-based extension, optimize prompts using 10 randomly sampled examples, refining them to maximize task complexity.
Table 2: Effectiveness of generated task data in prompt learning and depth-wise extension. The pass rate denotes the proportion of atomic tasks that successfully pass validation out of all generated candidates. For disth-wise extension, the pass rate is defined as the fraction of successful extensions out of six attempts.
Method
Pass rate
Time
Atomic Task
54.9%
29.1s
Optimization
68.1% %
23.5s
Depth-wise@r* Optimization 51.2%
41.0%
31.5s
30.2s
Table examines atomic task generation and depth-wise task extension before and after prompt learning, highlighting the role of generated tasks in enabling self-evolution within the workflow. These results validate the effectiveness of generated task data enhancing sampling efficiency and supporting workflow adaptation. The optimized prompts are presented in Appendix E.2
4.3
AGENT FOUNDATION MODEL TRAINING
Method
SET RL
GAIA (5)
WehWalker
BrowserComp
HLE
Qwen-2.5-7B- 2.mm-2.577BInssttuuc
R1-Searcher Song et 2025
20.4
WebSailor Li M..
37,9
6.7
5k MHQA
18.6
20.2
4.5
3.6
7.5% MHQA
20.4
23.4
4.2
7.5k TaskCruft
36.3
55.0
12.4
16.4
5k MHQA + 2.5k TaskCraft
34.0
$2.6
6.4
13.2
5k MHQA 2.5k TaskCraft (SFT) + 8k TaskCraft (RL)
40,8
13.4
16.0
7.5k MHQA
21.6
28.6
3.6
9.6
5k MHQA 2.5k TaskCraft
33.0
59.4
7.6
12.8
QwQ+32B
Scarch-o Li 2025b
39.8
34.1
SimpleDeepScarehee Sun et
2025
50.5
WebSailor Li al. 2025a
50.5
WehThinker
WebDancer
Wa 21.
$1.5
43.2
2.8
Scarch-o Li 22025b
2025bb
28.2
SimpleDeepXeareheei
2025
40.8
WebSailor Li et al.
2025a
v
53.2
10.5
5k MHQA
38.8
36.8
5.6
10.8
7.5k MHQA
12.6
7.5k TaskCraft
60.2
22.4
20.2
5k MHQA + 2.5k TaskCraft
60.2
21.0
20.0
5k MHQA 2.5k TaskCraft (SFT) 8k TaskCraft (RL)
60.8
24.8
20.6
Table 3: Performance across agentic task benchmarks. Methods are grouped according to the base model adopted.
To validate the effectiveness of our synthetic tasks, we apply SFT and RI. to refine a tool tool-integrated reasoning (TIR) model in agentic scenarios. In TIR, the LLM output is trained with explicit tags such as <tool>, <observation innvatiionoo or <think>, which structure the reasoning flow and trigger cor- responding tool calls. We conduct experiments using models from different families and scales, evaluating their performance on the GALA Mialon et al. (2023). WebWalker Wu et al. 2025b)
BrowserComp Wei et al. (2025). and HLEPPan et al. (2025).
For SFT learning, we sample solution trajectories for each task using Oagents Zhu et al. (025)) and convert them into the TIR model format. To ensure the performance gains not merely due

---

Published as a conference paper at ICLR 2025
to learning the output format, use two types of training data: 7.5k tasks sampled from existing multi-hop QA datasets (denoted as MHQA, including HotpotQA and NQ), and 7.5k synthetic tasks via pipeline. To further enhance model performance, incorporate additional generated data and apply DAPO Yu et al. (2025) for continued RI. training.
As shown in Table 3 pure 7.5k TaskCraft outperforms 7.5k MHQA across all benchmarks. Fur- thermore, replacing 2.5k MHQA with 2.5k TaskCraft produces 5-16x larger gains, far exceeding the improvements obtained by adding the same amount of MHQA. Even without RL, TaskCraft- trained models already match SOTA systems that rely on both SFT and RL. When scaled with more TaskCraft tasks and RL, performance further improves, reaching new SOTA. For example, on Web Walker, our Qwen-2.5-7B- Instructt exceeds the previous best-including the much larger QWQ-32B by a substantial margin. These results confirm that TaskCraft is highly scalable and substantially enhances agent model performance, enabling models to reach SOTA levels.
4.4 EFFECTIVENESS OF TOOL. CONTEXT IN CONSTRUCTING AGENTIC TASKS.
In atomic task generation, we incorporate the input index äT and the tool-answer relation R to structure tasks. To evaluate its efffcttiveness, we conduct ablation study where LLM directly generates single-tool tasks g without using ir or R. We assess performance via pass rate, resolution time, average tool usage, and usage variance.
Table 4: The effectiveness of tool context.
Method
Pass
Time
#Tool-use
o²
LLM only
18.5%
119.7s
2.8
Ours
43.0%
86.7s
2.1
Compared to direct GPT-4.1 prompting, our method significantly improves atomic task generation, achieving higher success rates and faster task construction. produces more atomic and consistent tasks, with fewer and more stable tool invocations, highlighting the limitations of vanilla LLMs in agentic task design and the robustness of our structured workflow.
4.5 SYNTHETIC TASKS ANALYSIS
Agent reasoning analysis. To practically assess task difficulty, we sample 1,000 tasks and deploy both Smolagents Roucher et al. (2025) and Oagents Zhu et al. (025). for execution and validation. While both agents performed identical tasks, Oagents incorporated advanced tool capabilities for refindd analysis.
100
Dagents
13.0%
Scom 2 scre)
30
49.2%
32.4%
55.0%
Percentage
77.9%
40
60.2%%
7%
15.8%
30.3%
htmi
pof
Image
Figure 6: score distribution comparison
Responses were evaluated by comparing the agents' outputs to the golden answer, following three- point scoring scheme: 2 for fully correct responses, for answers that included the golden answer but contained additional information, and for incorrect responses.
In figure 6
task failure rates increase from web pages PDFs and then images within PDFs, indicating that multi-hop web search tasks are more manageable for agents, while complex compre- hension challenges, such PDF extraction and image interpretation, remain difficult. Additionally,
8

---

Published as a conference paper at ICLR 2025
these results demonstrate that our generated tasks span varying difficulty levels, including those that pose significant challenges for current agent capabilities.
Scalability of Task Craft data. To evaluate TaskCraft's scalability, we trained Owen2.5-7B-insttruc models on 1k. 3k, and 5k randomly sampled tasks, using consistent settings and tested them on GAIA-103. As shown in Table 5 the results exhibit clear upward trend, suggesting that larger TaskCraft training sets yield progressively better performance.
Table 5: GAIA performance by data size.
Table 6: Accuracy comparison of Smolagents on GAIA and synthetic tasks.
Data Size
@ 3 on GAIA-103
Levell
Level2 43.02
Level3
Avg.
1,000
17.5%
GAIA
54.71
26.92
44,20
3,000
5,000
39.8%
31.1%
Synthetic Task
PDF
54.4
html
50.7
Image
22.1
Avg.
42.4
Comparison with the GATA Dataset. compeees Smolagent's accuracy on the GATA bench- mark and our generated dataset. The results show that tasks derived from diverse tool corpora reflect GAIA's stratified difficulty levels, with image understanding tasks presenting the greatest challenge and yielding accuracy comparable to Level 3. Unlike GAIA, which relies heavily on manual anno- tation, our framework automates task eneeration-elmmiatting the need for labor-intensive labeling while preserving scalability and adaptability for agent self-evolution and optimization.
5 RELATED WORK
5.1 INSTRUCTION DATA GENERATION
Synthetic data has emerged as a promising solution for enhancing performance and enabling new capabilities. STaR Zelikman et al. 2024) augments learning with chain-of-thought (CoT) ra- tionales but often requires substantial number of task queries beforehand. Methods such as Self-Instruct Wang et al. (2022). Self-Chat Xu et al. (2023b), Numinaaath Li et al. (2024). and al (2024) generate data from minimal seed examples using LLMs, yet they struggle to extend task generation for multiple tool invocations. WizardLM al. (2023a) employs Inst-unstt to incrementally enhance instruction complexity. However, it relies primarily on rule-based modifications, making its generated instructions unsuitable for agentic task scenarios. MetaMath Yu et al. (2023) generates mathematical data by rewriting questions, but adapting agent tasks to environmental feedback presents challenges beyond simple rephrasing. WebInstruct Yue jet al.] (2024) extracts question-answw
OpenMathInstrrut-2 Toshniwal Toshhri
pairs from a pre-training corpus across multiple domains; however, the generated questions often fail to incorporate tool utilization. AutoAct(Qiao et al. (2024) uses self-planning mechanism to generate planning trajectories for QA tasks.
5.2 LANGUAGE AGENT
Existing research agentic task execution advances along two main axes: role specialization and functional partitioning. Role-based approaches, such as AutoGPT Significant-Gravve (2023), Au- toGen Wu et al. (2023). and Camel Li et al. (2023), organize collaborative agents by dynamically assigning tools. In contrast, frameworks like Barcelona2, Omne, and AgentIM] adopt functional partitioning optimize modular efficiency. SmolAgents Roucher al. (2025) integrates ReAct Yao et al. (2023) and CodeAct Wann et al. (2024b) into hierarchical agent system for iterative code- based task execution. Magnetic-One Fourney et al. (2024) enhances multimodal performance by decoupling perception Yang et al. (2023a b). planning Song et al. 2023): Tordesillas & How (2021), and execution Qin et al. (2024): Wang et al. (2024b) modules. Dynamic orchestration mechanisms address real-time adaptation and robustness. Trase-Agent Trass] (2024) adapts strategies based on feedback, while TapeAgents Babdannuu et al. (2024) uses asynchronous communication to improve coordination. Studies show that stable sub-agent interactions outperform complex centralized or- chestration. To advance autonomy, AutoAgent Tang et al. (2025) supports no-code agent customiza- tion via natural language coordination, modular workflows, and self-me managing file systems. Hybrid
These are closed-source frameworks.

---

Published as a conference paper at ICLR 2025
systems like h2oGPTe-Agemt H2 (2024) explore multi-agent optimization, achieving strong sults code generation, though cross- modal bottlenecks remain a challenge.
CONCLUSION
We introduced TASKCRAFT, an automated workflow for scalable, multi-tool, and verifiable agentic task generation. By applying depth-based and width-based extensions atomic tasks, the frame- work constructs hierarchically complex challenges with incremental validation. To enhance sam- pling efficiency, we incorporated a self-evolving prompt optimization strategy inspired by boot- strap few-shot learning. Experiments across multiple LLMs demonstrate that TaskCraft-genan generated data significantly improves multi-hop reasoning and agentic capabilities, achieving performance comparable to state-of-the-art RL models using only SFT. Further scaling and RL fine-tuning with TaskCraft tasks yield additional gains, culminating in state-of-the-an results on four agentic bench- marks. The final dataset comprises 41,000 tool-intensive tasks spanning diverse difficulty levels, including 12,600 coll-inerraction trajectories and 5,000 multi-hop decompositions.
ETHICS STATEMENT
This research adheres to the ethical guidelines outlined by the ICLR conference. The proposed methods do involve human subjects, personally identifiable information, or sensitive data. All datasets used are publicly available and widely adopted in the research community. No data was collected from vulnerable populations, and no deceptive practices were employed.
The model outputs were evaluated for fairness and robustness. We conducted thorough error anal- ysis ensure that the system does not propagate harmful biases or stereotypes. Where applicable, mitigation strategies were applied to reduce unintended consequences.
Our work does not pose foreseeable risks to individuals or society. We acknowledge that any de- ployment of agentic systems should be accompanied by safeguards to prevent misuse. We encourage future researchers and practitioners to consider the broader societal impact of autonomous agents and to adopt responsible AI practices.
REPRODUCIBILITY STATEMENT
To support faithful replication, will release all artifacts referenced in this paper. Specifically: (i) the complete TASKCRAFT workflow code, encompassing atomic task generation, depth/width extensions, incremental validation, and rejection sampling, along with all associated prompts, tem- plates, and data pre-/post-processigg scripts; (ii) the full synthetic dataset comprising 41,000 tool- intensive tasks, formatted in a standardized JSON schema; and (iii) 12,600 tool-interaction trajecto- ries and 5,000 multi-hop decompositions, to be released subsequently.
REFERENCES
Dzmitry Bahdanau, Nicolas Gontier, Gabriel Huang, Ehsan Kamalloo, Rafael Pardinas, Alex Piché, Torsten Scholak, Oleh Shliazhko, Jordan Prince Tremblay, Karam Ghanem, Soham Parikh, Mitul Tiwari, and Quaizar Vohra. Tapeagents: holistic framework for agent development and opti- mization, 2024.
Adam Fourney, Gagan Bansal, Hussein Mozannar, Cheng Tan, Eduardo Salinas, Friederike Niedt- ner, Grace Proebsting, Griffin Bassman, Jack Gerrits, Jacob Alber, et Magentic-one: A gener- alist multi-agent system for solving complex tasks. arXiv preprint arXiv:2411.04468, 2024.
a2..ai. Autonomous agentic ai: execute multi-step workflows autonomously. [Online], 2024.
https: ai/platform /etper/rr.see-n2pptt////tggeeeee http:/hh......aaoooeeeeeeeeeeee/
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training 1ims to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.095 2025a.
10

---

Published as a conference paper at ICLR 2025
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Serean Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training Ilms to reason and leverage search engines with reinforcement learning, 2025b. /RLL/tts/////xxiorrrr http://..rrrab///55/.//// 250.099166600
L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement learning: A survey, 1996. URL https://arxix
Omar Khattab, Arna Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vard- hamanan, Saiful Haq, Ashutosh Sharma, Thomas Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. Dspy: Compiling declarative language model calls into self- improving pipelines. 2024.
Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Com- municative agents for" mind" exploration of large language model society, Advances in Neural Information Processing Systems, 36:51991-52008, 2023.
Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang, Ziju Shen, et al. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13:9, 2024.
Kuan Li. Zhongwang Zhang. Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baix- uan Li, Zhengwei Tao, Xinyu Wang, et al. Websailor: Navigating super-human reasoning for web agent. arXiv preprint arXiv:2507.0259)) 2025a.
Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-ol: Agentic search-enhanced large reasoning models. arXiv preprint arXiv:2501.05336, 2025b.
Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yutao Zhu, Yongkang Wu, Ji-Rong Wen, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability. arXiv preprint arXiv:2504.2177. 2025c.
Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: benchmark for general ai assistants. In The Twelfth International Conference on Learning Representations, 2023.
Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, Michael Choi, Anish Agrawal, Arnav Chopra, Adam Khoja. Ryan Kim, Richard Ren, Jason Hausenloy, Oliver Zhang, et al. Humanity's 's last exam, 2025. URL /ttp:////aaii..oor///////11111 14249.
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10. 2023, pp. 5687-5711. Association for Computational Linguistics, 2023. doi: 10.18653/V FII...IIIG FINDINGS-EMNLP.375 URL https:///oii.rrg org/1e 18653/ v1/2023 findings- emn emnlp. 378
Shuofei Qiao, Ningyu Zhang, Runnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Jiang. Chengfei Lv, and Huajun Chen. AutoAct: Automatic agent learning from scratch for QA via self- planning. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3003-3021, Bangkok, Thailand, August 2024. Association for Computational Linguisties. doi: 10.18653/w1/2024.ace/ 165.
acl- org 2024 http:/222.accllooooe acl--oog........ 165/
Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Xuanhe Zhou, Yufci Huang, Chaojun Xiao, et al. Tool learning with foundation models. ACM Computing Surveys, 57(4):1-40) 2024.
Aymeric Roucher, Albert Villanova del Moral, Thomas Wolf, Leandro Werra, and Erik Kau- nismäki. 'smolagents": a smol library to build great agentic systems. https: /ttts:///ithuut com/ wugginggface/mmllagentt 2025. 2025.
11

---

Published as a conference paper at ICLR 2025
Timo Schick, Jane Dwivedi-Yo Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539- 68551, 2023.
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugging- gpt: Solving tasks with chatgpt and its friends in hugging face. Advances Neural Information Processing Systems, 36:381 154-38180, 2023.
Significant SignificantGGraiitas Autogpt. [Online], 2023. http:://ggttuboooooo/a/aaaaataaa
AutoGPT
Chan Hee Song. Jiaman Wu, Clayton Washington, Brian M Sadler, Wei-Lun Chao, and Yu Su. Lim-planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings of the IEEE/CVF international conference computer vision, pp. 2998-3009, 2023.
Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-searcher: Incentivizing the search capability in llms via reinforcement learning. arXiv preprint arXiv:2503005592 2025.
Shuang Sun, Huatong Song, Yuhao Wang, Ruiyang Ren, Jinhao Jiang, Junjie Zhang, Fei Bai, Jia Deng, Wayne Xin Zhao, Zheng Liu, et al. Simpledeepsearcher: Deep information seeking via web-powered reasoning trajectory synthesis. arXiv preprint arXiv:2505. 16834, 2025.
Jiabin Tang, Tianyu Fan, and Chao Huang. Autoagent: A fully-automated and zero-code framework for I1m agents. arXiv e-prints, pp. arXiv-2502, 2025.
Jesus Tordesillas and Jonathan How. Mader: Trajectory planner in multiagent and dynamic envi- ronments. IEEE Transactions on Robotics, 38(1):463 2021.
Shubham Toshniwal, Wei Du Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Gitman. Openmathinstructt2: Accelerating ai for math with massive open-source instruction data. arXiv preprint arXiv:2410.01550 2024.
Trase. Meet trase systems. [Online], 2024. http://wwww.rrasssstssssssss com/
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving re- reasoning for knowledge-intensive multi-step questions. In Anna Rogers, Jordan L.. Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61 Annual Meeting the Association for Computational Linguistics (Volume I: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pp. 10014-10037. Association for Computational Linguistics, 2023. doi: 10. 11233//1120233ACL-- ACL-LONG.557, URL /0 wttp///....113///22222222 acl-long.
trieval with
Liang Wang. Nan Yang, Xiaolong Huang, Binxing Jiao. Linjun Yang, Daxin Jiang, Rangan Ma- jumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training, 2024a.
URL httpt:t/t///iiii www.l.orrg/abaae abs// 221.. 03533
Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Exe- cutable code actions clicit better Ilm agents. In Forty-firs International Conference on Machine Learning. 2024b.
Yizhong Wang, Yeganch Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannanch Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.2 10560, 2022.
Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung. Alex Tachard Passos, William Fedus, and Amelia Glaese. Browsecomp: simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:2504. /2516, 2025.
Jialong Wu, Baixuan Li, Runnan Fang, Wenbiao Yin, Liwen Zhang, Zhengwei Tao, Dingchu Zhang, Zekum Xi, Gang Fu, Yong Jiang, et al. Webdancer: Towards autonomous information seeking agency. arXiv preprint 2505:22505.22644 2025a.
12

---

Published as a conference paper at ICLR 2025
Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, Linhai Zhang, Yulan He, Deyu Zhou, Pengjun Xie, et al. Webwalker: Benchmarking llms in web traversal. arXiv preprint arXhv:250107572, 2025b.
Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. Autogen: Enabling next-gen Ilm applications via multi- agent conversation. arXiv preprint #rXiv23388.0815, 2023.
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244, 2023a.
Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. Baize: An open-source chat model with parameter-efficient tuning self-chat data. arXiv preprint arXiv:2304.01 196, 2023b.
Dingkang Yang. Kun Yang. Yuzheng Wang, Jing Liu, Zhi Xu, Rongbin Yin, Peng Zhai, and Lihua Zhang. How2comm: Communication efffiictttti and collaboration collaboration-pragmatie multi-agen percep- tion, Advances in Neural Information Processing Systems, 36:25151 11-25164, 2023a.
Kun Yang. Dingkang Yang, Jingyu Zhang, Hanqi Wang, Peng Sun, and Liang Song, What2comm: Towards communication-efficiii collaborative perception via feature decoupling. In Proceedings of the s1 ACM international conference multimedia, pp. 7686-7695, 2023b.
Zhilin Yang. Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. Hotpotqa: A dataset for diverse. explainable multi-hop question an- swering. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun iichi Tsuji (eds.). Proceed- Empirical Methods in Natural Language Processing, Brussels,
ings of the 2018 Conference Belgium, October 31 November 2018, pp. 2369 2380. Association for Computational Lin- guistics, 2018 doi: 10.18653/V I/D1/ 1259. httLhhttps://ddoo org/ /0tgg//0... 18653/v1/d18-- 1299
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhen- guo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309. 12284, 2023.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503. 14476, 2025.
Xiang Yue, Tianyu Zheng, Ge Zhang, and Wenhu Chen. Mammoth2: Scaling instructions from the web. Advances in Neural Information Processing Systems, 37:90629-90660, 2024.
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D Goodman. Star: Self-taught reasoner bootstrap- ping reasoning with reasoning. In Proc. the 36th International Conference on Neural Information Processing Systems, volume 126. 2024.
Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui, Tiannan Wang. Zhenxin Xiao, Yifan Hou, Ryan Cotterell, and Mrinmaya Sachan. Recurrentgpt: Interactive generation of (arbitrarily) long text, 2023a. Uttpsttps://arxv wwwprvv...rggbb/////ee 13304.
Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li, Jialong Wu, Tiannan Wang, Shi Qiu, Jintian Zhang, Jing Chen, Ruipu Wu, Shuai Wang, Shiding Zhu, Jiyu Chen, Wentao Zhang, Xiangru Tang, Ningyu Zhang, Huajun Chen, Peng Cui, and Mrinmaya Sachan. Agents: An spen-source framework for autonomous language agents. 2023b. URL sttps://arxiw.org/abss. 07870
Wangchunshu Zhou, Yixin Ou, Shengwei Ding, Long Li, Jialong Wu, Tiannan Wang, Jiamin Chen, Shuai Wang, Xiaohua Xu, Ningyu Zhang, Huajun Chen, and Yuchen Eleanor Jiang. Symbolic learning enables self-evolving agents. 2024. URL htttps://arxiw.org//baas 18532
13

---

Published as a conference paper at ICLR 2025
He Zhu, Tianrui Qin, King Zhu, Heyuan Huang, Yeyi Guan, Jinxiang Xia, Yi Yao, Hanhao Li, Ningning Wang, Pai Liu, Tianhao Peng, Xin Gui, Xiaowan Li, Yuhui Liu, Yuchen Eleanor Jiang, Jun Wang. Changwang Zhang, Xiangru Tang. Ge Zhang, Jian Yang, Minghao Liu, Xitong Gao, Jiaheng Liu, and Wangchunshu Zhou. Oagents: An empirical study of building effective agents, 2025. URL
15741
14

---

Published as a conference paper at ICLR 2025
A USE OF LLMS
Large language models (LLMs) are used in this work exclusively for text polishing and language refinement during the writing process. Specifically, LLMs assist in improving the fluency, clarity, and conciseness of the writing.
LLMs are not used for any aspects of experimental design, methodological development scientific interpretation. All scientific contributions and innovations presented in this work are entirely human- originated.
B DATA STATISTICS
Domain
web tool
50
Percentage
40
30
Figure 7: Analysis of all tasks.
As illustrated in figure 7 task generation exhibits hierarchical decay pattern across all domains as hop count increases, revealing distinct scalability trends:
pdf tool tool domain: Shows gradual performance attenuation with hop depth, 1 1-hop tasks accounting for 60.13 13.13% (8, 1,111 tasks), decreasing 13.49% (1,820 tasks) for 2-hop and 11.22% (1,514 tasks) for 3-hop. The sharp drop in 5-7 hop tasks (6.94% combined) indi- cates limited deep-extension capability, yet surpasses other domains in depth scalability.
image_ton tool domain: Presents the most pronounced performance decay, with 1-3 hops com- prising 87.10% (7.125/8, 180 tasks) but only 5.71% (467 tasks) for 5-7 hops, highlighting fundamental constraints in deep hierarchical task generation.
webltool domain: the web.tool domain, I-hop tasks dominate, constituting 70,01 % (13,467 tasks) of the total. However, this domain also has the highest absolute number of deep extensions, with 5-7 hop tasks accounting for 5.66% (1,089 tasks).
Cultual
Academic 2/99y
Grvammee
Paper AR
91%
Recert L4ft
Papar
Shs
Figure 8: Distribution of atomic data.
Atomic task analysis. We collect data from webpages, PDF files, and images to support the gen- eration of atomic tasks, which form the basis of the dataset, totaling 26,527 instances as shown in figure R
15

---

Published as a conference paper at ICLR 2025
Among them, atomic conclusions from web-based tools account for the largest proportion. reaching 50.77%, with sources spanning multiple domains: academic (27.11%) cultural (6.42%). economic (5.36%) and governmental (5.05%) resources. These derive from up-to-date news and curated online materials for relevance.
Image-based tools contribute 18.64% of the data, extracting structured insights (e.g., key trends, comparisons) from charts/tables financial reports and research papers. Strict verification excludes conclusions directly replicating source text avoid redundancy.
PDF-based extraction accounts for 30.59%, supplementing the dataset with findings from financial reports and academic publications. This multi-source approach enhances diversity while maintain- ing consistency in atomic fact representation.
By systematically integrating these extraction methods, we ensure high-quality task generation, pro- viding a robust foundation for downstream model training and optimization.
C EXPERIMENTS ON MULTI-HOP QA TASKS
We first evaluate our models across three established multi-hop question answering benchmarks: HotpotQA Yang et al. (2018), Musique Trivedi et al. (2023), and Bamboogle Press al. (2023). These datasets present diverse challenges reasoning and search, providing a robust evaluation platform.
We compare the baseline workflow (Search-R1 Jin et al. (002b)), which leverages reinforcement learning for LLM model optimization) with the agent workflow after applying SFT using the gener- ated tasks.
Method
HoipolQA
Musique
Barbbugle
Avg.
Qwen2.5-3B- Base
Search-R1
0.284
0.049
0.088
0.140
+ SFT 0.344
0.111
0.280
0.245
Qwen2.5-3B-Instree
Search-R1
0.324
0.103
0.264
0.230
+ SFT 0,340
0.104
0.264
0.236
Table Performance across three datasets and two models. Avg. denotes average.
As shown in Table our synthetic data proves valuable in SFT training, showing average per- formance improvements of +14.0% (Qwen2.5-3B (iven2.5-3B-Base and +6.0% --Be...s-B-)sttrrcc))) com- pared their respective base workflows, validating our data generation approach. Compared the Search-RI baseline, the trained model demonstrates substantial improvements. This suggests that our synthetic data not only enhances immediate task execution but also optimizes RL initialization effectively.
D VERIFICATION REQUIREMENTS FOR DEPTH-BASED EXTENSION
Effective n-hop task extension requires rigorous verification to ensure valid multi-hop reasoning. The transformation must preserve superset validity:
(qn+1, (44) = (fg(t) R=+1), it).
(5)
(g a) (fm .mmt1 227)
(6)
Current depth-based extension methods often introduce two critical flaws when replacing tool inputs ir without proper verification:
Pseudo-Superset Task: Superficial substitutions that preserve semantic equivalence but lack genuine superset relationships
16

---

Published as a conference paper at ICLR 2025
Information Leakage: Premature disclosure of information that should only emerge through proper multi-step reasoning
These issues undermine the intended multi-hop reasoning process.
D.1 USEUDO-SUPERSSE TASK
fundamental limitation arises when replacing ir with a semantically equivalent but non-superset index i Consider the following task extension example:
Original task
Input index (ir): Travel Trends 2025 Our Annual Report
Query (q"): How many travel trends for 2022 does "Travel Trends 2025 Our Annual Report' present?
Answer: 5
When the search agent retrieves the superset iT i of ir actually ends up retrieving the synonyms of ir instead. Based on this, an intermediate task is derived:
Intermediate task
New input index (cT+1 ): 2025 Annual Travel Trends Report
Query (gm++);: ); What is the title of 2025 Annual Travel Trends Report?
Answer Travel Trends 2025
Despite valid hop annotations, the intermediate question does not constitute an effective extension: does not represent necessary tool-use step. The issue lies in the absence of genuine superset relationship between ?7 and byr
mn+1 leading to superficial expansion.
Extended task
Query (g"+t)); How many travel trends for 2022 does '2025 Annual Travel Trends Report'
present?
Answer:
5
D.2 INFORMATION LEAKAGE
second failure mode occurs when expanded tasks inadvertently expose original answers, enabling large language models (LLMs) to bypass tool retrieval. For instance, consider the extended task:
Original task
Input index (kr); Sports In Brief
Query (q"): What is the merger value of Charter and Cox in the Sports In Brief?
Answer: 34.5B USD
Intermediate task
New input index (i/++)): ); AP News
Query (g=+1); What the section AP News that updates sports news daily?
Answer Sports In Brief
Extended task
Query (1)++);; In the AP Sports daily summary, Charter and Cox's proposed merger is valued at approximately $34.5 billion. What is the exact amount?
Answer 34.5B USD
17

---

Published as a conference paper at ICLR 2025
While q*+1 appropriately conceals the previous e ("Sports In Brief"), it directly reveals the answer "34.5B USD", allowing the LLM to bypass the intended retrieval process. This compromises the essential tool dependency required for multi-hop task answering.
D.3 VERIFICATION FOR TASK EXTENSION
To address these challenges, we propose a rigorous verification framework to ensure the validity of TT gua+t and q=++1 task extension.
D.3.1 STRICT SUPERSET VERIFICATION
a+1 T
must be the index of strict superset of 27,, and the relationship can be formalized as:
(G"+1, ( (fg 1I=+)), 27.).
where RT+ denotes hierarchical relations (e.g., contains, part.of). Valid extensions must introduce genuine depth, such as "Sports Brief" "AP News" (relation: the part that updates sports news daily), while rejecting synonymous substitutions. Additionally, invalid extensions that allow the LLM derive i directly should be excluded-ensuring the intermediate task qu+1 requires tool use.
D.3.2 INFORMATION LEAKAGE VERIFICATION
( (fm(""""""""""" ,2)),a)
(8)
The extended query n=+ must adhere the information-sealing principle ensure proper tool-use reasoning. This requires that the query does not directly expose the original answer, and any query from which the LLM can directly obtain the answer should be filtered out.
D.4 ADVANTAGES THE VERIFICATION FRAMEWORK
Our approach provides three key advantages:
Superset Integrity: Guarantees valid hierarchical progression (e.g., column - page website) without logical gaps.
Striet Tool Dependency: Enforces authentic multi-hop reasoning by eliminating solution shortcuts, ensuring mandatory tool-use.
Transparent Reasoning: Offers full explainability through explicit relation paths (R")..
Below shows the intermediate extension process from 1-hop tasks to 4-hop tasks, so as to highlight the increasing difficulty of the tasks:
Original Task (n=1)
Input index (it)): 2024's Rising Content and Fastest Growing Skills for 2025 Query (q ); What percentage of non-job seekers see the value Al upskilling according to '2024's Rising Content and Fastest Growing Skills for 2025'?
Answer 49%
Extended Task (n=2)
New input index (: Coursera Blog
Intermediate Query (q)): Which Coursera Blog article covers 2024 content trends and 2025 growing skills, and is easily identifiable on the blog's homepage? Query (q²): Referring to the article on the Coursera Blog that discusses 2024 content trends and 2025 growing skills, and can be uniquely identified from the blog homepage, what percentage of non-job seekers recognize the value of Al upskilling according to its findings?
Answer 49%
18

---

Published as a conference paper at ICLR 2025
Extended Task (n=3)
New input index (tz):: Coursera
Intermediate Query (g)): What is the official information and content update section of the Coursera online learning platform, which is content subset and can be accessed dedicated section on the main Coursera website?
Query (q"): Referring the official information and content update section of the Coursera online learning platform, which is a content subset available as a dedicated section on the main Coursera website and features discussion of the 2024 content trends and 2025 growing skills, and can be uniquely identified from the platform's 's homepage, what percentage of non-job seekers recognize the value of Al upskilling according its findings? Answer 49%
Extended Task (n=4)
New input index (it)): Global online learning platform
Intermediate Query (q"): Which global online learning platform collaborates with over 275 leading universities and companies to offer MOOCs and degree programs, enabling users to access and identify publicly available course content from authoritative educational institutions in one place?
Query (q"): Referring the official information and content update section of the global online learning platform that collaborates with over 275 leading universities and companies provide MOOCs and degree programs, and which offers dedicated content subset section easily identifiable on its homepage-including incluudin discussion the 2024 content trends and 2025 growing skills-what percentage of non-job seekers recognize the value of Al upskilling according to the findings available there?
Answer 49%
E
CORE PROMPTS
This section presents key prompts used in our framework.
E.1 ATOMIC TASK VERIFICATION
The following prompt is used in atomic task verification (Section 3.3
Atomic task verification
Task: Evaluate the consistency between the golden answer (GA) and another answer (AA, either agent or LLM-generated)) as follows:
2 points (Fully Consistent): AA and GA semantically equivalent, even if phrased differently.
...(Examplee...
1 point (Partially Consistent): AA includes all GA information but adds valid extra details.
...(xxappee)...
points (Inconsistent): AA omits key GA information or contradicts it. (...aaapple....
The criteria prioritize semantic equivalence while accommodating informative expansions reductions.
task retained as an atomic task and only the AgentScore strictly exceeds the LLMScore.
E.2 OPTIMIZED PROMPTS
The following prompts is optimized prompt mentioned in (Section 4.2
19

---

Published as a conference paper at ICLR 2025
Atomic Conclusion Extraction
Task: Extract standalone conclusions from document chunks meeting these criteria:
1. Atomicity: Extract only indivisible basic facts ...(Example)...
2. Verifiability: Include least one definite identifier (numerie value, time, unique name) and reject vague expressions .... ...(Exaaplee...
Timeliness Handling: Explicitly mark time ranges for time-sensitive information
...((xxappee...
4. Citation Integrity:
Embed complete content of cited references. ....EExaplee...
Example)...
Depth-wise Extension with 77 in+1 and R
Identify minimal unique superset for an input element based on its attributes, ensur- ing the superset--relationsiip uniquely points the element. ..... (x.EExaplee....
Task:
Relationship expression guidelines:
1. Clearly show hierarchicallonnership. Indicate position for series sub-items; clarify ownership for parts of a superset
2. Specify input content's positioning (e.g., time range, publication field, role in su- perset)
3. Use research/industry standard wording
4. Provide only necessary associations
Example)...
Logical Substitution: (q"+1 ,a)
(fm (4" gr+t1 .R").al
Task: Substitute elements in core queries using auxiliary queries while preserving:
1. Complexity Balance:
The new query should be slightly more complex than the original core Query and require more steps to solve. But do not make too many changes the core query.
2. Answer Uniqueness:
The new query should point to the unique answer: golden answer, and should not point to other answers.
3. Answer Concealment: The new query must reveal information about the golden answer.
4. Natural Language Polish:
After merging, polish the question to make it conform human expression habits without changing the original meaning. Do not modify the proper nouns appearing in it.
(Example)...
E.3 STRICT SUPERSET VERIFICATION
The following prompt is used in Appendix D.3.1
20

---

Published as a conference paper at ICLR 2025
Strict Superset Verification
Task: Verify index + uniquely determines subset ?4 under relation l given queries. Criteria:
1. SupersetSubset Relationship:
2+1 must be the index a superset that properly contains E
T
m+1
# 2 (excluding synonym pairs like CAR/AUTOMOBILE)
2. Relationship Validity:
The relationship R" must explicitly and uniquely link the superset to the sub- set (no many-to-one mappings)
F TOOL DETAILS
Our main tools implemented follows:
Wiki Search Tool: We use the same local WiKi Search tool as Search-RI Jiin et al. (2025a). which uses the 2018 Wikipedia dump Kaelbling et al. (199) as the data source and E5 Wang et al. (2024a) as the retriever.
Web Search Tool: We employ mechanism to access the Google search engine for information retrieval. Specifically, Serpapi (https://googll) serpe search operations. The core parameters configured for Serpapi include the search query string and the specified number of results to be returned. In practice, searches are conducted using queries generated by the model, with the system set retrieve the top 10 results for each query. Each result contains a title, a snippet, and the corresponding URL. This setup furnishes substantial support for subsequent analytical processes and decision- decision-making actions.
serper.rppeeee dwweeeeede///eehhheeeeeeeeee is utilized to execute web
Web Page Crawling Tool: We implement web page crawling tool with content summarization
capabilities. The tool accepts three core parameters: target URLs, web search queries, and reasoning context. Each URL is processed using Jina (https: //jis:://jina ai/) to extract information. We then use the Qwen2.5-72B 722ninttuut model generate summaries for each crawled page. These summaries are concatenated based on the reasoning context to form the tool's output. Importantly, our summary prompt instructs the model to preserve relevant URLs, enabling iterative use of the crawling tool for deeper web exploration.
PDF Tool: We use pdfplumber with multiprocessing read text from PDFs in parallel, and PyMuPDF to extract images from PDFs.
G FURTHER TRAINING DETAIL
For SFT training, we synthesize 3,202 multi-hop tasks and their trajectories and apply content mask- ing to search tool contexts in these trajectories.
For RL training, we follow the Search-R1 Jin et al. (025bb and use the 2018 Wikipedia dump knowledge source and the E5 embedding model as a retriever. For fair evaluation, we fix the retrieval depth to 3 passages for all methods. We merge the training sets of NQ and HotpotQA to form unified dataset. Eyaluation conducted on the test or validation sets three datasets to assess both in-domain and out-of-domain performance. Exact Match is used as the evaluation metric. In the PPO settings, we set the learning rate of the policy L.MM to le-6 and that of the value I.IM to le-5. Training is conducted for 500 steps, with warm-up ratios of 0.285 and 0.015 for the policy and value models, respectively. We use Generalized Advantage Estimation with parameters x= I and = 1. We employ vLLM for efficient LLM rollouts, configured with tensor parallelism degree of 1 and GPU memory allocation ratio of 0.6. Our sampling strategy utilizes temperature parameter of 1.0 and top-p threshold of 1.0. For policy optimization, we apply KL divergence regularization with coefficient ==0.001 and implement clip ratio e0222. The action budget is constrained to 4, with default retrieval depth of passages per query.
21


Reviews:
[{"review_id": "1F6Pw2YRfv", "reviewer": "Reviewer_doXQ", "summary": "Existing agentic task benchmarks such as GAIA, BrowseComp, and HLE are difficult to scale due to their reliance on costly human annotations, while previous self-instruct-style data generation methods primarily target static instructions rather than interactive, tool-using, multi-step tasks. This paper introduces TaskCraft, the first automated workflow for generating scalable, multi-tool, and verifiable agentic tasks with varying levels of difficulty. TaskCraft progressively increases task complexity through depth-based and width-based extensions, while employing incremental validation via rejection sampling and LLM-based linguistic analysis to ensure both scalability and reliability. Experimental results across multiple LLMs demonstrate that TaskCraft-generated data significantly enhances multi-hop reasoning and agentic capabilities.", "questions": "1. Are both the depth-based and width-based extensions generated using a few-shot approach? Does “prompt learning” (line 293) refer to this few-shot generation?\n2. The notation ( $q = f(i_T, R) \\rightarrow a$ ) is unclear.\n3. What prompts are used to extract ( $i_T$ ) and ( $C$ )?\n4. How do you ensure that the obtained ( $i_T^{n}$ ) forms supersets without causing cyclic generation? Is there an ablation study verifying the reliability of the LLM judge?\n5. How does the width-based extension propose new questions? Does it generate targeted, potentially mergeable questions, or are the questions generated randomly and then filtered through verification to retain only those that can be merged?\n6. Is your training and evaluation framework based on open-source code?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Originality: Proposes innovative depth-based and width-based methods for data generation.\n\n2. Significance: Effectively addresses the scalability challenge of agentic data, a key bottleneck in training and evaluating tool-using LLM agents.", "weaknesses": "1. The paper’s method description is unclear. For example, the function $f()$ is used inconsistently across different contexts (e.g., lines 190, 196, 863, and 871).\n\n2. Why not directly use the 7.5k TaskCraft data for SFT training? Could you include an additional experiment comparing its performance with the 7.5k MHQA dataset?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QKVFasFLOp", "reviewer": "Reviewer_WVwZ", "summary": "The paper introduces TaskCraft, an automated pipeline to generate multi-tool agentic tasks that require multi-step reasoning and tool use.\n\nThe method operates in three stages:\n\n1. Atomic task generation: Starting from unlabeled corpora (webpages, PDFs, images), the system extracts tool input indices and derives tasks through a structured formulation q = f(i_T, R) → a\n2. Task extension: Complexity is increased through depth-based extensions (sequential multi-hop reasoning) and width-based extensions (parallel sub-task decomposition)\n3. Verification: Tasks are validated through rejection sampling and LLM-based linguistic analysis, with incremental validation during extensions\n\nThe resulting dataset contains 41k tool-intensive tasks across varied difficulty levels, including 12.6k tool-interaction trajectories and 5k multi-hop decompositions. Experimental results demonstrate that models trained on TaskCraft data substantially improve performance on four agentic benchmarks (GAIA, WebWalker, BrowserComp, HLE), achieving state-of-the-art results. Notably, on GAIA, adding 2.5k TaskCraft tasks to 5k existing MHQA data improves performance from 38.8% to 60.2% (+21.4 points).", "questions": "Mentioned in Weaknesses section.\n\nI believe the paper has good contributions and strong results. There are couple of issues with writing and some experiments. If authors address those concerns I am happy to increase my rating.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The proposed task generation pipeline is a simple idea that leverages LLMs to synthesize high quality tool-use tasks\n2. The tasks and task execution traces generated through TaskCraft leads to training effective tool-use agents\n3. Proposed method is easy to scale to large number of tasks by gathering large amounts of unlabeled corpus of webpages, PDFs, and images.", "weaknesses": "1. The method section is a bit hard to follow. It’d be good if authors can take another pass at writing and improve the flow of the content to make the method easier to understand.\n2. There is a slight inconsistency in the types of datasets used (or at least in table descriptions for entries) for different model sizes for the experiments presented in table 3. For example, Qwen2.5-7B and DeepSeek R1 distill models have results for training on 7.5 MHQA tasks vs Qwen2.5-32B doesn’t mention the size MHQA dataset used. It would be great if authors could clarify or make the datasets used for these experiments consistent. In addition, I would also like to see results of just training these models on the 5k task subset from MHQA that was used in conjunction with TaskCraft 2.5k and 6k/8k tasks. This result should establish a concrete baseline that will clearly show what is the delta/improvement coming from task craft tasks. Currently, it is unclear whether improvement is coming from excluding 2.5k MHQA tasks or addition of 2.5k TaskCraft tasks. In addition, for RL finetuned 7B and 32B models I would suggest authors use a fixed number of tasks across model sizes (i.e. either 6k or 8k) to make the comparisons fair and make experiment setup consistent. It is unclear whether the 32B model is improving more from additional 2k tasks vs the model size.\n3. It is a bit unclear how the pass rate in tables 2 and 4 defined. Can authors clarify if the pass rate reported is the task execution + verification pass rate or is it another metric that is only measuring whether task generation is feasible?\n4. The paper does not include details on the initial corpus used to build the input index i_T and lacks details on how tools are constructed, what tools are used. Because of these missing details it is bit hard to reason about where the improvements in harder benchmarks like HLE and BrowserComp are coming from. It would be great if authors can add a section describing these details and also some preliminary analysis on whether the task craft generated tasks overlap with tasks from these benchmarks to check if there is any test set contamination.\n5. Task verification/extension section needs more work on writing. With the current description it is a bit unclear to me how the authors are using linguistic analysis to verify whether a proposed task extension is valid or not. More details are required in the main paper with appropriate references to the detailed prompts and method in appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xiUakblwty", "reviewer": "Reviewer_Ck8P", "summary": "Existing agentic benchmarks are constrained by the high cost of human annotation. TaskCraft addresses this challenge through automated task generation and extension, coupled with efficient verification mechanisms, to produce a scalable, multitool, and verifiable agentic task dataset. By applying SFT and RL on TaskCraft-generated data, the models demonstrate substantial improvements in multi-hop reasoning and agentic capabilities, showcasing the practical utility and potential of the TaskCraft framework.", "questions": "1. The number of generated tasks rapidly decays as the number of hops increases. What are the underlying reasons for this decay, and can you provide examples of longer-hop tasks to illustrate the generated complexity?\n\n2. The paper lacks an analysis of tasks that failed during generation or verification. What were the primary reasons for these failures? Providing such an analysis would be beneficial for future research by highlighting common pitfalls and suggesting directions for improvement.\n\n3. Both the task extension and verification processes heavily rely on LLM. Could this dependence lead to data bias, and would using LLMs with different styles result in distinct data distributions? This could impact the generalizability of agents trained on TaskCraft data.\n\n4. The scalability of TaskCraft data is only demonstrated up to 5k tasks in the current experiments. What would be the effect of further increasing the data size, and could a performance curve be plotted to observe whether there is a saturation trend? Investigating this would help prioritize between developing more rigorous verification methods and expanding the dataset size for future enhancements.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. TaskCraft's automated workflow supports adaptive difficulty progression through depth-based and width-based extensions, eliminating the annotation bottleneck that limits existing benchmarks.\n\n2. The generated tasks span varied difficulty levels across multiple tool modalities (web, PDF, image), including complex multi-hop reasoning tasks. This difficulty stratification mirrors human-curated benchmarks like GAIA while maintaining scalability.\n\n3. The paper provides thorough experimental analysis across multiple dimensions.\n\n4. The work ensures reproducibility through detailed methodological documentation, including formalized task generation procedures, verification prompts (Appendix E), training configurations (Appendix F), and commitment to release all artifacts.", "weaknesses": "Please refer to questions section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BvJEbuGN2y", "reviewer": "Reviewer_18Gy", "summary": "This paper introduces TaskCraft, an automated framework for generating scalable, verifiable agentic tasks that require multi-step reasoning and tool use. The method begins with atomic tasks solvable by single tool calls and expands them via depth-based (sequential, multi-hop) and width-based (parallel, multi-subtask) extensions. Each generated task is incrementally validated through rejection sampling, LLM-based linguistic checks, and selective agent verification. The workflow ensures that generated tasks require genuine tool interaction rather than being trivially answerable by LLMs alone.\nTaskCraft yields a dataset of 41k tool-intensive tasks, including 12.6k tool-interaction trajectories and 5k multi-hop decompositions. When used for SFT and RL, TaskCraft data significantly improves model performance across agentic benchmarks (GAIA, WebWalker, BrowserComp, HLE), achieving state-of-the-art results even with modest model sizes.", "questions": "1. The depth/width extensions are a core contribution of the framework. It would be interesting to see ablations on performance vs number of hops in the finetuning data, though I understand that this is a bit out of scope for the purpose of this study\n2. I appreciate that the authors included the code and data files in the materials. Will these be released to the public upon acceptance? \n3. After task extension, do you also check if any / how many of the extended tasks can already be solved by the task agent within a limited number of tool use steps (i.e., the setup used for atomic task verification)? The current framework checks for soundness, leakage, difficulty of atomic task, etc, but I do not see any quantification of how task extension further increases difficulty of tasks and whether this difficulty increase can be controlled / customized", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The framework is innovative: atomic task generation + validation and depth/width extension\n2. The contributions are well motivated by addressing the data bottleneck for agentic reasoning tasks\n3. Demonstrates performance gains on multiple benchmarks and shows that generated data can be useful in SFT and RL", "weaknesses": "1. The paper could benefit from including more information on SFT and RL experiments, such as hyperparameters used, etc \n2. Would be good to include more details on tools used for task generation, specifically the PDF tool. Since PDF parsing is often tricky and noisy, more information on the implementation used in this work would be insightful\n3. The paper should clarify what exactly are the tools used in the framework and when. Most figures mention web search, PDF, and image tools. Figure 2 mentions a music tool and suggests there are more with \"...\", but these are not mentioned anywhere else in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)