Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning
Authors: Shaokun Zhang
Yi Dong
Jieyu Zhang
Jan Kautz
Bryan Catanzaro
Andrew Tao
Qingyun Wu
Zhiding Yu
Guilin Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Tool-using
Reinforcement Learning
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Enabling large language models with external tools has become a pivotal strategy for extending their functionality beyond text space. To enhance LLMs' tool-calling abilities, previous approaches primarily rely on supervised fine-tuning (SFT) with trajectories distilled from stronger models, often resulting in imitative reasoning that limits generalization. In this work, we explore rule-based reinforcement learning to enhance tool-calling in LLMs, resulting in Nemotron-Research-Tool-N1, a series of tool-calling reasoning models. Rather than enforcing supervision over intermediate distilled reasoning traces, Tool-N1 is trained with a binary RL reward that assesses only the format validity and functional correctness of tool invocations. This lightweight supervision allows the model to develop reasoning strategies independently, without relying on annotated trajectories. Experiments on several major benchmarks show that Tool-N1-7B/14B clearly outperform GPT-4o. We conduct a systematic study on the design of rule-based reinforcement learning strategies for training tool-calling models. Using 5,518 distilled reasoning trajectories, we compare SFT, RL, and the SFT-then-RL pipeline, finding that the widely adopted SFT-then-RL paradigm does not necessarily outperform pure RL.

Published as a conference paper at ICLR 2026
NEMOTRON-RBBEARCHH-TOLLN.. EXPLORING TOOL- USING LANGUAGE MODELS WITH REINFORCED REA- SONING
Shaokun Zhang-.2 Yi Dong'
Jieyu Zhang Guilin Liu
Jan Kautz²
Bryan Catanzaro"
Andrew Tao
Qingyun 2u Zhiding Yu
INVIDIA
PPenssylvania State University 3 University of Washington
ABSTRACT
Enabling large language models with external tools has become pivotal strategy for extending their functionality beyond text space. To enhance LLMSs tool-calling abilities, previous approaches primarily rely on supervised fine-tuning (SFT) with trajectories distilled from stronger models, often resulting in imitative reason- ing that limits generalization. In this work, we explore rule-based reinforcement learning (Guo al. 2025) to enhance tool-calling in LLMs, resulting in Nemotron- Research-Tool.
series of tool-calling reasoning models. Rather than enforcing supervision over intermediate distilled reasoning traces, Tool-N is trained with a binary RI reward that assesses only the format validity and functional correct- ness of tool invocations. This lightweight supervision allows the model to develop reasoning strategies independently, without relying on annotated trajectories. Exper- iments on several major benchmarks show that Tool-NI-7B// 71111BB clearly outperform GPT-4o. We conduct systematic study on the design of rule- -asedd reinforcement learning strategies for training tool-calling models. Using 5,518 distilled reasoning trajectories, we compare SFT, RL, and the SFT-then-RL pipeline, finding that the widely adopted SFT-then-- paradigm does not necessarily outperform pure RL. We have souresourced our implementation here
INTRODUCTION
Recent years, equipping large language models with external tools or functions has attracted signifi- cant attention, demonstrating impressive performance across wide range of domains (Bran et al.) 2023 Qu et al.) 2025). For instance, LLMS augmented with search engines can answer questions that go beyond their training data (Komeili al. (02T), while those equipped with Python interpreters are capable of solving complex mathematical problems by leveraging external libraries (Wu et al.) 2023). These tools effectively enable LLMs to operate beyond purely textual tasks, substantially extending their functional capabilities.
To enhance LLMs tool-calling capability, existing research has primarily focused on synthesizing large volumes of tool-use trajectories using advanced LIMSs (Zhang et al. [024b] Liu et al.) 2024a).
followed by supervised fine-tuning (SFT) on the generated data. Although step-by-step reasoning has been shown to play critical role in enabling LLMs to solve complex tasks (Wei al. (022),
these synthetic datasets often lack explicit reasoning steps. Consequently, current pipelines typically supervise only the tool calls step. without providing guidance the underlying reasoning process during training. In many cases, reasoning is either omitted entirely in training stage or deferred to the inference stage via prompting techniques (Yao et al. 2023b). While workaround is to distill reasoning trajectories from advanced models and then train student models via SFT (Chen et al. 2025b), this approach often yields pseudo reasoning: models merely learn to mimic surface-level patterns without truly internalizing the decision
maaiiignnmmmiinggg making process CCen et al. 2025b).
Throughout this paper, refer
as Tool-N1 for brevity.

---

Published as a conference paper at ICLR 2026
Training Data
Summarize
Rule Uased
Pregroces
Policc Model
Policy Leamning
Query:
0 Ofeemise
I'm considering lnvestlnn and
KL Loos
Tesl Calling
like to know what's 's happening in
Eerrnne
the market right now. Could you
Rollou
Calcolatc Rewands
get me the top
trends
the US?
Tools:
<think>.... ctool call> market trends (id-"us") -tool, i X Fenmat Error
I. calculate sales tax
2. get futuress prices
<think>... <think> stool callemarket trends tid-"uk") stool call> x Tool-Calling Error
3. market_ trouds
<think> will utllix the markel tnmdl api iool... <lhnkk stool call>- market trands (id- "us") stool call>
Correct
Highlight V Fully Open Source
Strong pe formance
Reasoning:
Trace No Extra lastoanttoo
No Resspning Su Servisioons
Advanced Rewa rd Designing
simplc Biarry Roword Exoct Tool Match wi Dict
Thorough Recipe exploration
Simple Data Plaeline
Better CID General artinn
(Tool-N1). Starting from standard SFT tool-calling data comprising user queries and candidate tools, we train LLMs to produce structured reasoning and tool calls using binary reward function within the GRPO algorithm. As supervision is only applied to the format and tool-call correctness, the training process does not require curated reasoning trajectories.
Figure I: Overview of the training pipeline for
Recently, simple rule-based RI-style reinforcement learning (Guo et al. 2025 has been shown to substantially enhance the complex reasoning capabilities of L.LMs (Shen et al. 2025 Ma et al. 2025 Lu et al. 2025b). In these R1 R1-style RL training, rewards are assigned based the correctness of the final answer and output format, allowing the model to learn the intermediate reasoning steps without explicit reasoning supervision. This paradigm naturally inspired us explore the utilization of these rule-based RL in training tool call models, for the following reasons: (1) sule-based reward design provides interpretable supervision signals by verifying the correctness of tool calls. In contrast to SFT, which relies exact next-token prediction and enforces strict output matching, RL allows for more flexibility. Semantically equivalent tool calls, such as those with reordered arguments, can still be rewarded correctly. This enables models to generalize beyond rigid string-level imitation. (2) RI-style RL does not require tool-call data with explicit reasoning annotations. Instead, it imposes minimal constraints, requiring only that the model follow structured output format. This makes it possible to train reasoning capabilities directly from available SFT data. As a result, the model can acquire reasoning skills without curating reasoning traces.
Building on these insights, we address two key questions in this work: Can rule-based RL be effectively applied to train tool-calling models? How should the RL pipeline be designed? To this end, we conduct a systematic study on the design of rule-based reinforcement learning strategies for tool-call training, resulting in using LLMs trained with rule-based RL. The training enforces a simple-tructured reasoning-action format, guiding the model to produce explicit reasoning before invoking tools. We employ a binary reward function that evaluates the correctness of both the reasoning format and tool invocations. This reward design provides precise yet flexible supervision, allowing variation in argument ordering while ensuring functional correctness. The training procedure is shown in Figure
(Tool-N1), series of reasoning tool-
We conduct extensive experiments on BFCL (Yan et al. 2024). API-Bank (Li et al. 2023 and ACEBench (Chen et al. 2025a).. Empirical results demonstrate Tool-N1-7B/14B, built upon Qwen2.5- 7B/14B-Instruct, clearly outperform GPT-4o and other strong baselines. For instance, Tool-NI-14B achieves approximately 2% and 5% average performance improvements on BFCL and APIBank than GPT-4o. The smaller model, Tool-NI-7B also surpasses the closed-source GPT-4o and the specialized fine-tuning model Hammer2. on BFCL by 0.85% and 2.97%, respectively. We further conduct extensive experiments to analyze various aspects of the proposed methodology, including reward design and training data composition. addition, we curate a dataset of 5,518 tool-calling reasoning trajectories and perform systematic study on the roles of SFT, RL, and their combinations. Notably, we find that the widely adopted "SFT-then-RL" paradigm does not necessarily yield better performance than pure RL. To summarize, this work makes the following key contributions: (1) We propose a simple yet powerful cule-based RL algorithm that converts standard tool-calling data into fully verifiable training signals. This provides training paradigm that enables reinforced
2

---

Published as a conference paper at ICLR 2026
reasoning without requiring curated reasoning trajectories. (2) Building upon this paradigm, train Tool-N1-7B/14B and demonstrate strong empirical performance across BFCL, API-Bank, and ACEBench, outperforming GPT-4o and leading open-source baselines. (3) We conduct extensive experiments to disentangle the roles of supervised fine-tuning, pure RI., and the widely adopted SFF-then-RI, paradigam. Our findings reveal that the commonly used SFT-then-RI, paradigm does not always outperform pure RL in tool-calling scenario.
PROBLEM FORMULATION
We first provide the problem formulation. Consider a I.I.M and a set of external tools 2 = [=]]== that the L.M can access. Each tool could be represented 3-tuple (n4, di, k.) which includes essential information for tool usage: n, denotes the tool's name, d, provides natural language descriptions, and Y; specifies the tool's input parameters instructions. The model's objective is to respond to user queries in accordance with a policy т. To achieve this, the LLM may issue multiple tool calls throughout the interaction, each with appropriate input parameters.
At any decision step the L.MM receives types of input: (1) the historical context ct which consists of all preceding tool-call and observation pairs, and (2) the set of currently available tools Z that can be utilized this step. The LLM must then decide on the next action. Formally, the decision-maie makiing process is defined as: T(e,, 2) + dt, dz C Z, where a, denotes the action selected at step t. This action corresponds to one or more tool calls drawn from the accessible tool subset Z. c represents the historical context. Specifically,
a2 (PO(P)) ..... Zm (Pm))
G (ao, ....0.....p 0,)
where each 2m denotes the m-th tool invoked and pm its corresponding parameters. The value m indicates the number of tool calls made at time t. We employ or denotes the observation after action az is taken. The ultimate goal of tool learning is to enable LLMs with a generalized policy that effectively addresses user queries by producing a sequence of action-observation pairs Or).
3 MEMOTRON-RESSARCCH-TOON
In this subsection, explore the use of RI RI-style reinforcement learning for training tool-calling language models, resulting in Nemotron-Researchh - a series of general-purpose tool-using reasoning LLMss. Tool-N1 is built the GRPO algorithm (Guo et al., 2025 Shao et al.] (024). aiming to enhance tool-calling capabilities with reasoning in complex scenarios where the LLM must solve queries using set of accessible tools. We further conduct a systematic study on the design of rule-based reinforcement learning strategies for this training paradigm.
Formally, given the historical context c, and the set of currently available tools Z, the model generates set of candidate responses |0 0 O2....,O'' ONJ, where each O" E O. Here O denotes the space of possible output responses, each comprising: (1) textual reasoning. (2) an associated action O₂. These responses are evaluated using reward function, yielding reward set {r1, 12:,72 TN.. We then optimize the policy using GRPO, the optimization objective is shown below:
CORPO =Bea, Eoino min (p.A,. clip(p;, - c, e)A,)
( Cr, 2)
B 3KL(Fg) (To) {aua] where Ps Tota (O+ Cr, Z)
(2)
e and 3 represent tunable hyperparameters. A₁ represents the relative advantage of the i-th response, which is computed as follows:
A r; mmca(((m ....... rNN)) std(fri,, ,2....,TN
where mean and std represent the mean and standard deviation of the rewards, respectively. In subsequent sections, detail the following components of our approach: (1) the preparation of tool-calling data and its integration into the RL training pipeline, (2) the structured reasoning template adopted during training, and (3) the reward modeling strategy used compute the reward signal
3

---

Published as a conference paper at ICLR 2026
3.1 DATA PREPARATION
Numerous prior works have focused collecting large-scale tool-calling trajectories (Liu al. 2024a Zhang et al. 2024bb Yin et al. 2025 Qin et al., 2023), followed by supervised fine-tuning to improve LLMs tool-use capabilities. Each instance in these dataset typically consist of natural language user query Q, paired with sequence of ground-truth tool invocation steps in the form (0g, Og: dt. 0z). The model then trained predict each subsequent action 04+1, conditioned on the observed trajectory up to that point. However, SFT often exhibits limited generalization ability, as the model tends to memorize the training trajectories rather than developing robust, intrinsic reasoning capabilities which has been pointed out in recent work (Chen et al., 2025b). To fully leverage the available tool-calling data in the community, we unify and preprocess data from xLAM (Zhang et al. 2024b) and subset of ToolACE (Liu et al. 2024a), which provide single-tun furle-turn&ae and multi-turn synthetic tool-calling trajectories. As these datasets are generated by potentially unstable LLMss, they often contain inconsistencies and unstructured formats unsuitable for RL. training.
We standardize the data by filtering out samples with invalid tool calls, specifically, those involving tools absent from the candidate tool list. Available tools are extracted from the system prompt, and both candidate and ground-truth tools are parsed into structured dictionary formats. Instances that fail JSON parsing or contain formatting inconsistencies are discarded. We ensure this processing yields clean and consistent data suitable for reinforcement learning. For multi-turn data from the ToolACE, we further segment each trajectory into multiple single-step prediction instances, where each instance contains one target tool call and the preceding steps are treated context. We train LLMs using R -tyle GRPO to predict each tool invocation step based on this contextual information and provided tools. We present the statistic information of the proceed data in Table in Appendix C
3.2 THINKING TEMPLATE
Thinking Template
Here a of functions in JSON format that you can invoke: <tools> (tools] </tools>.
each action step, you MUST:
1. Think about the reasoning process in the mind and enclosed your reasoning within <think>-<think>
XML tags.
2. Then, provide a json object with function names and arguments within <tool, <tool_calls call>> </tool </tool_call>>> call> XML lags. ie., <tool_ <tool_call>s" ["name": <function-names, "arguments" : json-nnsos,,,,,, "name": <function -name2>, "arguments"::
jong-jjon-ubjeet22,, ....s// ...|</toolccali
Make both the reasoning and the tool call steps included together one single reply. A complete reply example is: <thinks To address the query, I need to send the email to Bob and <tool_ call> [" ""naam""""mall""" "Bob", "content": "I will bug banana through walmart", "name": "walmart", "arguments": "input": "hanana") _ello, Please make sure the type of the arguments is correct.
then buy the banana through walmart.</thiin
arguments"..rrcccive"
The resoning prompt template used during training and inference. The prompt guides the LLM to explicitly separate its internal reasoning (within <think> tags) from tool invocation actions (within stool_calll tags). We show the full prompt template in Appendix F
Figure 2:
Following prior work (Guo et al. 2025), adopt lightweight prompting template to elicit tool The prompt explicitly instructs the model to gener- t/thinkk> tags, followed by the tool invocation attool_call>> tags. The design philosophy behind this template minimize reliance on overly rigid formatting rules, which could reducing the risk of overfitting to specific prompt patterns. By allowing greater flexibility in how the model expresses its reasoning, we aim promote more robust generalization across diverse tool-use scenarios. Additionally, the use of this lightweight prompting design during training enables the resulting model to be more easily integrated with more sophisticated prompting strategies (Yao et al. 2023b
calls from the L.LM which is shown in Figure _ ate intermediate reasoning within <think> enclosed in <t stcol_call>
Zhang et al. 2024c).
3.3 REWARD MODELING
Following the data preparation described in Section 3.1 we construct a training dataset in which each ground-truth tool call is represented as a structured dictionary. This format enables reliable verification of tool names and argument- value pairs during reinforcement learning rather than simply string match. Leveraging this structure, we define a binary reward function in which jointly evaluates the correctness of the reasoning format and the accuracy of the tool call.
4

---

Published as a conference paper at ICLR 2026
Format Checking. Following prior work (Guo et al., 2025 Shao et al. 2024 Meng et al. 2025), we incorporate format checking during training to verify whether the model's 's out- put adheres to the expected structural conventions, specifically, whether the reasoning is enclosed within <think>... Barnnkk.....tttttii </think> tags and the tool call is properly placed within etool_call>>
tags. This structural constraint encourages the model to engage in explicit reasoning before tool calling, rather than shortcutting to the final answer. By enforcing format adherence, we aim cultivate the model's intrinsic reasoning ability, which may potentially contributes to improved generalization, particularly on out-of-distribun
inputs.
Tool-Calling Checking. We also check the correctness of the tool call. The tool-call output is parsed as dictionaries, enabling exact matching against the ground-truth. This involves checking whether the predicted tool name matches the ground-truth and whether all required arguments are present correctly. This strict matching criterion ensures that the model learns generate functionally precise and executable tool calls. Compared to the next-token prediction logic in SFT, this dictionary- based matching introduces greater flexibility. It allows argument order to vary without penalization, encouraging the model to focus on the underlying semantics rather than surface-level memorization.
Binary Reward Definition. Given previous context 6 and the L.M output O₁, we define a binary reward function r(c,, O+) E (0. that assigns reward of 1 if both of the following conditions are met: (1) Format Correctness: The model output adheres to the structural format, i.c., contains both <think>, </think> and stool_caa>>_
@ooolccallll tags. (2) Tool Call Correct-
ness: The predicted tool call 0 E O₂ exactly matches the ground- truth call az in both tool name and all argument vel-value pairs.
11 if (ooomttCCrreet((0) A roolCallMatch(an, az) r(c,, O,) = 0, otherwise
where F (ormatCorrect(O)) returns true if the output is correctly wrapped in both required tags, and the in structure and content. We find that a simple binary reward scheme most effective for training tool-calling LLMs, as validated through extensive ablation studies presented in Section 4.4.
of) returns true if a, matches the ground-truth tool call of exactly
EXPERIMENTS
We conduct experiments to prove the superiority of the proposed method. We begin by providing the experimental settings in Section 4.1 We then evaluate the training method representative benchmarks to verify its effectiveness. Finally, we perform in-depth investigations for the training paradigam.
4.1 SETTINGS
Datasets. We primarily utilize a subsets of ToolACE Liu et al., 2024a) and xLAM Zhang et al. 2024b) our training data. ToolACE (Liu et al. 2024a) encompasses wide range of tool-calling scenarios, including examples with multiple candidate tools and parallel tool calls, and covers a pool of 26,507 diverse tools. In contrast, xLAM focuses on single-turn tool calling, comprising 60,000 instances collected through APIGen (Liu et al. 2024b).
(Yang et al. 2024) as the primary backbone model throughout this work. To assess the generalization ability of our method, we also perform evaluations on alternative backbone models, including multiple variants from the LLaMA family in different scales. In experiments, we compare against both general- purposs open-source models, such as the GPT series and Gemini-2.0, as well as specialized tool-calling models, including ToolACE-SB (Liu et 2024a), XLAM-2 (Prabhakar et al. 2025). and Hammer2.1 1 (Lin et al. 2024).
Models. Unless otherwise noted, we use 1eeee2..-7B/44---Inns
Benchmarks. We primarily evaluate the performance on single-turn tool-calling scenarios with several representative benchmarks, including the Berkeley Function Call Leaderboard (BFCL) (Yan et al., 2024), API-Bank (Li et al. 2023) and ACEBench (Chen et al. (025a). For BFCL, we conduct evaluations on both the Non-live and Live subsets, corresponding to synthetic and real-world data, respectively. Each subset includes four categories: Simple. Multiple, Parallel, and Parallel Multiple. Simple and Multiple scenarios both involve the invocation of a single tool, with the Multiple category featuring multiple candidate tools. In contrast, Parallel and Parallel Multiple scenarios require the simultancous invocation of multiple tools. For API-Bank and ACEBench, we exclude multi-turn cases from our evaluation. Performance across all benchmarks is reported in terms of accuracy.
5

---

Published as a conference paper at ICLR 2026
Non-Live
Live
Overall
Models
Simple Multiple Parailel
Parallel Multiple
Sumple Multiple Purallel
Purallel Multiple
Non Nov-live
Live
Overall
GPT-4o
79.42
83.50
84.88
79.77
87.50
75.00
88.10
79.83
83.97
GPT--o-mini
80.08
90.50
89.50
87.00
81.40
76.73
93.75
79.17
86.77
76.50
81.64
051:3.5-Turbo-015
77.92
93.50
67. .0.00
53.00
80.62
78.63
75.00
58,33
72.85
68.55
70.70
Gemini-2.0-Plast-MM
74.92
89.50
86.50
87.00
75.58
73.12
81.25
83.33
84.48
81.39
82.94
DeepSeek-R1
76.42
94.50
90.05
88,00
84.11
79.87
87,50
70.83
87.35
74.41
80.88
Llama311-70B-nu
77.92
96.00
94,50
91. .0
78.2
76.16
87,50
66.67
89.98
62.24
76.11
Llama3.1-8B-in
72.83
87.00
83.50
74.03
73.31
56.25
54.17
84.21
72.65
Qwen2.5-7B-Ins.
75.33
94.50
91.50
84.50
76.74
74.93
62,50
70,83
86.46
67.44
76.95
XLA---2-70b-f--r (FC)
78.25
94.50
92.00
89.00
77.13
71.13
68.75
58,33
88.44
72.95
80.70
ToolACE-8B (FC)
76.67
93.50
90.50
89.50
73.26
76.73
81.25
70.83
87.54
78.59
82.57
Hammer2.1177 (FC)
78.08
95.00)
93.50
88.00
76.74
77.4
81.25
70.83
88.65
75.11
81.88
Tool-N1-7B
77.00
95.
94,50
90.50
82, 17
80 444
62,50
70.83
89 25
80.38
84.82
Tool-N1-14B
80.58
93.50
92.00
84.10
81.10
81.25
66.67
90.52
81.42
85.97
Table 1: Comparison the BFCL benchmark (last updated 2025-04-133 13). Average performance is calculated using the official script. The best results in each category are highlighted in bold, while the second-best results underlined.
80
APlBank
80
81.28
Accuracy on ACEBench
80
82:34
87.00
73.10
60
50.63
70-
Accuracy
60.41
60
50
LLAMA-3, 1-Baae B8
GPT-46 mini
GFT-40 ThottN1178
Nernatron-k
Nemotron-k -Tool -Tonl-N1-148
LLLMA-3. 1-t..---aaaea 1-Base-88
GPT- 4p-mint
Wemotran-ft -Took No-7B
Mermatrrn-- Tool 148
GPT-40
(a) API-Bank Results
(b) ACEBench Results
Figure 3: Performance comparison two additional benchmarks, ACEBench and API-Bank.
Other Implementation Details. We conduct all RL training using the open-source reinforcement learning library Verl (Sheng et al., [024). Training is performed with a batch size of 1024 and a learning rate of 10- The temperature is fixed at 0.7. We set the entropy coefficient to 0, as we observe that introducing entropy negatively impacts exploration. The KL divergence loss coefficient set to 10-3 across all experiments. All training runs are executed on a cluster of 4 nodes, each equipped with 8 NVIDIA H100 80GB GPUs. For SFT training, we employ LLaMA-Factory (Zheng jet al. 2024a). More details the implementations could be found in Appendix
4.2 MAIN RESULTS
Results on BFCL. Table Tabieprresentss presents the evaluation results the BFCL benchmark, including detailed accuracy for each subcategory. We adopt the official evaluation script from the BFCL leaderboard and report the average accuracy across categories. We observe that specialized fine-tuned models, such as ToolACE-83 and Hammer-2.1-7BB benefit significantly from domain specific training data, achieving better performance than GPT-4o-mini despite their smaller model sizes. However, neither surpasses GPT-4o. In contrast, Tool-N1-7B and Tool-N1- 144 models achieve clearly superior overall performance, outperforming both state-of-the-art closed-source models (e.g., GPT-4o) and domain- specialized fine-tuned models (c.g., 1LAA-2--70B and ToolACE-8P -B)) Notably, our trained tool-call reasoning models significantly outperform supervised fine-tuning (SFT) basclines trained on the same data sources, including the ToolACE (Liu et al.) 2024a) and xLAM (Zhng et al. 2024b) series. These results collectively demonstrate that the RI-style reinforcement learning paradigm provides more effective approach for improving the tool-calling capabilities of large language models.
Results on More Benchmarks. To provide a more comprehensive evaluation, we conduct exper- iments on two additional benchmarks: ACEBench (Chen et al. 2025a) and APIBank (Li al.) 2023). The results are shown in Figure 3 On these benchmarks, Tool-N1-7B achieves substantial absolute improvements of over 20% APIBank and over 30% on ACEBench compared to the base models. Notably, Tool-N1-7B outperforms GPT-4o-mini on both benchmarks, while Tool-NI- ---BBB--BBBBBB surpasses GPT-4o by 5.03%. These results demonstrate that models trained using our proposed

---

Published as a conference paper at ICLR 2026
method generalize effectively across diverse tool-calling scenarios, thereby opening opportunities for scaling performance in LLM augmented learning.
4.3
DEEP ANALYSIS
86.5
80
76.2
80
Accuracy
Accuracy
60-
40
50-
live
Non (raw)
20
Uve
Uve fraw
1.58
76
UL4MA-1-888
1LaM6-3.1-00
Dwen-25-77
Model Size
Non Live
Live
Scaling performance across model sizes using the Qwen2.5-Instruct series. We report re- sults for both RL-trained and original instruction- tuned models without further post-training.
Figure 4:
Figure 5: Performance across different backbones. All models use their instruction tuned variants. At comparable scales, Qwen consistently outper- forms both LLaMA and LLaMA 3.1.
4.3. SCALABILITY AND GLIERILIZABILITY
Scalability. The scaling law, which characterizes the relationship between model size and perfor- mance, plays a critical role in understanding the effectiveness of the training methods. We assess the scaling behavior of the proposed training method by evaluating range of model sizes, including 0.5B, 1.5B, 3B, 7B and 14B from the Qwen2.5 Instruct series. For comparison, we also report the performance of the original instruction-un average performance on both the Live and Non-Live categories of the BFCL benchmark, with detailed results presented in Figure 4 expected, larger models consistently outperform smaller ones both evaluation settings. Notably, performance improvements from post-training are limited for smaller models (0.5B and 1.5B), whereas larger models exhibit substantial gains. These findings suggest that the R 1-ytyll training method scales more effectively with increasing model size.
models without any additional training. We report
Generalizability. We further evaluate the impact LLM backbones Touvron et al. (023) to investigate the generalization capability of the proposed method. In addition to the Qwen. include experiments using LLaMA series: LLaMA3-8B-Ing -nntrructt and LLaMA3. 1AMMMA..--8B---Se Instruct. These evaluations conducted the BFCL. benchmark, with results presented in Figure 5 Our findings show that Qwen2.5 -nsttrruuutt significantly outperforms both LLaMA variants at the same model scale. This advantage is likely due Qwen's 's inherently stronger reasoning capabilities, as observed by Gandhi et al. (2025). ). As result, the R RI-style training paradigm is able to elicit better performance when applied Qwen series LLMs.
Finding
The R -syyle training method scales effectively with model size, yielding significantly greater perfor- mance gains for larger models compared to smaller with Qwen models outperforming LLaMA variants at the same scale.
also generalizes well actoss backbones,
4.3.2
SET OR RL? DEEP INVESTIGATION INTO THE TRAINING RECIPES
Method
No-Reason SFT
Reason-SFT
Reason-SFT+RL
RL
Recipes
50% SFT
100% SFT
50% SPT
100% SFT
50% SFT+50%RL
50% RL
100%
Non-Live
85,94
86.40
86.40
87.54
88. 19
87.12
88.23
Live 76.09
76.54
76.61
77.87
78. 16
78.90
78.24
Avg $1.02
81.47
81.50
82.71
83.17
83.01
83.24
Table 2: Performance comparison of different training recipes on tool-calling tasks with 5,518
instances from ToolACE. "No-Reason SFT" fine-tunes on tool-call data without reasoning traces; "Reason-SFT" uses reasoning trajectories; "Reason-SFT+RL" applies RL after Reason-SFFy Percent- ages indicate the proportion of data used for each stage.
Although the proposed R1 R1-style reinforcement learning paradigm demonstrates strong performance, we conduct a more comprehensive comparison against SFT, including both SFT with and without reasoning trajectories. Our goal is to identify the most effective training recipe for tool-using L.LMs
7

---

Published as a conference paper at ICLR 2026
with reasoning capabilities. To ensure fair comparison, we filter out single-turn data from ToolACE and distilled reasoning trajectories using D (Guo et al. (202). ). one of the most advanced tool-using language models. We prompt DeepSeek-RI R1 to generate responses with reasoning enclosed in <think>..</think> tags and tool calls in <tool tatp__llllll/ooo/ [itool_call> tags. Only trajectories that (1) follow the correct format and (2) produce correct tool calls retained, yielding 5,518 high-quality instances. All the experiments are performed upon Qwen2.5 1rnn2..-Instruc model.
Using these data, we construct several training recipes: (1) Reason-SFT, which performs SFT on distilled reasoning trajectories; (2) No-Reason SFT, which performs SFT only on corresponding data without reasoning traces; (3) RL, the proposed RL training method. (4) Reason-SFT+RL, which applies SFT followed by RL on the same dataset. We could observe that: (1) Although the combination of SFT on reasoning trajectories followed by RI is commonly regarded as the best practice in many domains, we do not observe improved performance under equal data budgets in the tool-calling setting (83. 17% vs. 83.24%) fact, SFT may even hinder performance, phenomenon also noted by Chen et al. (0025b). (2) Pure RL outperforms both Reason-SFT and No-Reason SFT under equal data budgets, achieving the best performance (83.24% with 100% RL vs. 82.71% with 100% Reason-SFT and 81 11.47% with 100% No-Reason SFT). This finding is consistent with prior observations Chen et al. (2025b), which suggest that SFT may encourage sseudo-reasoning rather than genuine task-oriented reasoning. (3) Interestingly, No-Reason SFT performs only slightly worse than Reason-SFT (e.g., 81.4 31.47% vs. 82.71 82)71%) suggesting that providing reasoning traces during SFT offers limited additional benefit.
Finding 2
The commonly adopted SFT-then-RL pipeline does necessarily outperform pure RL in tool-calling tasks. Notably, pure RL, trained without reasoning distillation data, demonstrates stronger performance than any other combinations involving SFT and RL.
4.4
VISUALIZATION
Nemotron- Be
5.60
1150
342
342
082
122
350
232
Memotmon-esserr www.eeeecccchoooooo.---.-0.
me
030
Memustnn-eeweecthe hew. -1
ad
23
60
100
300
alc
ida
Stop
Stop
Step
(a) KL divergence
(b) Average Response Length
(c) Validation Results
Figure 6: Learning curves across training steps for Tool-N1-7B and Tool-N1-14B. We report KI. divergence, average response length, and validation accuracy.
We present additional visualizations of the training dynamics in Figure including KL divergence, average response length, and validation accuracy over training steps. The KL divergence and validation accuracy, shown in Figures 6aa| and 6c exhibit consistent improvement, suggesting that the model get stable learning and exploration under the training setup. Notably, Figure (b) shows no significant increase in response length throughout training, which contrasts with trends observed in prior RI 1tylt training works (Guo et al. 2025 Liu & Zhang 2025 Shen et al. 2025). Tool-N1-7B maintains relatively flat trajectory from the beginning, while Tool-N1-144 gradually decreases to a comparable length. This observation suggests that longer reasoning chains do not necessarily lead to better tool-use performance; instead, there appears to be an vertain response length that could be sufficient for accurate tool calling.
4.4.1 ABLATIONS
Ablations on Reward Design. To assess how reward granularity affects model behavior, we evaluate Tool-NI-7B under two reward schemes: fine-grained and binary (Table 3). The fine-grained setting provides partial reward, 0.2 for correct reasoning format and an additional 0.2 for matching function names, even if the final function call is incorrect. In contrast, the binary setting only gives a reward of 1.0 when all components
correct, including reasoning, function name, and arguments. Here, w/ Reason Format requires the model to output intermediate reasoning in a specific format, while w/o removes this constraint. The results are shown in Table 3 Tool-NI achieves consistently better
8

---

Published as a conference paper at ICLR 2026
FrainGr Reward
Binary Reward
w/ Reason Format Partial Reward
w/ Reason Format
w/o Reason Furmat
w/
Split
Func name Partial Rewards
Reason Format
Non-Live
87.83
88.54
87.63
89.25
Live
79.64
76.61
76.24
80.38
Avg
83.74
82.58
81.94
84.82
Table 3: Ablation study on reward granularity. We compare fine- tine-grained reward designs, where partial credit given for correct reasoning format and correct function names function call stages, with binary rewards that assign full reward only when all conditions are fully satisfied. The results show that binary rewards consistently yield better performance, especially in the Live setting.
performance with binary rewards, particularly on the Live subset (80.38% vs. 76.61%), which involves more realistic inputs. We attribute this reduced reward hacking (Pan et al. 2024): under fine-grained schemes, the model may overfit to superficial cues such as formatting or partial matches, without ensuring full execution correctness, Furthermore, within the binary setup, we observe that removing the reasoning format constraint significantly hurts performance (dropping from 80.38% 76.24%). This highlights the critical role of structured reasoning in guiding Tool-N1-7B toward reliable and generalizable tool use, especially in complex, real-world scenarios.
Finding
Binary rewards lead to superior performance compared to fine-grained rewards, especially on realistic inputs. Additionally, enforcing structured reasoning is erucial, removing significantly degrades performance, underscuring its importance in guiding reliable tool use.
Ablations on Training Data Composition.
Raw Model
xLAM 8B
ToolACE 8B
XLAM w/o ToolACE
ToolACE w/o xI.AM
ToolACE w/ xL.AM
Recipe
Non-Live
73.94
84.40
87.54 78.59
87.77 76.24
87.67 79.58
89.25
Live
61.14
66.90
80.38
Avg 67.54
75.65
82.57
82.01
83.63
84.82
Table 4: Effect of training data composition on Tool-N1-7B performance. We compare the full training recipe against ablated variants that exclude xLAM or ToolACE data. Results show that both sources contribute to performance, with ToolACE providing greater gains.
We then investigate how different data composition strategies affect performance the BFCL benchmark. Experiments are conducted using the Tool-N1-7B model, with results presented in Table Our key findings are as follows: (1) ToolACE data yields particularly strong improvements in the live setting. Overall, these results suggest that progressively enriching the training data enhances the model's tool-calling proficiency. (2) Compared to the model (Qwen2.5-77B. (rren2557BB-nsttrect, RI R1-style training sigmificantly enhances tool-calling capabilities. (3) Compared to models trained using SFT on the same data source, the RI-style training consistently yields better performance. Specifically, the Tool-N1-7B model trained solely on xLAM data outperforms the xLAM 8B SFT model by 6.36%, and the Tool-Ni-7B model trained solely on the ToolACE subset exceeds the ToolACE-SP SFT model by 1.62%, despite using only subset of the data.
Instructt,,
CONCLUSION
We introduce Nemnotron-Researchh a rule-based reinforcement learning paradigm. Unlike previous approaches that rely on SFT with annotated reasoning trajectories, Tool-N1 employs reward function that supervises only the final answer and the structural format of the reasoning. This enables the model to learn effective rea- soning strategies without requiring step-by-step annotations. Experimental results demonstrate that Semnotron-Researee Tool-N1 consistently outperforms existing baselines across multiple benchmarks. In addition, we curate a of 5,518 distilled reasoning trajectories analyze the effects of supervised fine-tuning, reinforcement learning, and their combination. Our findings indicate that the commonly used SFT-then-RI. pipeline does not necessarily yield better performance than pure RI.
a series of tool-using language models trained using

---

Published as a conference paper at ICLR 2026
REFERENCES
Ibrahim Abdelaziz, Kinjal Basu, Mayank Agarwal, Sadhana Kumaravel, Matthew Stallone, Rameswar Panda, Yara Rizk, GP Shrivatsa Bhargav, Maxwell Crouse. Chulaka Gunasekara, al. Granite- function calling model: Introducing function calling abilities via multi-task learning of granular tasks. In Proceedings of the 2024 Conference on Empirical Methods Natural Language Industry Track, pp. 1131-1139, 2024.
Processing:
Andres M Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller. Chemcrow: Augmenting sarge-laguaage models with chemistry tools. NeurlPS 2023 Al for Science Workshop, 2023.
Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang, Xingshan Zeng, Shuai Yu, Dexun Li, Shuai Wang. Weinan Gan, Yuefeng Huang, et al. Acebench: Who wins the match point in tool learning? arXiv preprint arXiv:2501. 12851, 2025a.
Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rI? an early investigation into training rl-like reasoning large vision-language models. arXiv preprint arXiv:2504.11400 2025b.
Mingyang Chen, Haoze Sun, Tianpeng Li, Fan Yang, Hao Liang, Keer Lu, Bin Cui, Wentao Zhang, Zenan Zhou, and Weipeng Chen. Facilitating multi-turn function calling for Ilms via compositional instruction tuning. arXiv preprint arXiv:2410. 12901222... 2024a.
Yanfei Chen, Jinsung Yoon, Devendra Singh Sachan, Qingze Wang. Vincent Cohen-Addad, Moham- madhossein Bateni, Cuen-Yu Lee, and Tomas Pfister. Re-invoke: Tool invocation rewriting for zero-shot tool retrieval. arXiv preprint arXiv:2408.0185,, 2024b.
Fares Fawzi, Vinitra Swamy, Dominik Glandorf, Tanya Nazaretsky, and Tanja Käser. Scribe: Structured chain reasoning for interactive behaviour explanations using tool calling. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 29273-29298, 2025.
Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong, Retool: Reinforcement learning for strategic tool use in Ilms. arXiv preprint arXiv:2504. //536, 2025.
Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. arXiv preprint arXivv2503.01100) 2025.
Jiaxuan Gao, Shusheng Xu, Wenjie Ye, Weilin Liu, Chuyi He, Wei Fu, Zhiyu Mei, Guangju Wang. and Yi Wu. On designing effective rl reward at training time for Ilm reasoning. arXiv preprint 211 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in 1lms via reinforcement learning. arXiv preprint urXiv:2501.12945 2025.
Bowen Jin, Hansi Zeng, Zhenrui Yue, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training Iims to reason and leverage search engines with reinforeement learning. arXiv preprint arXiv:2503.095/6. 2025.
Geunwoo Kim, Pierre Baldi, and Stephen McAleer. Language models can solve computer tasks. Advances in Neural Information Processing Systems, pp. 99648-39677, 2023.
Mojtaba Komeili, Kurt Shuster, and Jason Weston. Interne--aummented dialogue generation. arXiv preprint arXiv:2107.07532 2021.
Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and Nikolai Grigorev. Internet- augmented language models through few-shot prompting for open-domain question answering. arXiv preprint arXiv:2203.05/11 15, 2022.
10

---

Published as a conference paper at ICLR 2026
Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. Api-bank: comprehensive benchmark for aol-auugented llms. arXiv preprint 11Xiv:2304.08244 2023.
Wendi Li and Yixuan Li. Process reward model with q-value rankings. arXiv preprint arXiv:2410. 11287, 2024.
Qiqiang Lin, Muning Wen, Qiuying Peng, Guanyu Nie, Junwei Liao, Jun Wang, Xiaoyun Mo, Jiamu Zhou, Cheng Cheng, Yin Zhao, et al. Hammer: Robust function-calling for on-device language models via function masking. International Conference on Learning Representations, 2024.
Jiawei Liu and Lingming Zhang. Code-rl: Reproducing rl for code with reliable rewards. arXiv
preprint arXiv:2503. 18470, 2025.
Weiwen Liu, Xu Huang, Xingshan Zeng, Xinlong Hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, et al. Toolace: Winning the points of Ilm function calling. International Conference on Learning Representations, 2024a.
Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng. Rithesh RN, et al. Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets. Advances in Neural Information Processing Systems, 2024b.
Pan Lu, Bowen Chen, Sheng Liu, Rahul Thapa, Joseph Boen, and James Zou. Octotools: An agentic framework with extensible tools for complex reasoning. arXiv preprint arXiv:2502.11277, 2025a.
Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Guanjing Xiong, and Hongsheng Li. Ui-rl: Enhancing action prediction of gui agents by reinforcement learning. arXiv preprint arXiv:2503. 21620, 2025b.
Peixian Ma, Xialie Zhuang. Chengjin Xu, Xuhui Jiang, Ran Chen, and Jian Guo. Sql-rl: Training nat- ural language to sql reasoning model by reinforcement learning. arXiv preprint arXiv:2504.08600, 2025.
Yuexiao Ma, Taisong Jin, Xiawu Zheng, Yan Wang, Huixia Li, Yongjian Wu, Guannan Jiang, Wei Zhang, and Rongrong Ji. Ompq: Orthogonal mixed precision quantization. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp.
2023.
Yuexiao Ma, Huixia Li, Xiawu Zheng, Feng Ling, Xuefeng Xiao, Rui Wang, Shilei Wen, Fei Chao, and Rongrong Ji. Affinequant: Affine transformation quantization for large language models. arXiv preprint arXiv:2403. 12544, 2024a.
Zixian Ma, Weikai Huang, Jieyu Zhang, Tanmay Gupta, and Ranjay Krishna. m & m's: A benchmark to evaluate tool-use for m ulti-step m ulti-modal tasks. In European Conference on Computer Vision, 2024b.
Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. arXiv preprint 0rXiv22003.07665, 2025.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Perey Liang. Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19399 2025.
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assistee question-answering with human feedback. arXiv preprint arXiv:2112.09332. 2021.
Alexander Pan, Erik Jones, Meena Jagadeesan, and Jacob Steinhardt. Feedback loops with language models drive in-context reward hacking, International Conference on Machine Learning, 2024.
Bhargavi Paranjape, Scott Lundberg, Sameer Singh, Hannaneh Hajishirzi, Luke Zettlemoyer, and Marco Tulio Ribeiro. Art: Automatic multi-step reasoning and tool-use for large language models. arXiv preprint 23X.v:2303.09014 2023.
11

---

Published as a conference paper at ICLR 2026
Akshara Prabhakar, Zuxin Liu, Weiran Yao, Jianguo Zhang. Ming Zhu, Shiyu Wang, Zhiwei Liu, Tulika Awalgaonkar, Haolin Chen, Thai Hoang, et al. Apigen-mt: Agentic pipeline for multi-turn data generation via simulated agent- heemt- interplay. arXiv preprint wrXiv:2504.3360) 2025.
Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. Toolllm: Facilitating large language models to master 16000+ real-world apis, 2023.
Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. Tool learning with large language models: A survey. Frontiers of Computer Science, 2025.
Timo Schick, Jane Dwivedi-Y Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539-68551, 2023.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03.: 03300, 2024.
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-rl: A stable and generalizable rl ri-style large vision-language model. arXiv preprint arXiv:2504. 07615, 2025.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng. Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409. / 19256, 2024.
Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith, Stephen Roller, Megan Ung. Moya Chen, Kushal Arora, Joshua Lane, et al. Blenderbot 3: a deployed conversational agent that continually learns to responsibly engage. arXiv preprint arXiv:2208. 08888 2022.
Ha Min Son, Huan Ren, Xin Liu, and Zhe Zhao. Automating android build repair: Bridging the caasoning-xeccutin gap in Ilm agents with domain-specific tools. arXiv preprint arXiv:2510.08644 2025.
Linxin Song, Jiale Liu, Jieyu Zhang, Shaokun Zhang, Ao Luo, Shijian Wang, Qingyun Wu, and Chi Wang. Adaptive in-conversation team building for language model agents. arXiv preprint 2rX1v:2405.19425, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro. Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.1397 2023.
Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better Ilm agents. In International Conference Machine Learning, 2024a.
Zhiruo Wang, Zhoujun Cheng, Hao Zhu, Daniel Fried, and Graham Neubig. What are tools anyway! survey from the language model perspective. arXiv preprint arXiv:2403 15452, 2024b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. shiin-of-though prompting clicits reasoning in large language models. Advances in neural information processing systems, 2022.
Mengsong Wu, Tong Zhu, Han Han, Xiang Zhang, Wenbiao Shao, and Wenliang Chen. Chain- of-tools: Utilizing massive unseen tools in the cot reasoning of frozen language models. arXiv 2025.
preprint ar arXiv:2503.167
Yiran Wu, Feiran Jia, Shaokun Zhang, Hangyu Li, Erkang Zhu, Yue Wang. Yin Tat Lee, Richard Peng. Qingyun Wu, and Chi Wang. Mathchat: Converse to tackle challenging math problems with Ilm agents. arXiv preprint 233.033306.11337, 2023.
12

---

Published as a conference paper at ICLR 2026
Fanjia Yan, Huanzhi Mao, Charlie Cheng-Jie Ji, Tianjun Zhang, Shishir G. Patil, lon Stoica, and Joseph E. Gonzalez. Berkeley function calling leaderboard. lattpa://gorila..c berkeley function forcttiinn calling Leaderboard..tmml
berzkeeey.ed//lloo 2024.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.51115. 2024.
Shunyu Yao, Dian Yu, Jefffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 2023a.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations, 2023b.
Fan Yin, Zifeng Wang.
Hsu, Jun Yan, Ke Jiang, Yanfei Chen, Jindong Gu, Long T Le, Chang, Chen- Yu Lee, et al. Magnet: Multi-turn tool-use data synthesis and distillation via graph translation. arXiv preprint arXiv:2503. 078266, 2025.
Xingshan Zeng, Weiwen Liu, Xu Huang, Zezhong Wang, Lingzhi Wang. Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Ruiming Tang, et al. Toolace-r: Tool learning with adaptive self-refinement. arXiv preprint arXiv:2504.0144 (4000,0400,0000 2025a.
Yirong Zeng, Xiao Ding, Yuxian Wang, Weiwen Liu, Wu Ning, Yutai Hou, Xu Huang. Bing Qin, and Ting Liu. Boosting tool use of large language models via iterative reinforced fine-tuning, arXiv preprint arXiv:2501.09765 2025b.
Daochen Zha, 7aid Pervaiz Bhat, Kwei-Hemg Lai, Fan Yang, Zhimeng Jiang, Shaochen Zhong, and Xia Hu. Data-centric artificial intelligence: A survey, ACM Computing Surveys, 2025.
Jianguo Zhang, Tian Lan, Rithesh Murthy, Zhiwei Liu, Weiran Yao, Ming Zhu, Juntao Tan, Thai Hoang, Zuxin Liu, Liangwei Yang, al. Agentohana: Design unified data and training pipeline for effective agent learning. arXiv preprint arXiv:2402. 15506, 2024a.
Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Hoang, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, al. xlam: A family of large action models empower ai agent systems. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume Long Papers), 2024b.
Shaokun Zhang, Jieyu Zhang. Dujian Ding, Mirian Hipolito Garcia, Ankur Mallick, Daniel Madrigal, Menglin Xia, Vietor Rühle, Qingyun Wu, and Chi Wang. Ecoact: Economic agent determines when register what action. arXiv preprint arXiv:241 1.01643, 2024c.
Shaokun Zhang, Jieyu Zhang, Jiale Liu, Linxin Song, Chi Wang, Ranjay Krishna, and Qingyun Wu. Offline training of language model agents with functions as learnable weights. In Forty-first International Conference on Machine Learning, 2024d.
Shaokun Zhang, Ming Yin, Jieyu Zhang, Jiale Liu, Zhiguang Han, Jingyang Zhang, Beibin Li, Chi Wang, Huazheng Wang, Yiran Chen, et al. Which agent causes task failures and when? on automated failure attribution of IIm multi-agent systems. arXiv preprint arXiv:2505.00212, 2025.
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhanghhi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thailand, 2024a. Association for Computational Linguistics. URL
org/ abs/ / /20/2443.222 13372
Yuanhang Zheng, Peng Li, Ming Yan, Ji Zhang, Fei Huang, and Yang Liu. Budget-constrained tool learning with planning. arXiv preprint arXiv:2402. 15960, 2024b.
13

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS
We used large language models (LLMs) only for light language polishing (e.g., improving grammar and readability). No LLMs were involved in research ideation, data analysis, or substantive writing.
B RELATED WORK
Tool Learning. Enhancing large language models (LLMs) with external tools has demonstrated
significant potential in addressing complex tasks (Qu et al. 2025 Wang et al. 2024bb Zhang et al.
2024d). Typical applications include integrating LLMs with search engines (Komeili et al. |02 I Lazaridou et al. 2022, Shuster et al. 2022] Zhang et al. 2025), calculator (Nakano et al. 2021
vision tools (Ma et al. 2024b),, and Python interpreters (Song et al., 2024 Wang et al., 2024a).
Research efforts aimed at improving LLMs' tool-use capabilities can be broadly divided into two categories.
The first one focuses on improving the backed LLM itself. This typically involves curating
large-scale supervised datasets (Qin et al. [023] Zhang et al. 2024b; Liu al. (024a) and applying either SFT (Qin et al. |0033 Zhang et al. 2024b] Liu et al. 2024a, Zeng et al.
2025a Abdelaziz et al., 2004| Chen et al. 2024a b) or DPO reinforcement learning (Zeng
et al. 2025b). Although these approaches may bring performance gains, they are fundamen- tally data-centric (Zha et al.] 2025) and rely heavily on access to high-quality, large-scale training corpora. Furthermore, enabling reasoning capabilities requires distilling reasoning trajectories from advanced models, which introduces additional costs.
The second category explores prompting- based techniques that do not alter the underlying model. These methods aim to elicit tool-use behavior through strategies such as in- contettext
learning with demonstrations (Paranjape et al.] [023] Kim al. 2023 Liu al.) 2024b
Zhang et al.
2024a) or the design of advanced prompting and workflow schemes Yao
et al. 2023b
Zhang et al.] [024c] Lu al.) 2025a [onn et al. 2025 [Schick et al.) 2023] 2025 Zheng et al.) 2024b; Wu et al. (025).. However, due to the inherent limitations of the base models, such prompting approaches often struggle with tasks that require intricate reasoning or complex multi-step actions. In addition, complex prompting workflows introduce substantial inference overhead.
Fawzi et al.)
LLM Reasoning and Reinforcement Learning. Recent efforts in the LLM research community have increasingly focused on improving reasoning capabilities, marking a shift from train-tine time scaling to test-time scaling (Muennighoff al. 2025), with the goal of enabling models to handle
complex problem-solving tasks (Wei et al. 2022) without perfoming heavy-lift modle traingrequiring
computationally intensive additional model training Ma al.) (2023) 2024a). Earlier approaches
often relied on step-level supervision or learned reward models to guide the model's reasoning
trajectory (Gao et al.] 2024: Li& Li 2024). More recently, DeepSeek-RI (Guo et al., 2025) has
demonstrated that simple based reinforcement learning can effectively induce strong reasoning behaviors. In this framework, reward functions are defined based on predefined criteria, i.c., checking whether the model's final answer matches the ground truth in math problems. These rewards target only the correctness of the final answer, allowing the model to learn the intermediate reasoning steps without explicit reasoning supervision. This RI Rstyyl reasoning paradigm has shown success across various domains, including mathematics (Shao et al., 2024), coding (Liu & Zhang 2025), and vision tasks (Shen al. 2025). In parallel, several recent studies have integrated external tools such as search engines (Jin et al. 2025) and code interpreters (Feng et al., 2025) into reinforcement learning frameworks for LLM reasoning. These approaches primarily focus on training LLMs to effectively utilize a single, general-purpose tool to support one single reasoning chain. In contrast, our work aims to enhance L.LMs tool-calling ability more general tool-use settings, where multiple tools with complex functionalities and argument structures are available.
C DATA DETAILS
In this section, we provide more details of both the training and test dataset.
C.I TRAINING DATA
XLAM (Zhag et al. 2024b) is a tool-calling dataset curated for supervised fine-tuning, consisting exclusively of single-turn interactions. Each turn involves either single-tool call or multi-tool call.
14

---

Published as a conference paper at ICLR 2026
In contrast, ToolACE includes both single-tur and multi-turn tool-calling interactions, with each turn containing either a single-tool or multi-tool call. After formatting and unifying the data, the final training set statistics presented in Table [ More information could be found in Section 3.1
xLAM
ToolACE
Single-Tun Tum
Multi-Ture
Single-Tum
Multi-Turn
Raw Data
60000
0
10500
800
After Process
60000
0
8183
1470
Table 5: The statistic information the training data preparation stage.
C.2 TEST DATA
We then provide a brief introduction for the benchmark involved in this paper.
BFCL. BFCL evaluates the LLM's s ability to call functions accurately. The data and leaderboard is consistly evolving and the benchmark widely-used the tool learning research. The current version (v3) categorizes the data into three types: Live, Non-Live, and Multi-turn. This paper primarily focuses on the Live and Non-Live categories. Non-Live refers to test cases generated synthetically or curated by researchers, while Live data consists of real user-contributed queries.
ACEBench. ACEBench is designed to evaluate tool-use capabilities with fine- fine-grained categorization which could be divided into three primary categories: Normal, Special, and Agent. In this study, concentrate on two cutecotgo within the Normal category: Atom and Single-turn. Atom cases consist of API calls that utilize specific parameter types. The Single-turn subset encompasses both sequential and parallel tool-calling scenarios.
APIBank. API-Bank tool-calling benchmark with two modes: Call and Retrieve + Call. The model is expected to interpret user intents from the dialogue and execute the corresponding local Python tools. Evaluation is based on whether the tool outputs align with the expected results. In our experiments, we specifically utilize the Call mode.
D MORE LEARNING CURVES
0.16
Nemotron- Reeeen--Reeaaar---ooooeo B-o-ard--Too-----88
Nemotron Research Tool N1 24b
0.6-
0.12
0.10
0.2
0.06
25
Nemotron- Tocl
20
Nametron- Cerrccn-Rseerrda
Nemotror-Resiarde Ao------r-------------
NemestonnRRseart dh-----N--48
0.02
40
50
53
200
120
140
20
40
50
20
200
320
140
20
60
150
120
140
Stop
Stop
Stop
(a) Reward
(b) Entropy Loss
(c) Minimum Response Length
Figure 7: More Learning curves across training steps for Tool-N1-7B and Tool-N1-14B, We report reward, entropy loss, and the minimum response length.
We further analyze the training dynamics in Figure 7 The reward values increase rapidly, while the entropy loss decreases gradually for both models, indicating stable learning process. Additionally, Figure(7c]shan shows the minimum response lengths over training. We observe that Tool-N1-7B exhibits steady increase, eventually converging to the value achieved by Tool-N1-14B. In contrast, Tool-NI- 14B maintains relatively stable response length throughout training. This suggests that the smaller model undergoes greater improvements in reasoning ability compared to the larger one.
E MORE IMPLEMENTATION DETAILS
We conduct RL experiments using the open-source Verl library. We perform hyperparameter tuning via grid search to ensure a stable and efficient training, focusing primarily on the learning rate, KL coefficient, entropy coefficient, number of rollouts, and batch size. The hyperparameters used during
15

---

Published as a conference paper at ICLR 2026
Hyperparameters
Value
Hyperparameters
Value
Batch Size
I024
Epoch Number
Max Prompt Length
4096
KL Coefficient
le-3
Max Response Length
8192
Entrophy Coefficient
0
Learning Rate
Te-6
Rollout Number
Temperature
0.7
Gpu Memory Utilization
0.6
Table 6: The detailed hyperparameters used for RL training.
training are summarized in Table The key hyperparameters used during training are summarized in T For both Tool-N1-7B and Tool-N1-14B, we adopt the same set of hyperparameters.
F PROMPTS
We use identical prompts across model training, evaluation, and reasoning trajectory distillation from advanced model DeepSeek-RI. The specific prompt is provided below.
Full Prompt Template
You are an expert in composing functions. You are given question and of possible functions. Based on the question, you will need to make one or more function/tool calls to achieve the purpose. If none of the function can be used, point it out. If the given question lacks the parameters required by the function, also point it out. You should only return the function call in tools call sections.
# Tool
Here is a list of functions in JSON format that you can invoke:
<tools>
[tools)
</tools>.
In each action step, you MUST:
1. Think about the reasoning process in the mind and enclosed your reasoning within <think><<thinke> XML tags.
Then, provide a json object with function names and arguments within <tool call> </tool shool_call>>>o XML tags. i.e., <tool_ <tool_call><"nanl call>>"name": <function-name>, "arguments"": <args-json- objects, "name": <function nnme2>> "arguments": cargs-json-objecte ...]o/tool___.....oooooo//llls_ _aal_> 3. Make sure both the reasoning and the tool call steps are included together in one single reply.
A complete reply example is: <think>To address the query, need to send the email to Bob and then buy the banana through walmart,</hinke <tool_ [tool_call>>>>>>t "naame"""mmai""" "argu- ments" ":"""""""""" "receiver": "Bob". "content": "I will bug banana through walmart", "name": "walmart", "arguments": "input": "banana" "aaoaal"/ttoo_caaa calls. Please make sure the type of the arguments correct.
Figure 8: Full system prompts used during model training and inference.
16

---

Published as a conference paper at ICLR 2026
G EXAMPLES
Example 1: Single Function Calling
Query: Can you search for a hair stylist in Lafayette in Louisiana for me, please? Output: <think>To search for a hair stylist in Lafayette, Louisiana, will use the Services___FinProviinee
function with the required city parameter.</hinks <tool call>[["name" "Services FindProvider", "arguments": f"city" "Lafayette, LA"]]]/ttol call>
Example 2: Multiple Function Calling
Query: Imagine you are a drone operator. You are currently operating a drone that is at a point (5, 7) in the sky. You asked to move the drone
point (10, 15). After reaching the new point, you are again asked to move the drone to another point (20, 25). Can you calculate the total distance the drone has traveled using the Euclidean norm method?
Output: <think>To calculate the total distance the drone has traveled, I will use the myathhhypot function calculate the Euclidean distance between each pair of points. First, will calculate the distance from the initial point (5, 7) to the new point (10, 15). Then, will calculate the distance from the new point (10, 15) to the final point (20, 25). The sum of these two distances will be the total distance traveled by the drone.</think> <tool [rool_calls[[ ["nname": "math.hypot "arguments": ("x": "y" 7} {" ("name": "math.hypot "arguments": ("x": 10, "y": 15 ("name": "math.hypot" "arguments": ("x": 20, "y": 25) }]</tool_call>
17


Reviews:
[{"review_id": "cx1xwWN108", "reviewer": "Reviewer_K5pn", "summary": "The authors propose a rule-based reinforcement learning method to enhance the tool calling abilities of LLM based on GRPO. Their method uses a binary reward function with light supervision, e.g. it only checks functional correctness and format validity. They claim that it improves the generalization abilities vs conducting SFT (no experiments are provided to back up such claim). The authors present results that their method improves performances by conducting evaluation using a variety of tool calling benchmarks: BFCL, ACEBench and API bank. The authors also present a thorough set of ablations to validate their design choices.", "questions": "- What percentage of data gets filtered out when you remove the samples with invalid tool calls?\n- You claim throughout the paper that it improves reasoning and doesn't lead to pseudo-reasoning. However, there is no experiment to back up this claim. How can you show case that this is indeed the case experimentally?\n- The QWEN model used is outdated, it would be good to present some results with more recent models.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The authors present a very thorough set of ablations to validate their design choices: \n- SFT vs RL vs SFT+RL, \n- Scaling laws that showcase that using a more powerful model is better and good generalization accross different LLMs as backbones\n- Ablation on reasoning format vs not using it\n- Ablation on binary vs more fine grained reward function\n- Ablation on data decomposition\n\n- The paper is well written and easy to follow", "weaknesses": "- Novelty and originality: It is hard to assess the overall contribution since no other baselines are presented of works that provide ways to interweave reasoning with tool calling.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ICoZz5WPsH", "reviewer": "Reviewer_Emqt", "summary": "This paper presents Nemotron-Research-Tool-N1 (Tool-N1), a series of tool-calling language models trained using a rule-based reinforcement learning (RL) paradigm. The core contribution is the application of a \"R1-style\" RL approach, which uses a simple binary reward to judge the format and functional correctness of tool calls, without supervising the intermediate reasoning steps. The authors conduct a systematic empirical study, demonstrating that their method outperforms strong baselines, and intriguingly, that a pure RL training pipeline can surpass the commonly adopted \"SFT-then-RL\" paradigm in this domain.", "questions": "- How the name \"NEMOTRON-RESEARCH-TOOL-N1\" is determined? There seems to be no evident hint in the paper. Considering to use a shorter name instead?\n- How is the Reward Model trained? There should be some explicit introduction (even simply refering to some popular methods such as Bladley-Terry) to make the methodology self-contained.\n- The training is conducted by GRPO. How would the performance change if using PPO instead? Is there any insights compared these two RL methods?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper effectively argues that supervised fine-tuning (SFT) on distilled reasoning traces can lead to \"imitative\" or \"pseudo\" reasoning. The proposed rule-based RL method is a compelling alternative that incentivizes the model to develop its own internal reasoning strategies, promoting better generalization. The motivation is clear and grounded in current limitations of the field.\n- Strong Empirical Evidence.\n- Rigorous and Insightful Ablation Studies.", "weaknesses": "- There is no computational effiency analysis. How fast is it running compared to baselines? Especially, how the speed becomes when the number of candidate tools becomes large? (since LLM receives the textual input with all the tools included)\n- There should be some brief introductions to recent tool-calling baselines, either in Section 4.1 or a Related Work section.\n- There is no public code or models which makes the experiment replication difficult.\n- There is no explicit paragraph of major contributions in Introduction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qsAsUffx9y", "reviewer": "Reviewer_5Uca", "summary": "This paper presents a family of open-source tool-calling LLMs trained via rule-based reinforcement learning (R1-style RL) rather than conventional supervised fine-tuning (SFT). The key innovation lies in using a binary reward that verifies only output format and tool-call correctness, omitting supervision on reasoning trajectories. Besides the main experiments, the authors further conduct systematic ablations, finding that pure RL outperforms SFT-then-RL, that binary rewards outperform fine-grained ones, and that structured reasoning tags (<think>, <tool_call>) are essential for generalization.", "questions": "No extra question. See weakness above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper demonstrates that pure rule-based RL can outperform traditional SFT and SFT+RL pipelines for tool-use, which is an important insight contradicting existing practice.\n\n2. Tool-N1 models surpass GPT-4o and other closed- and open-source tool models (e.g., Hammer2.1, xLAM-2) on all tested benchmarks, validating the method’s effectiveness.\n\n2. This paper includes ablations on reward granularity, data composition, model scaling, and backbone choice, offering valuable design guidance for future RL-based tool-use research.", "weaknesses": "1. The paper’s contribution is largely empirical; it lacks theoretical insight into why rule-based RL generalizes better than SFT or how binary rewards shape reasoning policies.\n\n2. A binary {0,1} reward could cause credit assignment inefficiency or instability, yet no analysis or mitigation (e.g., shaping, curriculum) is discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)