AutoTool: Automatic Scaling of Tool-Use Capabilities in RL via Decoupled Entropy Constraints
Authors: Yirong Zeng
Xiao Ding
Yufei Liu
Yuxian Wang
Qunyao Du
Yutai Hou
Wu Ning
Haonan Song
Duyu Tang
Dandan Tu
Bing Qin
Ting Liu
Venue: ICLR 2026 Poster (2026)
Keywords: llm
RL
tool use
auto think
Primary area: reinforcement learning

Abstract: Tool use represents a critical capability for AI agents, with recent advances focusing on leveraging reinforcement learning (RL) for test-time scaling to achieve better performance through more deliberate reasoning. 
However, there are some key challenges in current RL-based scaling approaches: 
(a) direct RL training often struggles to scale up thinking length sufficiently to solve complex problems, 
and (b) scaled-up models tend to overthink simpler problems, resulting in substantial token inefficiency.
To address these challenges, we propose a novel training paradigm that first employs warm-up supervised fine-tuning to help models distinguish between simple and complex problems, followed by RL that enable models to automatically determine appropriate reasoning trajectories. 
Furthermore, to tackle the issue of automatic thinking-length scaling, we discover that entropy-based optimization objectives effectively maintain model diversity while successfully unlocking the model's scaling capabilities.
Based on this insight, we introduce an entropy-based long-short reasoning fusion RL strategy. 
Our experiments on three benchmarks demonstrate that model successfully achieves auto-scaling for efficient tool use, achieving significant 9.8\% accuracy improvements while reducing computational overhead by ~81\%.

Published as a conference paper at ICLR 2026
AUTOTOOL: AUTOMATIC SCALING OF TSOL-USE CA- PABILITIES IN RL VIA DECOUPLED ENTROPY CON- STRAINTS
Yirong Zeng', Xiao Ding"", Yufei Liu?, Yuxian Wang', Qunyao Du', Yutai Hou Wu Ning', Haonan Song", Duyu Tang", Dandan Tuè, Bing Qin', Ting Liu!
Harbin Institute of Technology, SCIR Lib
Peking University
Huawei Technologies Ltd.
"Corresponding authors
(yrzeng, xding) @iir. hit edu. en, cn, houyutai@huawei.cm
ABSTRACT
Tool use represents critical capability for Al agents, with recent advances focus- ing on leveraging reinforcement learning (RL) to scale up the explicit reasoning process to achieve better performance. However, there some key challenges for tool use in current RL-based scaling approaches: (a) direct RL training of- ten struggles scale up thinking length sufficiently to solve complex problems, and (b) scaled-up models tend to overthink simpler problems, resulting in sub- stantial token inefficiency. To address these challenges. we propose novel train- ing paradigm that first employs warm-up supervised fine-tuning to help models distinguish between simple and complex problems, followed by RL that enable models to automatically determine appropriate reasoning trajectories. Further- more, to tackle the issue of automatic thinking-length scaling, we discover that entropy-based optimization objectives effectively maintain model diversity while successfully unlocking the model's scaling capabilities. Based on this insight, we introduce entropy-based long-short reasoning fusion RL strategy. Our exper- iments on three benchmarks demonstrate that model successfully achieves auto- scaling for efficient tool use, achieving significant 9.8% accuracy improvements while reducing computational overhead by -81 %o
INTRODUCTION
Integrating agentic large language models (LLMs) with external tools has emerged as a pivotal advancement, and has become a defining feature of advanced agentic models (OpenAl 2025 K2 2024). It significantly enhances a model's ability to address complex tasks (Qu et al. 2025 Wang al. 2024), and opens up many practical uses across different fields. For example, supports the automation of reasoning tasks (Jin et al. 2025 Li et al., 2025b), and enables agent applications (Zihan et al.) 2025; Ouyang al.) (025). Therefore, research on agentic tool use represents a critical pathway toward artificial general intelligence. In this task, models respond to queries by dynamically
selecting and invoking relevant tools from an available pool.
Test-time scaling (TTS) is a approach to language modeling that uses extra test-time compute to improve performance (Muennighotf et al., 2025). Currently, scaling up a model's explicit reasoning length via Reinforcement Learning with Verifiable Rewards (RLVR) is effective to achieve TTS. Compared to the prevalent Supervised Fine- Fine-Tuning (SFT) approach, which imitates reasoning pat- terns from labeled high-qualit examples to teach models using tools (Liu al. 2024 Zhang al.) 2024a), RLVR better fosters intrinsic reasoning, rather than making models memorize training tra- jectories (Cen et al. 2025). It has demonstrated robustness in mathematics (Shao al. 2024) and

---

Published as a conference paper at ICLR 2026
0.-
Keep Lengt
ACDUSSS Rovard
0.40
0.70
0.65
24
1420
095
Bowerd
07-
035
Respimer Lengh Lenghhl0
0.65
1200
Respeese Token OUT
Accurdty
43-
86-
0.30
Accuray
1000
-085
9.5
0.23
0.60
04
0.20
0.58
35-
0.15
0.55
400
Accormty
-0700
0.-
1/
0.33
0.10
32
50
100
011
25
125
0.50
328
7B
Troin Step
Troin Shep
Oven Rodd
(a) direct RL training in Math
(b) direct RL training in tool use
(c) Cost in scaled-up models
The training paradigms for TTS in tool-use: (a) direct RL enables scaling up response length as accuracy improves in mathematical tasks; but (b) it fails to scale in tool-use tasks, where reasoning collapses into short trajectories; (c) scaled-up models (e.g., distillation models) incur significant token costs, as they require lengthy reasoning trajectories for all queries.
Figure 1:
coding (Pan & Liu 2025), while also driving a paradigm shift from SFT to RL in LLM training. Thus, exploring suitable scaling strategies is critical to advance effective agentic tool-use.
To this end, pre-analyze the training paradigm for TTS in tool use, as shown in Figure Under direct RL training, contrary to mathematical tasks where response length scales with improving accuracy, observe that models suffer from reasoning collapse in tool use, phenomenon where models fail to sufficiently extend thinking"leneee length to solve complex problems. More importantly, we find many tool-use problems can be solved with short reasoning trajectories, yet scaled-up models generate excessively long trajectories that cause unnecessary resource consumption. Therefore, an adaptive model that dynamically integrates short and long reasoning is highly desirable.
More analysis in Section reveal that low entropy. which quantifies a model's response certainty and exploration capability, leads to insufficient reasoning length, limiting the robustness of L.LMs in tackling hard problems. These findings motivate our proposed method, which decouples long and short reasoning to prevent dominance interference while incorporating entropy constraints to enable long-trajectory reasoning. Therefore, propose decoupled adaptive entropy constraint strategy for RL. first performs warm-up using a constructed mixed dataset of long and short reasoning trajectories to perceive data difficulty. The strategy decouples the policy loss between short and long reasoning, then applies varying entropy constraint strengths to regulate the thinking mode while maintaining concise responses for simple problems. This enables differentiated exploration control across reasoning modes, with adjusted entropy strength set above a target entropy in long reasoning to preserve exploration capacity.
Experiments on three benchmarks show: (1) Our -BB model leads at comparable size models (e.g., +11.9%% compare SFT-model). (2) Beyond performance, our auto-scaling model boosts accu- racy by 9.8% compared to the distilled model and cuts inference token cost by -8 1 %. Notably, our model's 's thinking rate reaches 45% in complex scenarios but 0% in simple ones. Moreover, visu- alizations of the training process confirm that our approach generates concise responses for simple cases while extending reasoning trajectories by for complex questions. This contrast demon- strates that the model has learned to automatically adjust test-time scales based on sample difficulty, ultimately supporting improved inference efficiency.
PRELIMINARY STUDY
In this section, present extensive experiments to highlight the challenges of achieving test-time scaling for agentic tool use, and thereby motivate our method.
In this paper, the terms thinking and long reasoning used interchangeably, both referring to responses that contain an explicit reasoning process.
2

---

Published as a conference paper at ICLR 2026
Performance bifficulty Level
Revard
Length
Enirropy
100
Gay Mind
Eate
290
0,6
0.
500
Miet
50
Wixe
Easy
Madua
100
150
200
300
250
100
250
Diffieutty Level
Step
Step
Step
Figure 2: Impact of difficulty distributions. Easy and Mediam converged successfully, while Hard failed (a, b). However, collapse occurred across all three subsets (c), with the same trend observed entropy (d). This indicates that data distribution no correlation with collapse, whereas low entropy exhibits strong positive correlation.
2.1 TASK OVERVIEW
In agentic tool use, the LLM receives a user query q along with candidate tools, represented as
T = 447.. The purpose of LLM is to fulfill the user's intent by executing a specific sequence of tools. We formalize this decision-making process as y tl 9,8; T), where x x-)) represents the policy model, denotes the task state (e.g., historical context and y represents the actions taken by the model, such as selecting or executing tool calls. A review of related work is provided in Appendix A
2.2 TRAINING PARADIGMS ANALYSIS
We analyze training paradigms for scaling reasoning process in tool use, including RL training and SFT with distillation, using Qwen2.5-serie models to conduct training on the public ToolACE dataset (Liu et al. 2024) and evaluation via BFCL (Yan et al. 2024) (details in Section 4.1.
(1) For direct RL, we applied RL (specifically GRPO Shao et al. 2024)). ) As shown in Figure 3b observe a divergence between the model's performance and response length: training steps increased, performance improved while response length decreased sharply. This indicates the rea- soning patterns collapsed into short reasoning trajectories, failing scale-up in test-time. This result contradicts widely accepted findings from training on complex reasoning tasks (e.g., mathematics) (Hugging Face 2025 Zeng al.] 2025b), as shown in Figure 3a where adopt experimental sults from Zeng et al. (022a)). The evaluation results presented in Table [[inicate that performance in complex tool-use scenarios (e.g.. Multi-Turn) decreased noticeably compared to the distilled SFT model. This indicates that the reasoning collapse phenomenon limits the model's 's robust performance on complex problems.
(2) For SFT with distillation. we conducted base SFT and distillation from reasoning LLM (DeepSeek-AI] (2025b). respectively. As shown in Figure Ic. distilled models showed no notice- able accuracy gain over the base SFT, but increased output token costs by more than 10x. This suggests that many agentic tool-use problems can be solved with short reasoning trajectory while excessive long trajectory leads to unnecessary resource consumption.
2.3 PRE-STUDY ON REASONING PATTERN COLLAPSE.
To investigate the causes of reasoning pattern collapse, we conducted an in-depth analysis of data difficulty distribution and information entropy.
2.3.1 DATA DISTRIBUTION
Intuitively, we hypothesized that the sample difficulty distribution might exert critical influence. Guided by this hypothesis, we used a base model to perform 8 rounds of reasoning on the training data and calculated @ The resulting distribution (shown in Appendix B) reveals that easy samples (with 8/$ correct inferences) and hard samples (with 0/8 correct inferences) accounted for 47% and 31.8 31.8% of the dataset, respectively, while medium samples made up a smaller proportion.
3

---

Published as a conference paper at ICLR 2026
Table 1: Evaluation results on the BFCL benchmark, which includes three sub-metries: Non-live, Live, and Multi-Turn (including multi-turn and long-context tool-use scenarios). denotes target response length in response constraint, and denotes coefficients of entropy constraint.
Model
Think?
Non-Live
Live
Multi-Tum
Overall Acc
Base LLM
86.46
67.44
7.62
53.69
SFT
86.65
75.11
6.75
56,90
w/ distilled SFT
87.35
79.59
16.95
59.23
w/ GRPO
87.06
78.22
8,38
57.81
w/ length constraint
+ 100
87.30
71.23
7.92
55.37
+| 50
87.76
78.43
8.78
58.12
+1 10
89.76
77.33
8.89
58.27
w/ entropy constraint
+3 = 1c-2
87.47
79.13
9.48
59.33
+ 5c-2
87.21
77.96
10.02
58,91
+ 1e-1
88.32
80.42
15.86
61.86
Resporar Length
(b) Achor
00 Respante Length
Action
150
400
350
175
300
300
075
250
0.0
200
100
023
100
0.25
0.00
0.00
Shop
Shop
(a) RL with entropy constraint
(b) RL with length penalty
Figure 3: Training dynamics visualized for entropy constraint (a) and length penalty (b). The entropy
constraint partially increases response length, yet the length penalty does not mitigate low entropy.
We then conducted separate RL training runs on these three subsets, evaluated the resulting models, and reported their training dynamics Figure / Notably, reasoning pattern collapse persisted across all three subsets: after an initial exploration phase, easy samples led to rapid convergence, intermediate samples resulted in slower convergence, and hard samples showed no convergence. These findings indicate that the sample difficulty distribution can slightly reduce convergence speed, but no correlation with collapse has been observed.
The actor model's information entropy quantifies its exploration capability during training. As shown in Figure 20, entropy decreases rapidly, with dynamics closely aligning with reasoning pat- tern collapse. Additionally, comparing three subsets, the final converged entropy increases sequen- tially from simple to complex samples. This reveals: simple problems elicit high certainty in short reasoning (perceiving extended exploration risks suboptimal solutions); complex problems face in- herent challenges, with short reasoning advantage dominance further discouraging exploration and driving default to brief responses. This finding demonstrates strong positive correlation between low entropy and collapse.
2.3.2 INFORMATION ENTROPY CONSTRAINTS
To explore reasoning collapse-entropy connections, we incorporated entropy constraints into the policy loss function. Inspired by He (025), we designed mechanism to maintain the entropy (e) at reasonable level throughout training. The entropy loss is defined as:
loss = 3 I(ex S tgt-eet]
(1)
where is the training step, 3 is the coefficient, and we set tgt-ent=0.1. Notably, the entropy loss is only activated when 6 tgt-ent, ensuring the model's entropy remains lower-bounded by the target value. For comparative purposes, we also implemented a short-response penalty by configuring the reward function penalize response below the target length 1. The evaluation results
4

---

Published as a conference paper at ICLR 2026
loss C = C. + H, KHH, & Hij
H
B
Train Stage
First tokens distribution
[think] Correct Resp
+0.5
Query
[think]
rollont
sample
[think] Wrong Resp
-0.5
Answer
Policy Model
[na_think]]
[ne_thinkk Correct Resp
+1
uprm-up
other Tokens
[no__ [oi_think] Wrong Resp
Loss C = Co H; 14H H.) 8,
Inference Stage
Irput tokens
Output tokens
think//made] resp with think
Auto
user promptoasiistant//moda
no. think[mmode resp
Policy Model
[mode]think//mode
with think
Controllable
resp without think
Figure 4: The overview decoupled adaptive entropy constraint. It achieves automatic scaling by decoupling different reasoning modes through the application of differrnniated entropy constraints. Adaptive entropy constraint strength for long reasoning. During the inference, the model can auto- matically controllably switch inference modes by pre-pending response prefix in Input tokens.
(presented in [ show that the length constraint did not improve the model's ' test performance. contrast, the entropy constraint yielded partial performance gains (visualizing the training process in Figure 3). However, the effectiveness is highly sensitive to 3: in multi-tum, the maximum positive gain was achieved when | = 1e 1, whereas results for other 3 are comparable to w/ length constraints. This sensitivity highlights the difficulty of pre-selecting indicating that dynamically adjusting 3 during training is necessary.
optimal entropy coefficient,
Therefore, we propose novel strategy: decoupled adaptive entropy constraints. It decouples tropy constraint for short and long reasoning and adaptively tunes the entropy coefficient, addressing both the collapse caused by low entropy and the sensitivity of static coefficients.
METHODOLOGY
Our method first performs warm-up SFT perceive sample difficulty (details in 3.1). followed by RLVR training with decoupled adaptive entropy constraints, as shown in Figure
3.1 DATA PREPARATION AND WARM-UP
To support robust general-purpose tool use via RI,, we constructed a mixed dataset covering diverse tool-use scenarios from public sources: ToolACE (Liu al. 2024). xLAM (Zhang al.) 2024a Prabhakar et al. 2025). Hermes Function-Calling (interstellarninjj] 2024). More details are provided Appendix B To create a balanced dataset encompassing diverse complexity levels and tool usage scenarios, we randomly downsampled the raw data. Moreover, we adopted the following strategies develop Public agentic Tool-use dataset (PubTool), as presented in Table
Warm-up Training. To help the model initially perceive data difficulty, we propose SFT for warm- up training by mixing long and short reasoning data. To construct such mixed thinking data, we performed multiple inferences (calculating pass@8) on the training data using Qwen2.55-BBInntrrect (no-thinking model) and Qwen3-32B (thinking model), respectively. For each response turn, we adopted the ground truth as the label if the no-thinking model's output was correct; otherwise, adopted the thinking model's answer with explicit long reasoning if it was correct. More Details of data preparation Appendix {E) to enable the model to select reasoning modes based on data difficulty. Finally, we conducted SFT the base model for warm-up, preparing for subsequent RI. scaling.
shown in Appendix {B] We design auto- thinking template (details in
5

---

Published as a conference paper at ICLR 2026
Table 2: Data statistics of Tool in data collection and construction. Subscript text in the SFT data table indicates the thinking rate in all turns.
ToolACE
XLAM
Hermes Function- Calling
Raw Data
11.3k 11.3k
65k
7.1k
Downsampled
15k
7.1k
PubTool
SFT data 8.2k().2%)
RL data
Processed
7k
Quality Refinement for RL data. To efficiently support auto-scaling RL training, we employed the following data enhancement strategies: First, from data distribution analysis in Section $2.3.1 we observed that the original dataset was dominated by overly simple and excessively difficult samples. Overly simple samples offer limited value for RL training and lack generalization, while overly difficult samples either exceed model capabilities or contain noise. We therefore randomly removed half of both simple and difficult samples to balance the dataset distribution. Additionally, inspired by Li et al. (2025a), we prioritized training samples based on their alignment with model learning trajectories. Specifically, changes in their reward scores, and calculated each sample's ' variance relative to the average reward. Lower variance indicated higher alignment. Through these processes, we downsampled the RL dataset from 2 to 7k samples. For detailed analysis of its effects, please refer to Appendix
performed multi-epoch GRPO training on all training data, computed
3.2 DECOUPLED ADAPTIVE ENTROPY CONSTRAINTS
To enable automatic scaling in agentic tool use, we propose a decoupled adaptive entropy constraints strategy for RLVR. The objective policy loss integrates the surrogate objective from native RLVR (e.g.. GRPO) with a mechanism that: (1) decouples entropy regulation between short and long trajectories; (2) adaptively adjusts the entropy strength in long reasoning trajectories to preserve exploration capacity.
Specifically, let Te be the policy, H; - -Ba-re(-lsa) [log ma (als;)] is the entropy at step and m. f0, 1} indicator variable: equals 1 if the action step is short trajectory and 0 if it is long trajectory. We apply decoupled entropy constraints based on policy model's response trajectory type: (1) 3r fixed coefficient for short paths (to prevent excessive exploration), (2) Br: adaptive coefficient for long paths (learned dynamically).
The sample-level policy loss is defined as:
Bi 3g m -IH, S H.} + B1 (1-m.) I[H, S Hi).
(2)
1 N c. = N
Σ min (piÅ,, clip(p;, c,1 ()te))))))) cc A.) B.H.
(3)
where 3; adapts the entropy penalty per sample, P: To (a(a,(s))/ ( (agls)). Hi and H, denote target entropy of long reasoning and short reasoning, and Å, is the estimated advantage based on reward scores in Section 3.3 The key design is the decoupling of entropy weights via m;, enabling distinct regularization strategies.
Adaptive Entropy Coefficient Loss. Entropy regularization is highly sensitive to the choice of coefficient, making it difficult to select optimal coefficient in advance. This motivates dynamic adjustment of the entropy loss coefficient. To automatically adjust the entropy strength for long trajectories, we introduce an adaptive loss that updates 3r based on the deviation of actual entropy from a target level. The loss is computed only on steps belonging to long trajectories (m, = 0) and defined as:
N
C3
1
m.) B (H. Hi).
(4)
E, m,)

---

Published as a conference paper at ICLR 2026
where Hi is a predefined target entropy. The coefficient 3t is updated by minimizing C if H; < H1, 31 increases to encourage exploration; if H; > Hi, it decreases to suppress excessive randomness. In contrast, 3g remains fixed during training.
3.3 AUTO THINKING REWARD MODULE
In this module, the model's output is evaluated using a rule-based reward (DeepSeek-Al 2025a Meng et al. 2025) to compute the estimated advantage for the objective loss Lp. Specifically, for each question q. model generates G completions {0r 02,...,ocii using T0... This reward module combines format and answer rewards score each completion.
Format Reward. The format reward (o)) E {0, 1 evaluates whether the output adheres to the required structural template. We define two valid reasoning modes: think and no-think, each with strict syntactic constraints:
[mode] think [mode] mno_think//mooeeeooo [no_think] \n [/nn_tthinn answer
htinkt [/mode] [think] reasoning process here [/think] answer anawer
This design encourages explicit reasoning for complex problems via the think mode, while allowing direct generation for simple queries via no-think, reducing computational overhead. During the inference stage, controllable reasoning modes are achieved by prepending special tokens to the input, as depicted at the bottom of Figure 4
Answer Reward. We check the correctness of the tool call by comparing it against the ground-truth annotation y" Tool-call outputs are parsed into structured dictionaries, enabling exact matching of both the function name and all required arguments. To encourage balance between reasoning efficiency and accuracy, we design an asymmetric reward based on the mode (think or no-think):
+1.0, if o, y' no-think,
+0.5, if o, think,
Ramswer (o.)
(5)
0.5, o, # y" think,
1.0, o. # y" no-think,
incentivizes short responses when they are correct, while encouraging long reasoning when mis- takes occur, prompting more careful processing in uncertain scenarios.
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
We the open-source Owen2.5-7B-Insttne Base, SFT-trained, API-based Frontier, and RLVR-traine models. Additionally, we compare the series of models trained the base model using PubTool data. See Appendix [ for more details.
base model. We compared four baseline types:
(1) BFCL Yaan et al., 2024) provides a comprehensive dataset comprising 4k+ instances (updating), consisting of Non- live (with expert-curated simple tools), Live (with user-contributed complex tools), Multi-turn (with multi-turn & multi-step tool use) samples. (2) API-Bank (Li et al., 2023). which consists of 314 tool-use dialogues and 753 API calls. This dataset evaluates models abilities to correctly invoke known API (L-1) based on query and retrieve and call APIs from tool list (L-2). (3) ACEBBenc (Chen et al. 2025) is 2k-entry benchmark for assessing agentic tool use, using its summary score in "normal" evaluation type (covering single-turn and multi-turn scenarios).
Evaluation Dataset. The following benchmarks are used for evaluation:
4.2 OVERALL PERFORMANCE
The overall performance of models are shown Table and Figure S Firstly, the results indicate that model consistently achieves corresponding best performance at comparable scales (-7B). For instance, compared to PubTool-SFT, Auto Tool-BB with automatic think achieving +1 1.95 point improvement. And relative Base model, it also has remarkable boost with 16.43%. Secondly,

---

Published as a conference paper at ICLR 2026
model demonstrated its more superiority in challenging scenarios (e.g.. achieves +28.5% im- provement compare to PubTool-SFT Multi-turn). This demonstrates that our method realizes a strong robustness enhancement in complex scenarios.
Moreover, our model outperforms most SFT-trained and RLVR-trained models BFCL, and demonstrates comparable performance with the frontier models. also shows consistent advanta- geous performance on API-Bant and ACEBench compared with baselines in Figure For example, on ACEBench, our model achieves 6.5 improvement compared to GRPO and 5.9 improvement compared to Distilled. Finally, in the inference controllable mode, when forced to think, the overall performance on par with auto think; when forced not think, the effect on Multi-turn is signifi- cantly improved compared to no-think models (e.g., PubTool-SFT
Table 3: Comparison on the BFCL benchmark. Overall Acc denotes the average performance on three subsets. indicates a single-turn tool use model; denotes models trained on PubTool data with a specific method. The subseript denotes the thinking rate.
Type
Model
Non-Live
Live
Multi-Turn
Overall Acc
1LaMA-331188 1-88u-nnttrct
84.21
61.08
9.62
50.87
Base
Owen2.5-7B-Instre
86.46
67.44
7.62
53.69
Qwen2,5-32B-
85.81
74.23
17.75
59.67
GPT-44--024-11--20 03-2025-04-16
87.67
79.88
43.00
70.42
Frontier
81.42
73.43
56.12
70.32
Gemini-2.5 P.5-Prr
89.54
76.83
30.62
65.48
10mmmer2.117h(aaaa et al.
2024)
88.65
75.11
23.50
61.83
ToolACF-8B (L.uu et al.
2024
87.54
78.59
7.75
58.42
SFT
Xl.n
2024a
81.06
75.22
10.00
54.75
PubTool-SFT
88.98
77.28
9.68
58.17
BubTool-Distilles
87.73
78.64
15.65
60.30
DeepSeck-RR R1-528
75.20
77.30
38.88
63.79
Qwen3-8h Team
2025a
88.81
78.54
33
66.34
QwQ-32B Team
2025b
87.33
75.61
14.50
58.30
<LLVR
Tool-N1-7B iZhang
2025b
89.25
80.38
ToolRL-7B Qran el al. 2025
82.21
74.90
18.12
58.38
Thinkless(Fang et al. 2025
86.92
77.62
24.64
63.06
Adactri(Huang cf al.
2025
86.36
73.12
15.63
58.37
PubTool-GRPO'
88.87
78.93
10.77
60.13
AutoTool-7B + think
89.76u%
80.22485
38.18.5%
70. 70.12
Ours
89.86
80.43
39.28
70.71
+ no-think
87.36
78.60
27.63
63.34
(a) Accuracy APIBank
(b) Accuracy on ACEBench
75
74,5
85
83.2
Accuracy
70
68.3
69.3
80
77.6
76.7
65
75
57.5
50
Dishill
Figure 5: Performance of methods using training data PubTool on APIBank and ACEBench.
4.3
DEEP ANALYSIS STUDY
4.3.1 ABLATION STUDY
To evaluate the effectiveness of key components in our method, we conducted an ablation study with the following variations: (1) Replaced the adaptive entropy coefficient with fixed one (w/o
8

---

Published as a conference paper at ICLR 2026
Overal Performnnee
Taken Cost
10
70
500
0.8
60
#TToeons
600
50
40
200
Bode
arpa
adotaal
thite
9064 dst
-ml
thisk
guse 5cn0
599
Tool auf
Figure 6: Inference efficiency analysis results, including performance, token cost, ACU.
adapt coeff): (2) Replaced the decoupling loss with a unified loss with fixed entropy constraint (w/o decouple); (3) Removed data quality refinement (w/o data refine). We also included Qwen2.5-7B- Instruct Base Model for comparison. As shown in Table compared with the baseline, our full model delivers a significant improvement of 16.43 points in Overall performance. All components are essential to our method, and removing any component causes clear performance drops: (1) w/o duta refine brings the largest 6.43% Overall reduction, highlighting high-quality data as core foundation. (2) w/o adapt coeff' leads to a 10.53% Multi-turn decline, proving its value in stabilizing multi-round interactions; (3) w/o decouple results in 2.34% Overall drop, showing decoupling avoids objective interference.
Table 4: The strategy ablation performance (1 increase, decrease, values are relative percentage changes from the Our (w/ all) model).
Models
Non-live
Live
Multi-turn
Overall
Base Model
86.46
67.44
7.62
53.69
Our (w/. all)
89.76
80,22
38.18
70.12
w/o. data refine
88.22 11.54
73.29
16.93
26.84
J11.34
63.69
J6.43
w/o. decouple
87.35
75.98
14.24
27.65
J10.53
64.23
15.89
w/o. adapt coeff
88.73
11.03
78.73
11.49
32.14
16.04
67.78
12.34
4.3.2 INFERENCE EFFICIENCY ANALYSIS
Given the trade-off between reasoning path length, model size (-B), and performance, we introduce new metric, Accuracy per Computation Unit (ACU), to better capture this balance and assess model inference efficiency (Ma et al. 2025). It is defined as:
Accuracy
ACU
(6)
#Params #Tokens
Since the ACU value typically falls within the range of 10-5 to 10-3, we report it in units of 10 for improved readability. In addition, report the thinking rates of model across all submetrics.
The experimental results are summarized in Figure From the results, we observe that AutoTool achieves the second-best overall performance: it reduces token cost significantly by 81%, requiring only about -183 tokens compared to the distilled model (-966 tokens). Notably, with the forced no-think inference mode, AutoTool attains the optimal ACU score (0.97), demonstrating excellent inference efficiency. Even with the think inference mode, it still delivers the highest accuracy while cutting the token cost by half relative to the distilled model. Additionally, T that the our model's 's thinking rate reaches 45% in the Multi-Turn scenario but 0% in the No-Live scenario. The training process visualized in Appendix Section shows our model extends reasoning trajectories for complex questions the model has learned to automatically adjust the test-time scale based on sample difficulty, which effectively supports the improvement of inference efficiency.
x5, while enabling concise responses for simple ones. This suggests

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
This study focused on addressing challenges in integrating agentic LLMs with tools by optimiz- ing the RLVR paradigm. Our research first identified two critical issues: excessive resource sumption caused by unnecessary long-trajectory reasoning, and the reasoning collapse phenomenon under the direct RL training, hindering effective scaling. To solve these, we proposed decou- pled adaptive entropy constraint strategy, which enables the model to automatically adjust reasoning scales based problem difficulty, thereby balancing performance and inference efficiency. Exper- iments on three benchmarks confirmed the strategy's effectiveness, boosting accuracy while cutting inference token cost significantly. This work advances RL-based agentic tool-use training and pro- vides practical auto- scalins solution for efficiently handling tasks.
ACKNOWLEDGEMENTS
The research in this article is supported by the New Generation Artificial Intelligence of China (2044YE00003000), National Natural Science Foundation of China under Grants U22B2059 and 62576124.
ETHICS STATEMENT
This work strictly adheres to the ICLR Code of Ethics: it involves no human subjects, uses datasets compliant with original licensing agreements (ensuring privacy and legal compliance), and avoids discriminatory biases experimental design/results; all authors confirm adherence, with no con- flicting sponsorships.
REPRODUCIBILITY STATEMENT
For reproducibility, key details are referenced across the main text (methodology Section exper- imental setup Section 4.1 appendix (hyperparameters Section[]] data processing details Section full prompt Section D.. and supplementary materials (anonymous source code). We ensure data splits, random seeds, and environment configurations are explicitly stated, allowing researchers to independently verify our findings under identical conditions.
REFERENCES
Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang, Xingshan Zeng, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Yuefeng Huang, et al. Acebench: Who wins the match point in tool learning? arXiv e-prints, pp. arXiv-2501, 2025.
Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen. Program thoughts prompt- ing: Disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint arXiv:2211.12588, 2022.
DeepSeck-AL. Deepseek-rl: Incentivizing reasoning capability Ilms via reinforcement learning.
2025a. URL http:pss////....ooo////////////s 12948
DeepSeek-AL. Deepseck-rl: Incentivizing reasoning c in 1lms via reinforcement learning,
2025b. URL htltttt///axxiv.oot http://wwww........1....1.....
Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Firdaus Janoos, Larry Rudolph, and Aleksander Madry. Implementation matters deep policy gradients: A case study on ppo and trpo. arXiv preprini arXiv:2005.112799 2020.
Gongfan Fang, Xinyin Ma, and Xinchao Wang. arXiv:2505.13:: 133799 2025.
Thinkless: Llm learns when to think. arXiv preprint
Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. Retool: Reinforcement learning for strategic tool use in llms. arXiv preprint a50iv:2504.1 /136,, 2025.
10

---

Published as a conference paper at ICLR 2026
Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, Siyuan Li, Liang Zeng, Tianwen Wei, Cheng Cheng, Bo An. Yang Liu, and Yahui Zhou. Skywork open reasoner 1 technical report. arXiv preprint arXiv:2505.222: 2025.
Shijue Huang. Hongru Wang, Wanjun Zhong, Zhaochen Su, Jiazhan Feng, Bowen Cao, and Yi R Fung. Adactrl: Towards adaptive and controllable reasoning via difficulty-awarr budgeting. arXiv preprint arXiv:2505. 18822, 2025.
Hugging Face. Open rl: A fully open reproduction of deepseek-rl, January 2025. cog.uuub.coo//ugggiggo.occcsee
URL https:
Teknium interstellaminja.
Hennes-ffuntiinnnalllgggdaaaeee. 2024. ling--v
URL
https:/
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training Ilms to reason and leverage search engines with reinforcement learning, arXiv preprint arXiv:2503.095 2025.
Kimi K2, Kimi-rescarcchr: End-to-end rl training for emerging agentic capabilities. https:
/ /moonshotai.. wwwpaaai.ggttubiiioo Kimi Researcher/ 2024. Accessed: 2025-09-10. 10.
o
Angeliki Lazaridou, Elena Gribovskaya, Wojeiech Stokowiec, and Nikolai Grigorev. Internet- augmented language models through few-shot prompting for open-domain question answering. arXiv preprint wwX.:x2223.051111 2022.
Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. Api-bank: A comprehensive benchmark for tool-augmented Ilms. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp- 3102-31 16, 2023.
Xuefeng Li, Haoyang Zou, and Pengfei Liu. Limr: Less is more for rl scaling. arXiv preprint arXiv:2502. 1886, 2025a.
Xuefeng Li, Haoyang Zou, and Pengfei Liu. 2rX1v:2503.23383, 2025b.
Torl:
Scaling tool inte-interrted rl. arXiv preprint
Qigiang Lin, Muning Wen, Qiuying Peng. Guanyu Nie. Junwei Liao, Jun Wang. Xiaoyun Mo, Jiamu Zhou, Cheng Cheng, Yin Zhao, et al. Hammer: Robust amettin-calling for on-device language models via function masking, arXiv preprint arXiv:2410.04588) 2024.
Weiwen Liu, Xu Huang, Xingshan Zeng, Xinlong Hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, et al. Toolace: Winning the points of Ilm function calling. arXiv preprint arXiv:2409.0008 2024.
Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. Cot-valve: Length- compressible chain-of-thought tuning. arXiv preprint arXiv:2502.09601, 2025.
Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. CoRR, 2025.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint arXiv:2501.19393 2025.
Reiichiro Nakano, Jacob Hilton, Suchir Balaji. Jeff Wu, Long Ouyang, Christina Kim. Christo- pher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint 21X00:21 2021.
/ wwweeeann..cmmmiimeeeeeee ott.p//eeeeeeeaaaccoooo////aotte
2025.
Accessed: Sep 10, 2025.

---

Published as a conference paper at ICLR 2026
Jie Ouyang, Ruiran Yan, Yucong Luo, Mingyue Cheng, Qi Liu, Zirui Liu, Shuo Yu, and Daoyu Wang. Training powerful Ilm agents with end-to-end reinforcement learning, 2025. bttps::
// com/ OrwsswestO/Agennttt..
Zhenyu Pan and Han Liu. Metaspatial: Reinforcing 3d spatial reasoning in vims for the metaverse. arXiv preprint arXiv:2503. 1000./8470 /8470, 2025.
Akshara Prabhakar, Zuxin Liu, Ming Zhu, Jianguo Zhang, Tulika Awalgaonkar, Shiyu Wang. Zhiwei Liu, Haolin Chen, Thai Hoang, et al. Apigen-mt: Agentic pipeline for multi-turn data generation via simulated agent-human interplay. arXiv preprint arXiv:2504.0360 03600,, 2025.
Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkami- -ürk, Gokhan Tur, and Heng Ji. Toolrl: Reward is all tool learning needs. arXiv preprint arXiv:2504.1399 2025.
Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu,, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toollim: Facilitating large language models to master 16000+ real-world apis. In The Twelfth International Conference on Learning Representations, 2023.
Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji- Rong Wen. Tool learning with large language models: A survey. Frontiers of Computer Science, 19(8):198343, 2025.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 2024.
Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith, Stephen Roller, Megan Ung, Moya Chen, Kushal Arora, Joshua Lane, et al. Blenderbot 3: deployed conversational agent that continually learns to responsibly engage. arXiv preprint arXiv:2208.03/180 2022.
Linxin Song, Jiale Liu, Jicyu Zhang, Shaokun Zhang, Ao Luo, Shijian Wang, Qingyun Wu, and
Chi Wang. Adaptive in-conversation team building for language model agents. arXiv preprint arXiv:240S.19425, 2024.
Qwen Team. Qwen3 technical report, 2025a. URL httpss/ http://axxvvvv. wttp://...ggaab/225..00
Qwen Team. Qwq Embracing the power of reinforcement learning, March 2025b. URL
Jiaqi Wang, Kevin Qinghong Lin, James Cheng, and Mike Zheng Shou. Think or not? selective rea- soning via reinforcement learning for vision-language models. arXiv preprint arXiv:2505. 16854, 2025.
Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Exe- cutable code actions elicit better Ilm agents. Forty-first International Conference on Machine Learning. 2024.
Shijie Xia, Yiwei Qin, Xuefeng Li, Yan Ma, Run-Ze Fan, Steffi Chern, Haoyang Zou, Fan Zhou, Xiangkun Hu, Jiahe Jin, al. Generative ai act ii: Test time scaling drives cogmition engineering. arXiv preprint arXiv:2504.13828 2025.
Fanjia Yan, Huanzhi Mao, Charlie Cheng-Jie Ji, Tianjun Zhang, Shishir G. Patil, Ion Stoica, and Joseph Gonzalez. Berkeley function calling leaderboard. 2024.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503. 14476, 2025.
Yuanqing Yu, Zhefan Wang, Weizhi Ma, Zhicheng Guo, Jingtao Zhan, Shuai Wang, Chuhan Wu, Zhiqiang Guo, and Min Zhang. Steptool: A step-grained reinforcement learning framework for tool learning in llms. arXiv preprint 1rXiv:2410.07745, 2024.
12

---

Published as a conference paper at ICLR 2026
Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl- zoo: Investigating and taming zero reinforcement learning for open base models in the wild. arXiv preprint arXiv:2503.18892 2025a.
Weihao Zeng, Yuzhen Huang, Wei Liu, Keqing Hc, Qian Liu, Zejum Ma, and Junxian He. 7b model and 8k examples: Emerging reasoning with reinforcement learning is both effective and efficient. www.ioniiooooom wit. simplerriirree reason 2025b. Notion Blog.
https: httpass//kkuttnine wet.ttionnnnooiionnoooom
Yirong Zeng, Xiao Ding, Yuxian Wang, Weiwen Liu, Wu Ning, Yutai Hou, Xu Huang, Bing Qin, and Ting Liu. Boosting tool use large language models via iterative reinforced fine-tuning. arXiv preprini (rXXx:2501009600 2025c.
Jiajie Zhang, Nianyi Lin, Lei Hou, Ling Feng, and Juanzi Li. Adaptthink: Reasoning models can learn when to think. arXiv preprint arXiv:2505. 1..5005.1417,, 13417, 2025a.
Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Hoang, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, Zhiwei Liu, Yihao Feng, Tulika Awalgaonkar, Rithesh Murthy, Eric Hu, Zeyuan Chen, Ran Xu, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Silvio Savarese, and Caiming Xiong. xlam: A family of large action models empower agent systems. arXiv preprint arXiv:2409.0322 2024a
Shaokun Zhang, Jieyu Zhang. Dujian Ding, Mirian Hipolito Garcia, Ankur Mallick, Daniel Madri- gal, Menglin Xia, Victor Rühle, Qingyun Wu, and Chi Wang. Ecoact: Economic agent determines when to register what action. arXiv preprint arXiv:2411.01646 2024b.
Shaokun Zhang, Yi Dong, Jieyu Zhang, Jan Kautz, Bryan Catanzaro, Andrew Tao, Qingyun Wu, Zhiding Yu, and Guilin Liu. Nemootron-reseaarch-oooo---- Tool-using language models with rein- forced reasoning. arXiv preprint 2505: 000244, 2025b.
Wang Zihan, Wang Kangrui, Wang Qineng, Zhang Pingyue, Li Linjie, Yang Zhengyuan, Jin Xing, Yu Kefan, Minh Nhat Nguyen, Liu Licheng, Eli Gottlieb, Yiping Lu, Cho Kyunghyun, Wu Jiajun, Fei-Fei Li, Wang Lijuan, Choi Yejin, and Li Manling. Ragen: Understanding self-evolution in IIm agents via multi-turn reinforcement learning, 2025. URL. 2504 20073
USE OF LLM
LLMs (GPT-4o) were only used as general-purpor tools to draft baseline literature summaries and proofread minor grammar, LLM contributed ideation, algorithm development, analysis, or writing, and all M-sM-sisitted content was verified for accuracy/integrity. No L.M is eligible for authorship.
A RELATED WORK
A.1 AGENTIC TOOL-USE
Enhancing LLMs with external tools has emerged as pivotal direction for addressing complex tasks
in open domains (Qu et al. (2025 Wang et al.) (2024). Typical applications include integrating LLMs
with search engines (Zhang et al. 2024b) Lazariou et al. 2022) Shuster et al.)
2022). calculators
Nakaano et al. 2021), ) and Python interpreters (Wang et al. [204] [oog al.
2024 Chen et al,
2022). Three common paradigms are widely adopted for training tool-use LLMs: (1) SFT: imitates
the reasoning patterns from labeled high-quality examples, enabling models to learn standard tool-
use workflows (Liu et al. 2024 Zhang al. 2024a| Qin et al. 2023 Prabhakar et al.) 2025). (2)
RL with direct preference optimization: aligns model tool
tool-use use behavior with human intentions by
optimizing against human preference signals (Zeng et al.] 2025c: Yu et al. 2024). (3) RL with
Verifiable Rewards (RLVR): as a novel approach, leverages scalable test-time inference and utilizes
verifiable signals rewards to refine the model's 's tool-use decisions (Li al.
2025b).
13

---

Published as a conference paper at ICLR 2026
A.2 RL SCALE-UP
Reinforcement learning (RL) has gained traction as a more scalable and generalizable training paradigm. Models like R1-Zero leverage group relative policy optimization (GRPO) (Shao al. 2024) to unlock the model's reasoning capabilities at test time (DeepSeek-A] 2025a Yu et al.
2025 This RI-style reasoning paradigm, marking a shift from train-time scaling to test-time scal-
ing Muennighoff et al., 2025 Xia et al. 2025), has demonstrated success in mathematics (Shao
al.
2024). coding (Pan & Liu 2025), and agentic tool use (Feng et 2025 Jin al. 2025).
Recently, several works have explored automatic scaling, i.e., enabling models to adaptively select the optimal reasoning mode based on problem difficulty (Fang et al. 2025| Zhang et al.] 2025a] Huang et al. 2025 Wang et al., 2025). In agentic tool-use tasks, auto-scaling is particularly critical: many such problems can be solved with short reasoning, whereas excessively long reasoning leads to unnecessary resource consumption. While RL-based scaling for tool use in open open-domain rea- soning has been investigated (Zhang et al. 2025b unexplored in agentic tool use.
Qian et al. 2025). RL with auto-scaling remains
B DETAILS IN DATA PREPARATION
Source of Training Data Details. The raw data was sourced as follows:
ToolACE (Liu et al.
2024): general tool-use dataset teaching models when invoke
tools vs. respond directly, enhancing multi-step decision-making.
xLAM (Zhang et al. 2024a;
Prabhakar et al. 2025): A compositional dataset requiring one
more tool calls per turn We mixed the original 60k xLAM with its multi-turn variant APIGen-MT-5k (Prabhakar et al. 2025).
Hermes Function-Calling (interstellarnijja) 2024): Designed to train LLMs in function calls and structured output from natural language. We extracted function call-related sam- ples.
The dataset features various conversational scenarios where AI agents are required to interpret queries and execute appropriate single or multiple function calls. In Section 2 data distillation em- ploys Deepseek-R1-0522 (DeepSeek-A) Al (025a). Subsequently, in Section $3.1 to mitigate model bias by aligning with a no-think model, data distillation is carried out using Qwen3-32B (Tam 2025a).
Data Processing Pipeline & Distribution Details. We obtained PubTooo from raw data through following data processing workflow: (I) We randomly downsampled xLAM balance the sample sizes across the three datasets. (2) We removed overly simple and excessively difficult samples; Figure 7 shows the raw-data distribution of successful reasoning counts (pass@8) 8). Guided by this distribution, we partitioned the data into hard (31.8%), medium (21.2%), and easy (47%) subsets (Figure (7b) and re-balanced the difficulty distribution accordingly. (3) For RL data, we further refined the set by prioritizing samples that align closely with the model's 's current learning trajectory (details in the next paragraph). For comparison, we visualized the PubTool distribution in the same way (Figure 7p). We observed that the original corpus is concentrated in the easy and hard extremes, whereas PubTool peaks in the hard subset and is sparse in the easy subset. We argue that training on data of moderately high difficulty better elicits the model's test-time scaling capability (He al. 2025).
RL Data Refine Details. In the second phase of data processing, prioritize train- ing samples by their alignment with model learning trajectories, measured through the variance of reward scores from the mean, lower variance indicates better alignment. Better alignment corresponds to lower variance of reward scores, defined as:
Var(r) =
Σ (r. (r--4,7 1=1
1,,
14, Σ 1=1
where lower Var(r) indicates better alignment. This sampling result is illustrated in Figure From the figure, we observe that the average reward ranges between 0.7 and 0.9, with aligned samples showing higher scores in the upper-left region and misaligned samples displaying lower scores. To
14

---

Published as a conference paper at ICLR 2026
50
50
465
40
Hord 37.85
Medium 21
Eory
Eory
60
Percentage
Percont
20
10
10
71
22
23
2
Number Cornect Infonenons(e/88)
Number of Cornect Enfermoed(e/1))
Data Distribution of Raw data
(b) Data Distribution of PubTool
Figure 7: The number of correct inferences distribution with performing 8 rounds of reasoning on the training data (a). The distribution of PubTool after data processing (b).
align the dataset size with that of SFT (&K samples), we rank examples by reward variance and filter out the bottom 55%, a simple yet effective heuristic to retain higher-signal instances while controlling scale.
Mi Samplex Average Reward
0.8
Roward
0.8
0.2
Algrwd Sumple
0.2
0
10
2
6
(a) Retained: with aligned samples
(b) Removed: with misaligned samples
Figure 8: We retain aligned samples (i.e., those with low variance (a) ) and remove misaligned
samples (i.e., those with high variance (b)).
Effectiveness of Data Refinement. We experimentally verified its effectiveness. After warm-up SFT, Figure shows GRPO training processes with and without data refinement. Results indicate data refinement increases accuracy reward score by +15%, reduces training fluctuation variance, and enhances stability. Additionally, the model's thinking rate converged to lower level, indicating improved memory capacity. BFCL, evaluation results show GRPO with data refinement reached 66.82% versus 60. 78% without, an improvement of +6.04%. These enhancements are attributed to data refinement filtering substantial noise while retaining high-contribution samples.
(a) Accuracy Reward by Iteration Step
(b) Think Rate by Iteration Step
0.35
w/o data refine
w/o data refine
data refine
0.30
w data refine
0.9
accuracy score
0.25
0.8
0.20
(M/ Ma wadyy Wymw
0.7
0.10
0.6
0.05
0.00
0.5
Iteration Step
Iteration
Figure 9: RL training processes (with and without data refinement) shown, along with accuracy scores and thinking rates.
15

---

Published as a conference paper at ICLR 2026
C VISUALIZATION OF TRAINING DYNAMICS
To demonstrate auto-scaling effects, visualized the training process (Figure T0 As training progressed, accuracy improved while the thinking rate gradually decreased to 5%, indicating fewer problems required long reasoning, suggesting enhanced intrinsic tool-using capabilities. Addition- ally, response length and entropy achieved decoupled control: think mode enabled 500% longer rea- soning trajectories than no-think mode, with corresponding higher actor entropy reflecting greater exploration tendency. These visualizations confirm that training enhanced tool-using abilities and successfully enabled auto test-time scaling based on problem difficulty and model proficiency.
Accuracy
Thinking Rate
1.0
0,30
hogal/ou Ap M Min
Accuracy Reword
0.9
0.25
Thinking Rate
0.20
0.8
0.15
0.7
0.10
0.05
0.6
100
150
Entropy Thinking Thinking
Response Length Thinking Thinking
mode
600
1.0
Thaik Ertrapy
Think
Response Length
500
wwprr M
0.8
Entropy
400
0.6
0.4
200
mode
0.2
Thinking Length
100
Non-Think Thinkiiinkng www.ghh..oooorrrtt.....oo
100
150
200
50
100
150
200
Iteration Step
Iteration Step
Figure 10: The visualization of training dynamics.
D COMPLEMENTARY EXPERIMENTS
D.1 MORE IMPLEMENTATION DETAILS
The experiments were executed using the publicly accessible training framework MindSpeed-RI +] an end-to-end reinforcement learning acceleration framework based on the Ascend ecosystem. The BFCL is an evolving benchmark and utilized the version checked out on June 14, 2025. For the training model, we selected the best performance checkpoint on the valid dataset. In the Test- time Scale paradigms analysis ($2). we used instruct model for SFT and base model for GRPO. We employ a full-parameter training strategy for all SFT. In baseline trained on PubTool data, we trained on the complete dataset using specific methods (e.g.. SFT, Distilled SFT, and GRPO). In the PubTool-GRPO training, we adopted the widely used think prompt pattern, which fol- lows the format: <think> reasoning process here </think> <answa swer> answer here </answer>, Each RL training run for the 7B model completed within 4 hours on cluster of Ascend 910b NPUs (configured as nodes 8 NPUs). The hyperparameters used are detailed in 5
Baselines (1) Base Model: the original model without additional training (e.g., Qwenn.5-series. LLaMA3.1-serien) (2) SFT-trained Model: ToolACE-SB (trained on the full ToolACE dataset (Liu
16

---

Published as a conference paper at ICLR 2026
et al. 2024)).
XLAM-ssries (trained on the full xLAM dataset Zhang et al. )024al). and Hammer- series (trained on xLAM with function masking (Lin et al. (024)). (3) API-based closed-souree
frontier models (e.g.. GPT-series, Gemini-series). (4) RLVR-trained Model: models trained using GRPO as the RI. paradigm, such as QwQ-32B (Team 2025b). Qwen3-series (Team] 2025a), Tool- N1 series (single-turn tool-use models trained mixed ToolACE and xLAM data (Zhang al. 2025b)), and (triRet in subset of mixed ToolACE and xLAM data (Qian et al.) [025 Given that we are the first to investigate adaptive reasoning in agentic tool use, there is significant lack of prior works on this domain, we compare our method with recent adaptive reasoning strategies
adapted from other domains (e.g.. math). We select two representative methods for comparison:
(1) Thinkless (Fang et al., 2025): Achieves adaptive reasoning via control token loss. (2) Adactrl
(Huang et al. (0025): The model self-assesses problem difficulty during the RLVR rollout to facilitate
adaptive reasoning. We replicated both methods using the same dataset (PubTool) and the same base
model (Qwen2.5-7-IIsttruct.
Hyperparameter
Value
Hyperparameter
Value
Data Configuration
RL Optimization
Global Batch Size
128
Learning Rate
le-6 constant 128
Max Prompt Length
12000
LR Decay Style
Max Response Length
2048
Mini Batch Size
Micro Batch Size
KI. Loss Used
False
Train Steps
200
Rollout Configuration
Entropy Constraints
Rollout Name
vllm
Clip Higher é
0.28
GPU Memory Utilization
Think Target Entropy H;
Number of Rollouts
8
No-Think Target Entropy H,
0.1
Temperature
1.0
Init Adaptive Coefficient St
0.1
Tensor Model Parallel Size
Fixed Coefficient 8,
0.1
Top_P P
1.0
Table 5: The configurations for RL training with GRPO.
D.2 HYPERPARAMETER SENSITIVITY ANALYSIS
In this section, we conduct a hyperparameter sensitivity analysis to explore the effects of key param- eters, including the target entropies (H, and H₂) and the entropy penalties (8, and initial 8r). We evaluated the performance on the BFCL benchmark, reporting the Overall Accuracy.
Sensitivity to Target Entropy Table d presents the sensitivity analysis with respect to the target entropy values. The results indicate that model performance is sensitive to the specific target entropy settings. The optimal performance is achieved when the targets are set around H, 0.1 and H; = 0.2.
H,
H
0.1
0.2
0.5
0.1
69.7
70.1
68.5
0.2
67.7
66.3
0.5
62.1
Table 6: Sensitivity analysis in target entropy on the BFCL benchmark. Values represent Overall Accuracy.
Sensitivity to Entropy Penalty Table 7 demonstrates demeo the sensitivity analysis regarding the entropy penalty coefficients. The model exhibits considerable stability with respect to 3,, while lower values of B, generally yield better performance.
17

---

Published as a conference paper at ICLR 2026
3
e-1
e-2
e-3
0
70.7
68.4 4
71.4
70.6
e-1
68.8
70.1
69.5
68.7
e-2
69.3
e-3
70.44
Table 7: Sensitivity analysis in entropy penalty on the BFCL. benchmark. Values represent Overall
Accuracy.
D.3 GENERALIZATION ACROSS MODEL ARCHITECTURES AND SIZES
To verify the robustness and generalizability of our proposed method, AutoTool, we conducted ex- tensive evaluations across diverse model architectures and parameter scales. Specifically, we utilized the Llama 3.2-inst and Qwen 2.5-inst series, covering wide range of sizes from 1.5B to 32B pa- rameters.
Figure 11 illustrates the Overall performance on BFCL for both the base models and our method. The visual comparison demonstrates that AutoTTool consistently improves performance across all tested configurations.
Base Model w/ Auto Tool
15.1
16.4
70.12
Overall Performance (%)
12.8
59.67
6.8
53.73
8.5
65.20
53.69
60
46.92
46.71
47.09
40
30
Llama-3B
Qwen-1,5B
Qwen-3B
Qwen-7B
Qwen-328
Figure 11: Performance comparison across different model architectures and sizes BFCL bench- mark.
Key Observations:
Substantial Gains in Qwen Series: The Qwen models exhibited the most significant im- provements. As shown in the figure, models in the 3B 32B range achieved accuracy increases of over 12 points. For instance, the Qwen2.5-7B model saw remarkable lift from 53.69 70.12.
Effectiveness on Compact Models: Even the smallest tested model (1.5B) demonstrated a substantial performance lift (approx. 8.5 points), confirming the method's efficacy for resource-constrained scenarios.
Consistency: The Llama 3.2 architecture also benefited from our approach, with the 3B model improving from 46.92 to 53.73.
These findings confirm the generalizability and effectiveness of AutoTool across diverse model land- scapes, validating that the adaptive mechanism not limited specific architecture.
18

---

Published as a conference paper at ICLR 2026
0.9
0
03.49
$3,37
Overal Perfor Performnne
0.7
62.5
60.22
0.6
60.0
57.5
modk_ongg mide equore
96.0
0.4
100
150
200
Steration Step
Argle
Made
Mode Square
Accuracy Reward Iteration Step
(b) Performance Comparison of Four Prompt De-
sign
Figure 12: Visualization of training processes and evaluation results for four prompt designs.
E PROMPT DESIGN FOR AUTO THINK
To explore suitable prompt design for Auto Think, conducted a preliminary analysis of the four kind of prompts listed below:
Controlled reasoning mode with square tags: [onde]nno_tink//mmode [no_think] \n [hon_h [tool_call]] tool calls here [/tool_cat call]
Uncontrolled reasoning mode with square tags: [no thinkjlnl/no....... think] [cool_cal]] tool calls here //tool [coo]_cal]]
Controlled reasoning mode with angle tags: <mode>no__thnkk</mede think</mode> eno_thinks In </no_think> think> <tool _all_ tool calls here </tool_call>
Uncontrolled reasoning mode with angle tags: <no wwwinhlll/oooothhiiiii think>> <tool_call> tool calls here </tool_call
We trained the model starting from Qwen2.5-7B-he --Intructt using the original GRPO algorithm with PubTool LRL data. Their training processes and evaluation results are presented in Figure [2 From the results, two key observations emerge: (1) Square tags (()) exhibit better adaptability than gle tags (<>). This may be because the model used angle tags for segmentation in the pre-training phase, reusing these tags for different purpose (reasoning mode control) is likely cause signal interference. (2) Additionally, the explicit "reasoning mode" prefix does not obviously affect per- formance. The evaluation results show that the controlled reasoning mode with square tags achieves the best performance; thus, we adopt this prompt design for auto scaling.
F LIMITATION AND FUTURE WORK
Despite promising results in tool-use scenarios, our method has latent concerns to clear. First, we only tested it on specific model size, future work should verify its scalability across different model scales (e.g., 3B, 13B, 32B parameters) and architecture series, e.g., LLaMA-series.. Second, our method's generalizability beyond tool-use tasks unproven. It is valuable evaluate its perfor- mance other complex reasoning tasks (e.g., mathematics, logical deduction) to confirm if it can similarly enhance reasoning steps or reduce computational costs. Third, our method currently relies specific RL algorithm. Future research should test its compatibility with other RL algorithms (c.g.. PPO (Schulman et al., 2017 Engstrom et al. 2020) and DAPO (Yu et al. (025)) to verify the decoupled entropy constraint strategy is effective across different algorithmic paradigms. We will address these limitations in future work.
19

---

Published as a conference paper at ICLR 2026
The Full System Prompt for Automatic Think RL
You advanced function composition agent. Your goal is solve queries efficiently. In the multi-turn dialogue loop: the interaction cycle: You [tool call], you receive [tool response],, and you MUST use new information plan your next step.
# Tools
You provided with function signatures within [tools] and [/tools] tags: [tools] [functions] [/tools]
Action Phase
1. Choose Action Mode: For every turn, you MUST start your response by choosing action mode (think no think) based on the task's complexity.
think: Use complex reasoning. Enclose your detailed thought process within [think] and
[/tinn]] tags.
no_think: Use for simple, straightforward tasks. You MUST use an empty, self-closing
block: [no_thinke /in/o// _hink]].
Your response MUST begin by enclosing selected mode name within [mode] and [/mode] tags.
2. Decide on Action Path: After that, you must choose ONE of the following two paths:
## Path A: Call Functions
WHEN: The user's intent is tool-related and you have all required functions and parameters. The tool_calls field is JSON object with function names and arguments within [tool_call] and [/tool_cal] XML tags. i.c., [tool_ col__all]] [("name": <function-namer "arguments": <args-json- object>], ("name": <tunction-name2>, "arguments": jangs--son-objeet235 ...I [tool_call]
EXAMPLE:
[mode]no think[/mode] [no th/nn]nn//ooooo_o [think] [tool_cale tool calls here [/tool_ed _alol__al]]
EXAMPLE: [mode/eiink/mmde]] [think] reasoning process here [/think] [tool_call] tool calls here [/too[_cal]]
## Path B: Respond Directly to the User
WHEN: You need to provide natural language text response. This happens in three main
scenarios:
(1) After receiving tool execution feedback enclosed within [tool_ respoose] and
tags, continue to respond to user queries based on this feedback.
(2) The user's 's query is general conversation and not related to any tool.
(3) Ask for more information if the given conversational context lacks the required functions parameters.
EXAMPLE:
[mode]no think[/mode] [no_ boo_thinkJJn[/nn [hink] natural language sentences you talk with user
EXAMPLE:
[mode]think[/moot) [think] reasoning process here [/think] natural language sentences you talk with user
Figure 13: System Prompt Design for Automatic Scaling Tool-Use in Multi-Turn Dialogue.
20


Reviews:
[{"review_id": "QckbyRBVDi", "reviewer": "Reviewer_dVK5", "summary": "This paper proposes a novel reinforcement learning training paradigm to address two key challenges in tool-based AI agents: (1) direct RL training struggles to scale up thinking length sufficiently to solve complex problems, and (2) scaled-up models tend to overthink simpler problems, resulting in substantial token inefficiency.\n\nMain contributions include:\n1. Proposed a decoupled adaptive entropy constraint strategy, which separates entropy constraints for short and long reasoning paths, enabling the model to automatically adjust its reasoning scale based on problem difficulty.\n2. Designed an automatic test-time scaling reward module, which uses an asymmetric reward mechanism to encourage short reasoning for simple problems and long reasoning for complex ones.\n3. Constructed the PubTool dataset and optimized RL training effectiveness through a data quality refinement strategy.\n4. Validated the method's effectiveness on three benchmarks, achieving a significant 9.8% accuracy improvement while reducing computational overhead by approximately 81%.", "questions": "- The legend for Figure 2 could be added to each of the three subplots or provided as a common legend separately.\n- What is the specific theoretical rationale behind the asymmetric reward in Formula (5) (only +0.5 for correct answers in 'think' mode versus +1.0 in 'no-think' mode)? Could this design potentially bias the model towards preferring the 'no-think' mode?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The decoupled entropy constraint mechanism is novel. By applying different entropy constraint strengths to short and long reasoning paths, it effectively solves the reasoning collapse problem prevalent in traditional RL training.\n- The paper is well-structured, with logical coherence from problem analysis and method design to experimental validation. The figures are well-designed, particularly the training dynamics visualization, which effectively demonstrates the auto-scaling effects.\n- The experimental design is comprehensive, featuring systematic evaluation on three benchmarks: BFCL, API-Bank, and ACEBench.", "weaknesses": "- Entropy constraint hyperparameter selection: Although an adaptive mechanism is proposed, the initial choices for H_l and B_s lack sufficient theoretical justification or ablation analysis.\n- While sample filtering based on reward variance is mentioned, the specific threshold settings and selection criteria are not described in detail.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "j8xJWyCLio", "reviewer": "Reviewer_URMW", "summary": "This paper aims to improve Tool-use LLMs' performance through the control of thinking length. It proposes applying distinct entropy constraints to long and short trajectories during RL.", "questions": "See above.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 1, "contribution": 1, "strengths": "* This paper proposes a new loss function designed to apply an entropy penalty that is conditional on the trajectory length within the RL process.", "weaknesses": "* The paper suffers from a critically confusing narrative and fundamental conceptual ambiguity. The core concept of TTS introduced in line 40 is unsupported in the literature and largely diverges from the definitions provided in the very papers cited. This narrative choice, which mixes TTS terminology with RL problem, is highly confusing and obscures the paper's actual contribution.\n* For the RL component, the idea of activating or deactivating thinking via special tokens is a well-established and standard paradigm. The paper's contribution is reduced to proposing a new training loss within this existing framework, making the technical novelty incremental. The authors fail to properly contextualize their work within the relevant RL literature, which is a major omission.\n* The experiments also lack rigor.\n  * The paper largely ignores previous work on critical related issues, such as thinking length and entropy constraints in RLVR.\n  * The evaluation is demonstrably unfair due to a vast disparity in the inference budget. Figure 1-c explicitly shows the proposed method uses an inference budget greater than 1k, while the baselines are restricted to less than 100. This makes the reported performance gains unconvincing, especially since many *actual* TTS methods can achieve similar gains without requiring any additional RL training.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0NfvQtKaba", "reviewer": "Reviewer_nwbB", "summary": "The paper proposes AutoTool, an RLVR-style training recipe for tool use that decouples entropy regularization between short (no-think) and long (think) trajectories and adapts the long-path entropy coefficient toward a target entropy. The pipeline warms up with SFT to expose both modes, then runs GRPO with: (i) a mode tag ([think]/[no_think]) and a format reward, (ii) an answer reward, and (iii) an entropy loss that activates only when entropy drops below a target (adaptive β for think, fixed β for no-think). On BFCL (non-live, live, multi-turn), API-Bank, and ACEBench, a ~7B model improves accuracy vs. SFT/GRPO baselines while cutting inference tokens, and the model auto-scales its \"think rate\".", "questions": "1. Sensitivity & ablations. Will you provide sweeps over target entropies for think/no-think and over β (fixed vs. adaptive), and include a single unified entropy-penalty baseline to isolate the benefit of decoupling?\n2. Format reliance. Can you evaluate without the [think]/[no_think] tags (no format reward), or on prompts that prohibit tags, to test format-agnostic generalization? \n3. Data & overlap checks. Would you release the curated training data (or a representative subset) and report an overlap analysis with evaluation sets, and add results where any dataset family used for training is excluded from the evaluation domains?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Clear motivation & mechanism. The paper links tool-use failures under RL to entropy collapse and proposes a targeted fix—decoupled, adaptive entropy—implemented with minimal changes to GRPO. \n2. Solid empirical suite. Evaluations span BFCL (non-live/live/multi-turn), API-Bank, and ACEBench, with consistent gains over SFT/distillation/RLVR-like baselines; ablations remove each component (data refine, decouple, adapt-coef) and show clear drops. \n3. Efficiency analysis. The paper reports token-cost reductions, defines ACU (Accuracy per Params×Tokens), and visualizes training dynamics (entropy, think-rate, lengths).", "weaknesses": "1. Data construction bias and potential leakage. The curated training set mixes downsampled public data with RL-refined, low-variance samples, which likely skews toward cleaner/easier cases; possible overlaps with evaluation sets are not audited. \n2. Hyperparameter sensitivity. In multi-turn settings, performance appears sensitive to target entropies and the choice of β; broader sweeps or learned schedules would strengthen the claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XrUbSWwR6b", "reviewer": "Reviewer_LhKY", "summary": "The paper introduced an adaptive entropy constraint during RL training phase to address the reasoning collapse issue in the hard tool-use task and overthink in the easy cases.\n\n* It provides an new entropy constraint to adaptively prompt long or short reasoning trajectories based on the task difficulty.\n* It proposes to use a dataset with emphasis on the difficult task to improve the model performance.\n* It provides a thorough analyze on the approach's performance against different approaches proposed to improve tool-use performance and an ablation study on the contribution from different parts of the proposed approach.", "questions": "* Line 075, what's the in-depth experiments.\n* Line 133, the citation is on the open-source code which does not provide useful information on the finding.\n* Table 2, why is the # of data in ToolACE and Hermes Function-Calling remain the same after downsampling.\n* Sec 4.3.2, if the ACU includes the accuracy and # of tokens, shouldn't the ACU score be the overall performance metric?\n* Line 752, it states `Results indicate data refinement increases accuracy reward score by +15%`, but the figure 9 shows the other way around. The blue line with data refinement does not have much accuracy score increment.\n* Line 911, PPO citation incorrect?\n* While the # of turns might correlate to the task difficulty, but it would be nice to have an analyze on the performance (accuracy, # of tokens) in different task difficulty.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Overall. the paper is well-written, it provides an clear illustration on the proposed approach. The experiments setup and results are presented clearly.\n* The proposed approach is easy to follow and implement. It provides detailed information on replicating the results.\n* The proposed approach improves the model performance on the multi-turn scenario significantly, proving the promoting long reasoning strategy is beneficial to solving complex multi-turn task.\n* The ablation study demonstrates the importance of all 3 parts of the proposed approach.", "weaknesses": "* While the author puts testing the approach in different architecture series in the future work, but I personally would prefer includes it in the experiments to prove the proposed methods works not just on the Qwen2.5-7B-Instruct.\n* Missing how $H_l = 0.2$ and $H_s = 0.1$ are defined, such as a experiment on the performance with different choices of $H_l$ and $H_s$.\n* The ablation experiment w/o data refinement does not describe what's the data used in the experiment.\n* Fig 4, the loss for short trajectories uses the long trajectory's target entropy.\n* It's unclear how are the short and long path/trajectory defined?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)