STAR: Similarity-guided Teacher-Assisted Refinement for Super-Tiny Function Calling Models
Authors: Jiliang Ni
Jiachen Pu
Zhongyi Yang
Jingfeng Luo
Conggang Hu
Venue: ICLR 2026 Poster (2026)
Keywords: Language Models
Knowledge Distillation
Reinforcement Learning
Primary area: foundation or frontier models, including LLMs

Abstract: The proliferation of Large Language Models (LLMs) in function calling is pivotal for creating advanced AI agents, yet their large scale hinders widespread adoption, necessitating transferring their capabilities into smaller ones. However, existing paradigms are often plagued by overfitting, training instability, ineffective binary rewards for multi-solution tasks, and the difficulty of synergizing techniques. We introduce STAR: Similarity-guided Teacher-Assisted Refinement, a novel holistic framework that effectively transfers LLMs' capabilities to super-tiny models. STAR consists of two core technical innovations: (1) Constrained Knowledge Distillation (CKD), a training objective that augments top-k forward KL divergence to suppress confidently incorrect predictions, ensuring training stability while preserving exploration capacity for downstream RL. STAR holistically synergizes these strategies within a cohesive training curriculum, enabling super-tiny models to achieve exceptional performance on complex function calling tasks; (2) Similarity-guided RL (Sim-RL), a RL mechanism that introduces a fine-grained, similarity-based reward. This provides a robust, continuous, and rich signal for better policy optimization by evaluating the similarity between generated outputs and the ground truth. Extensive experiments on challenging and renowned benchmarks demonstrate the effectiveness of our method. Our STAR models establish SOTA in their size classes, significantly outperforming baselines. Remarkably, our 0.6B STAR model achieves the best performance among all open models under 1B, surpassing even several well-known open models at a larger scale. STAR demonstrates a training framework that distills capabilities of LLMs into super-tiny models, paving the way for powerful, accessible, and efficient AI agents.

Published as a conference paper at ICLR 2026
STAR: SIMILARITY-GUIDE TEACHER-ASSISTED RE- FINEMENT FOR SUPER-TINY FUNCTION CALLING MODELS
Jiliang Ni', Jiachen Pu', Zhongyi Yang", Jingfeng Luo, Conggang Hu¹
Algorithm Platform Team, Al Hardware Division Alibaba
conggang. hogeallllllooo baba-iiaaeaa
ABSTRACT
The proliferation of Large Language Models (LLMs) in function calling is pivotal for creating advanced Al agents, yet their large scale hinders widespread adoption, necessitating transferring their capabilities into smaller ones. However, existing paradigms are often plagued by overfitting, training instability, ineffective binary rewards for multi-solution tasks, and the difficulty of synergizing techniques. We introduce STAR:
Refinement, novel holis- tic framework that effectively transfers LLMs' capabilities to super-tiny models. STAR consists of two core technical innovations: (1) Constrained Knowledge Dis- tillation (CKD), training objective that augments top-k forward KI. divergence to suppress confidently incorrect predictions, ensuring training stability while pre- serving exploration capacity for downstream RL. STAR holistically synergizes these strategies within a cohesive training curriculum, enabling super-tiny mod- complex function calling tasks; (2)
Teacher-Assisee
els to achieve exceptional performance mmnilarity-guided RL (Sim-RL), a RL mechanism that introduces a fine-grained, similarity-based reward. This provides robust, continuous, and rich signal for better policy optimization by evaluating the similarity between generated outputs and the ground truth. Extensive experiments on challenging and renowned bench- marks demonstrate the effectiveness of our method. Our STAR models establish SOTA in their size classes, significantly outperforming baselines. Remarkably, our 0.6B STAR model achieves the best performance among all open models un- der IB, surpassing even several well-known open models at a larger scale. STAR demonstrates a training framework that distills capabilities of LLMs into super- tiny models, paving the way for powerful, accessible, and efficient Al agents.
INTRODUCTION
Large Language Models (LLMs) have demonstrated remarkable capabilities as agents that inter- act with external tools and APIS via function calling (Patil et al. 20244: Jii et al. 2025). This has driven a new generation of applications, from automated personal assistants to complex data anal- ysis systems. However, the prohibitive computational cost of state-of oo-tth--rttttttt art models driving these advancements, often with tens to hundreds of billions of parameters, hinders their accessibility and practicality for on-device deployment and large-scale services (Guo et al., 2025). This necessitates transferring the capabilities of large models to smaller, more efficient models. However, the con- ventional strategy (DeepSeek-Al al. 2025 Cui et al. 2025a) achieve this, which involves sequence of Supervised Fine- MunnnTuning (SFT) and Reinforcement Learning (RL), proves inadequate for such super-tiny models. The inherently limited capacity of these models makes them prone to overfitting when trained with SFT on finite, high-quality datasets; they memorize specific tool-use patterns rather than generalize. Concurrently, applying RL directly to small models notoriously unstable and inefficient (Sarangi & Salam 2025
Dang & Ngo 2025)
"Equal Contribution.
1 "ooresponndig Author.

---

Published as a conference paper at ICLR 2026
These limitations suggest more promising approach: combining Knowledge Distillation (KD) to provide a robust, generalizable initialization for RL without the risk of overfitting. Yet, this KD+RL paradigm introduces its distinct and formidable challenges: (1) KD instability and constrained exploration: To manage computational costs, standard KD often employs top-k trun- cation, leaving the student's long-tail probability distribution unsupervised. This lack of guidance frequently leads to training instability and model collapse, while simultaneously stifling the ex- ploratory capacity essential for the subsequent RL phase: (2) Ineffective RL rewards: For multi- solution problems such as function calling, standard discrete or binary success/failure rewards can excessively penalize valid, alternative solutions, thereby impeding effective learning (Wei et al.) 2025;; (3) Synergistic integration challenges: Achieving true synergy between KD and RL, rather than interference, presents a significant practical hurdle.
This context motivates our work, aiming to create effective and stable training framework that overcomes these obstacles. We introduce STAR: Similarity guided Teacher-Assisted Refinement, holistic framework designed to meticulously transfer and refine LLMs' capabilitiess into super-tiny models. Our contributions are threefold:
We introduce Constrained Knowledge Distillation (CKD), a novel training objective that enhances top-k forward KL-divergence with targeted regularization term the student's probability distribution. This suppresses high-confidence but erroneous predictions without foreing the long-tail distribution to zero, ensuring stability under top-k truncation while preserving the crucial exploratory capacity for downstream RL.
We propose novel RL mechanism, Sim-RL, that augments the standard task reward with fine- -raiini similarity- reward. This reward computed from the similarity between generated outputs and the ground truth, providing robust, continuous, and rich signal to enhance policy optimization without increasing system complexity.
We present unified training curriculum that effectively synergizes the strengths of CKD and Sim-RL, culminating in STAR models that establish new SOTA on the challenging and renowned benchmarks for their own sizes. Notably, our 0.6B STAR model achieves relative gains of 9.2% on BFCL and over 50% on ACEBench against baselines. It outperforms all open-source models under IB and even several significantly larger models. The code link is htttp:://ggtthaae com/ /wwnn//wen-eee Applications STAR
The immense inference cost of highly capable large models mainly hinders their large-scale appli- cation, making a critical research goal elevate small models' performance to near-large-mode levels. Our work validates that well-designed training framework can transfer LLMs' capabilities into super-tiny models. This unlocks their potential in specialized fields, broadens the real-world deployment of advanced AI, and enables the creation of powerful, accessible, and efficient agents.
2 TASK DEFINITION: FUNCTION CALLING AS A GENERATION PROBLEM
We formalize the task of function calling conditional sequence generation problem. The model is provided with a context, which includes the user's query, a set of available functions F {f1, ..., fN } and other information. Each function f. is defined by its name, descrip- tion of its purpose, and its parameters.
The model's goal is to generate sequence of function calls P = (pi, (p..,P........ Pa) that solves the user's query. A function call a structured output, typically specific format like ISON, e.g., (" ["name" ""..", targumens" : :f" ("arg" : ""..." ....... Additionally, the model is also required to provide natural language responses when no function calls are needed.
THE STAR METHODOLOGY
The STAR methodology is a comprehensive training framework designed to imbue a super-tiny student model (Ms) with the advanced function calling capabilities of much larger teacher model (Mr). It consists of two core technical components CKD and Sim-RL RLaappleed within a carefully structured training curriculum, as illustrated in Figure
2

---

Published as a conference paper at ICLR 2026
CKD
Sim-RL
Teacher Model
Student Model
Final Model
Distillation
Refinement
Figure 1: The overview of the STAR training curriculum.
3.1 CONSTRAINED KNOWLEDGE DISTILLATION (CKI))
Knowledge distillation (KD) is cornerstone for aligning a student model (Ms) with a teacher (Mr). A central design choice in KD for language models is the divergence metric, typically oscillating between the distribution-covering Forward KL-divergence (LFKi) and the mode-seeking Reverse KL-divergence (CRKl) (Gu et al. 2024; Li et al., 2024). CrKL forces the student model (Ms) to focus the high-probability tokens of the teacher (Mr) while ignoring the vast, often uninformative tail of the distribution, defined as:
LFKL.
Σ
DKL. (P
RED
LRKI.
Da. (Pt(ss(s))))))P Σ RED
(2)
where Ps and Pr represent the output distributions over vocabulary for given context Some methods, like Adaptive Kullback-Leiber divergence (AKL), combine both (Wu et al.) 2025).
AKL
RL-from-ttebillied AKL RL-from-FKL
Format Reward
RKL
Entropy Loss
RL from Stabilized RKL
MMl Mi
Training Step
Training Step
(b)
Figure 2: FKL vs. RKL/AKL. We compare format rewards and entropy losses with different KL divergences during KD and RL training. Left: The RKL/AKL Ieads to catastrophic training collapse during KD. Right: The entropy losses with stablized-RKL/AKI are constantly smaller during RL.
3.1.1 INSTABILITY WITH TOP-K TRUNCATION
For computational efficiency, KD often performed using top-k truncation, where the loss is com- puted only the teacher's 's top-k tokens (Vx(x)). However, discover that combining this strategy with the mode-seeking RKL. (or its variant AKL) leads to catastrophic training collapse, as shown in Figure 2a Our analysis shows this is caused by the RKL. component, which imposes instable supervision on any token outside (x). destabilizing the optimization. In contrast, top-k FKL re- mains stable as simply ignores the tail distribution, imposing such constraint. theoretical justification for this instability is provided in Appendix A.3
3.1.2 THE HIDDEN COST OF RKI.
Beyond instability, we identify a more fundamental limitation of RKL-based methods: dimin- ished exploratory capacity. Even with stabilized variant of top-k RKL. and its variant AKI.

---

Published as a conference paper at ICLR 2026
observe that it consistently yields models that underperform simple top-k FKL baseline in downstream RL fine-tuning. We attribute this performance deficit to RKL's mode- seeking nature, which aggressively prunes the tail of the student's distribution. While this behavior promotes high-fidelity imitation, it critically reduces the student model's output entropy (Figure 2 thereby limiting its capacity for exploration prerequisite for successful reinforcement learning.
(Appendix A.4),
3.1.3 OUR APPROACH
These findings motivate our method, Constrained Knowledge Distillation (CKD). We start with the stable and asploration-friendee top-k FKL and introduce a targeted regularization term Lmi to control the most problematic part of the student's tail distribution. This term Cual applies an penalty only to tokens that the student considers probable (in its top-m set, Vn (x)) but the teacher deems irrelevant (outside its top-k set, (z(z))..
Our final CKD loss function combines the top-k FKL objective with this targeted tail penalty:
LCKD LFKLAX + Aust Crall
(3)
where:
Pr(v|z) Ps(v|x)
LFKL.X = Σ
Σ
Pr(vl=) log
(4)
RED (a))(a)
Luil Σ
Σ
Ps(vlæ)
(5)
RED WE (Z)
and Auail is a balancing hyperparameter. This approach directly suppresses the student from confi- dently predicting tokens that the teacher has dismissed. Moreover, according to the detailed gradient analysis (see Appendix A.5), this penalty encourages the redistribution of probability, which implic- itly regularizes the student's predictions within the top-k set and discourages over-confidence. also beneficial for downstream RI. as it retains the capacity for exploration.
3.2 SIMILARITY- REINFORCEMENT LEARNING (SIM-RL)
Reinforcement Learning with Verifiable Rewards (RLVR) shows significant promise in enhancing the reasoning capabilities of large language models (Lambert et al. 2025). Because the function calling task typically admits multiple valid solutions and meets the challenges of simulating realistic API feedback during training, the reward design often depends on process reward model (PRM) or abstract syntax tree (AST) parsing (Goldie et al. 2025). In this work, propose Sim-RL, method that generates reward signals through low-cost computation of similarity between model outputs and ground-truth responses. This approach enables fine-graa -raiggra similarity-based reward discrimination while effectively mitigating issues of over-rewarding or excessive penalization.
3.2.1 REWARD DESIGN
Format Reward. A prerequisite for successful function call is the generation of a response in the correct format. To enable parsing into a structured function call object, the model output must be constrained by strict format. We illustrate one implementation of format reward us- consid-
ing the Qwen tool calling template (see Appendix II.I) as an example. generation ered valid if it adheres the following rules: (1) The output must contain exactly one pair of <thi i nk>..... ........ </thi n>> tags, encapsulating the model's reasoning process; (2) If the model de- cides to invoke functions, each invocation must be wrapped in <tool_call>_ </ttoooooaall tags; (3) The content must be a single JSON object containing two keys: "name" and "arguments": (4) The value of the "name" " key must be present in the of available func- tions, F; (5) Furthermore, all keys within the "argument"" object must be subset of the keys defined for that specific function in F.
The format reward Rfomat is binary signal defined as:
{1
if all format rules are satisfied
Rioemat
0 otherwise
(6)
4

---

Published as a conference paper at ICLR 2026
Function Call Reward. Conditioned correct format (Riernat 1), we evaluate the accuracy of the tool invocations. Inspired by the Intersection over Union (loU) principle, the tool call reward compares the predicted sequence of tool calls P { Pm against the ground- -ruth sequence G = (9n.....9n It defined as:
Li=l spm (o(1))
min(
Ric
Pl+ |Png]
where is greedy matching scheme that establishes one-to-one correspondence between ele- ments of and G (see Algorithm 2 in Appendix A sim(p, g) is an argument-level similarity function between predicted call and ground-truth call g:
sim g) = Emekossp)nkessta) (PA, 9k)
(8)
kkey(p) keys(g))
The function 8(*,, 9t) computes the similarity for a specific argument key k. with its definition varying by data type: (1) String: ROUGE-L score (Lin 2004;; (2) Numeric/Boolean: An exact match (1 if equal, 0 otherwise); (3) Other types: An exact match after converting both values to
their string representations. Please refer to Algorithm 3 Appendix
Response Reward. In our task, the model may also generate natural language response directly without invoking any functions. For such text-only generations, the response reward is defined as the ROUGE-L F1 score between the predicted response p and the ground-truth response g:
Rresgonse ROUGE-L(P.9)
(9)
Total Reward. The total reward R is composite function that unifies these components:
R =
(Rrrmaa 1) + Rformat (Bre + Reesponse
(10)
term
answer term
This structure ensures that any format error (Btormat = 0) results in a strong penalty -1 from the format term. If and only if the format correct (Proemat 1), the reward transitions to the answer term, evaluating its correctness. The total reward is thus bounded the range [-1, 1]. guiding the model towards both correct answer formatting and content accuracy. For specific implementation details, please refer to the Appendix(A..
3.2.2 OPTIMIZATION METHOD
We employ GRPO Shao et al. 2024 as our RL algorithm. GRPO enhances stability by using G' rollouts for each prompt and computing the advantage Á via reward standardization across the group. The objective function is:
JaRvo (0) (lq.a) N
ledl
-B min (Ir,())A..,,iip((rrauu -0,1+))))) - BDa (Tol| (Ttee) )] (II)
where the advantage A for each token is derived from the standardized reward R, of its corre- sponding rollout o;:
Å moan (1 (
(12)
std( (R,) 1
Given that answer term reward is bounded in [0, 1]. group-wise mean reward of exactly 0 or implies that all rollouts in the group are either entirely incorrect or perfectly correct, respectively. In such cases, the advantage is zero for all samples, so these homogeneous groups contribute no gradient signal. Inspired by DAPO Yuu al. 2025). introduce filtering mechanism to discard these groups from each training batch. This simple yet effective strategy prevents wasted computation and accelerates the RI. training process.
5

---

Published as a conference paper at ICLR 2026
3.3 THE STAR TRAINING CURRICULUM
Our full training process consists of model distillation and model refinement, as shown in Figure
Effective distillation requires the selection of good teacher model. A capa- ble, instruction-turne model (c.g.. Qwen3-8B) can serve this purpose. Additionally, inspired by the teacher correction (Sreenivas et al., 2024). we employ the Sim-RL mechanism (see Section 3.2 to better adapt the teacher model to the distillation dataset. Next, we use our stable Constrained Knowl- edge Distillation (CKD) method to distill the refined teacher's 's knowledge into the student model. This step effectively transfers the teacher's core capabilities while preventing training instabilities.
Model Distillation:
Finally, we polish the distilled student's ' policy with final application of Sim- RL. This phase corrects minor distillation artifacts and directly optimizes the student's performance and reliability on the most difficult problems.
Model Refinement:
4 EXPERIMENTS
We conduct series of solid experiments validate the effectiveness of our STAR methodology.
4.1 EXPERIMENTAL SETUP
Models. We use the Qwen-family of models Yang et al. 2025). The teacher model, Mr, is Qwen3-8B fine-tuned with Sim-RL. The student models, M.. comprise Qwen3-0.6B, Qwen3-1.77B and Qwen3-4B, which are trained under the guidance of Mr. Details can be seen in Appendix A.2
Datasets. We construct our initial training set, D, by merging four datasets:
ToolACE (Liu et al.
2025):
11.3k instances of diverse tool usage patterns.
xLAM (Luu al. 2024): 60k high-quality, validated function calling samples.
XLAM-irreleyance (Lin et al. 2025): ) 6.7k filtered samples for irrelevant function detec-
tion, with answers synthesized using Qwen3-32B.
-ool-use-sytthetii]
50k sampled instances of multi-step and multi-turn interactions.
Data in D is formatted to the Qwen chat specification, with responses validated by format checker Riormat- The teacher MT then generates rollouts on to create an augmented dataset Dr, which includes the teacher's is reasoning and final answer. These trajectories are also filtered by Riormat to ensure structural correctness. Detailed prompt formats are available in Appendix A.I
Baselines. We compare our method, STAR, against several strong baselines:
Base-model: The pre-trained model without any fine-tuning.
SFT: Standard supervised fine-tuning on the dataset D.
SFT-think: SFT on the sugeee-- dataset Dr.
FKL. Training on Dy with a top-k (k=100) forward KI. divergence loss, guided by Mr-
ToolRI QQian et al. reward function.
2025 Training the SFT-think model with GRPO and a specialized
LUFFY (Yan et al. the Sim-RL reward.
2025
A hybrid offline-online approach using both D and Dr with
An online knowledge distillation method trained jointly with RL on D, using the Sim-RR RL reward and guidance from Mr.
GKD (Agarwal al.
2024
Benchmarks. We evaluate all models on two established benchmarks. See details of each evaluation category of benchmarks in Appendix A.9
httpa//cc....ooooo.otoooaaaaaaar use weteeessttttttctitceeeeeeee
p

---

Published as a conference paper at ICLR 2026
BFCL (Patil al. 2025) The de facto standard for function calling evaluation, assessing serial/parallel calls, multi-language support, and multi-step reasoning.
ACEBench (Chen et al. 2025): A new function calling benchmark that enforces a specific output format, challenging model's instruction-folloiing and generalization abilities.
4.2 MAIN RESULTS
Table 1: Performance comparison of different fine-tuning methods on Qwen3-0.6B, evaluated on the BFCLv3 benchmark.
Method
Overall Acc
Non-Live Ace
Live Acc
Multi Turn Acc
Standard methods
Base-model
47.33
71.81
65.66
1.88
SFT
44.58
66.29
62.15
1.62
SFT-think
47.59
71.54
64.46
4.50
FKL
49.51
76.44
65.93
5.12
Recent methods
ToolRL
47.35
64.81
66.55
6.75
LUFFY
49.23
76.75
64.59
5.48
GKD
47.32
67.62
67.61
3.25
Our methods
CKD
49.84
75.92
66.15
5.62
Sim-RL
49.35
75.21
67.39
3.25
SFT+Sim-RI." 50.41
76.27
66.99
6.13
CKD+Sim-RL
51.70
78.65
68.19
7.00
II refers to Sim-RL SFT-think.
The detailed statistics show in Table
Table 2: Performance comparison of different fine-tuning methods on Qwen3-0.6B, evaluated on the ACEBench Normal benchmark.
Method
Summary
Atom
Single- Turn
Multi-Turn
Similar API
Preference
Standard methods
Base-model
27.20
37.70
19.50
10.00
36.00
6.00
SFT
2.10
1.70
0.50
0.00
14.00
0.00
SFT-think
28.70
42.30
14.00
9.00
34.00
10.00
FKL
36.80
52.30
16.00
16.00
42.00
22.00
Recent methods
TooIRL
29.40
45.00
12.50
10.00
34.00
4.00
LUFFY
44.40
59.30
26.50
26.00
50.00
22.00
GKD
40.10
54.00
21.50
23.00
46.00
22.00
Our methods
CKD
39,00
55,00
21.00
19.00
48.00
10.00
Sim-RL
39.30
53.30
23.50
21.00
52.00
10.00
SFT+Sim-RL.
38.90
53.00
21.50
21.00
46.00
18,00
CKD+Sim-RL
53.00
69.30
35.00
32.00
62.00
20.00
"It refers to Sim-RL on SFT-think.
The detailed statistics show in Table

---

Published as a conference paper at ICLR 2026
Table 3: Model performance on function call- ing benchmarks across scales.
Table 4: Model performance on function calling
Model
BFCLv3 Overall
ACEBench Normal
benchmarks with different KD strategies.
Qwen3-8B
66.34
72.90
BFCLx3
ACEBench Normal
Llama3. 1-8B
49.57
46.60
Method
Overall
Toot-B
67.79
75.60
Hammer2.1-77
62.25
62.80
w/o RL
w/ RL
w/o RL
w/ RL
Teacher-8B
67.74
72.70
CE
47.59
50.41
28.70
38.90
Qwen3-4B
63.39
71.80
FKL
49.51
51.46
36.80
50.00
Llama3.2-30
45.86
29.60
RSKD
49.03
50.65
35.40
49.80
Hammer2. 1-3B
59.56
18.70
RKL.
49.26
50.49
35.30
41.30
STAR-4B
65.24
74.10
AKL
49.47
50,29
44.20
49.00
Qwen3-1.7B
54.70
51.60
CKD
49.56
51.70
39.00
53.00
STAR-1.7B
56.05
60.90
"The stable variant.
Qwen3-0.6B
47.33
27.20
STAR-0.6B 51.70
53.00
Overall Performance. As shown in Table and Table 2 our proposed STAR framework, which combines CKD and Sim-RL, establishes new SOTA for function calling on the 0.6B model scale. On BFCLv3 benchmark, STAR (CKD+)im-RI.) achieves an overall accuracy of 51.70, and on ACEBench, scores 53.00, outperforming all standard and recent methods significant margin. Notably, STAR's individual components are also highly effecttve; CKD and Sim-RL alone surpass most baselines, but their combination yields synergistic improvement, boosting the BFCLv3 score by over 2 points and the ACEBench score by 14 points compared to their individual applications. The detailed statistics show in Appendix A.. and the additional results show Appendix A.
Superior Generalization and Robustness. STAR's generalization capabilities
key advan- tage of our framework. Standard Supervised Fine-Tuning (SFT) leads to performance collapse ACEBench, the model severely overfits to the JSON format of the training data and fails to adapt to the benchmark's s Python-style function call syntax. In stark contrast, the STAR trraind model, despite being trained the same data, demonstrates exceptional robustness. It successfully generalizes its learned function calling abilities to the unseen format, highlighting that our KD+RL. paradigm teaches the model underlying reasoning rather than mere format mimicry.
Performance Across Scales. We validate the effectiveness of STAR across various model sizes, as detailed 3 Our STAR-trained models consistently outperform their base model counterparts and other models of similar scale. The results demonstrate that STAR significantly closes the per- formance gap with much larger models. For instance, our STAR-0.68 model (53.00 on ACEBench) substantially surpasses the much larger Llama3. 1-BB (46.60). And STAR-4B (74.10) outper- forms Qwen3-8B (72.90) on ACEBench. This showcases the framework's 's potent ability to distill and refine capabilities into smaller, more efficient models across various scales.
4.3 ANALYSIS
Why KD+RL over SFT+RL for Super-Tiny Models? The prevalent SFT+RL paradigm, while effective for large models, proves suboptimal for super-tiny models. SFT's hard supervision forees small, limited-capaciny models to overfit and "memorize" specific output formats This leads to policy with limited generalization, as evidenced by its failure and low Pass@k score on ACEBench (Figure 3 and Table 2. and creates poor initialization for RL that limits refinement potential. In contrast, our STAR framework forces the student to mimic the teacher's 's full probability distribution using "soft" supervision through KD training. This encourages learning the teacher's 's reasoning and uncertainty, resulting in a more robust and generalizable initial policy as a stronger foundation for the subsequent Sim-RL refinement.
8

---

Published as a conference paper at ICLR 2026
Sude
Srar
5m RL CKD Sim
Figure 3: Comparison of @ass@k performance for different methods.
Figure 4: Comparison of Pass performance and entropy for different KD methods.
The Role of Constrained Distillation. Our ablation various KD strategies (Table 4) justi- fies choosing CKD. While all KD methods, including recent approaches like RSKD (Anshumann) al., 2025), are better initializers for Sim-RL than cross-entropy (CE), CKD consistently yields the best final performance. Crucially, the CKD-initialized policy already exhibits superior reasoning capacity before RL, achieving the highest Pass@k scores among all initializers (Figure bottom). This metric is a vital indicator of model's potential, measuring its ability to generate a diverse set of correct solutions rather than relying on single, high-confidence prediction (Deng et al. [025] Kang et al. 2025). This advantage stems from CKD's unique re-balancing of learning signals: it preserves the teacher's top-k probabilities while introducing targeted suppression term that pe- nalizes Cconfident-but-wreee
logits more forcefully. This focused distillation creates superior policy initialization that is more amenable to RL refinement by endowing the model with signifi- cantly higher policy entropy at the start of RL training (Figure 4 top). Such entropy is essential for effective exploration and preventing premature convergence in RL. (Sutton 1988; Cui et al. 2025b). This approach synergizes most effectively with Sim-RL, because CKD achieves higher Pass @ k and policy entropy than other advanced methods like Stabilized AKL, which are limited by a suppressed policy entropy that prevents their gains from translating well post-RL. underscores the importance of our constrained approach.
Similarity-basee Reward Design. Our ablation over reward designs (Table demonstrates the inadequacy of standard metrics, like binary reward (Hao al. (0225), for complex tasks. A binary reward proves brittle, failing to generalize as it harshly penalizes functionally correct yet syntac- tically varied solutions. While more advanced methods like the specialized ToolRL reward and SWiRL (Goldie et al. 2025). Process Reward Model (PRM) based variant, offer improvements, our Sim-RI. consistently achieves superior performance, especially on the challenging generaliza- tion benchmark. This advantage stems from its fine-grained continuous reward signal, which eval- uates output similarity rather than a strict pass/fail criterion. This richer signal more effectively guides the policy towards diverse set of valid solutions, enhancing generalization and confirming that a task-aligned similarity metric crucial for optimal policy refinement. The case study shows in Appendix(A.I1
Table 5: Ablation study reward designs.
Method
BFCLv3 Overall
ACEBench Normal
CKD+Binary Reward
51.05
35.70
CKD+To
48.59
40.50
CKD+SwiRL
51.10
40.30
CKD+Sim-RU
51.70
53.00

---

Published as a conference paper at ICLR 2026
5 RELATED WORKS
LLM for Function Calling Function calling is fundamental capability for agentic AI, enabling models to interact with external tools. Early research showed that self-supervised learning could improve zero-shot tool-calling capabilities (Schick et al., (023). Subsequently, a dominant paradigm has been supervised tinettnnng (SFT) on large-scale, synthetically generated datasets with verifiable tool calls (Liu et al.] 2024; 2025] Li et al.] (023). To mitigate impaired generalization caused by naive SFT, researchers have introduced strategies like masking (Lin et al., [025]. More recently, reinforcement learning (RL) has been applied on top of SFT to further enhance performance (Qian et al. 2025). Notably, these advancements are not exclusive to large models, as targeted training has
enabled even IB-scale models to achieve practical tasks like web browsing (Erdogan al. 2024).
Knowledge Distillation Knowledge Distillation (KD) trains a compact student model to mimic larger teacher, originally by matching its output probability distribution (Hinton al.
2015)
Prevailing methods distill knowledge from the teacher's output logits (Gu et al.] 2024| Kim et al.,
2024), intermediate features (Yang et al., 2023). or entire sequences (Kim & Rush (016). Logits-
based approaches, which are common, typically minimize the forward KL divergence (FKL) (Sanh)
al. 2020 Kim et al., 2023), reverse KL. divergence (RKL) (Gu et 2024 et al. 2024). or
both (Wu et al. (0025). More recently, top-k distillation has been explored to improve computational
and storage efficiency (Anshumann et al. 2025
Peng et al. 2025).
Reinforcement Learning Enhancing the reasoning abilities of Large Language Models (LLMs)
through RL has emerged as a prominent research direction (Hu et al. 2025b Xie et al.) 2025 Pan
et al. 2025). This line of inquiry has yielded several high-pergorming models, including DeepSeek
RI DeepSeek-Al et al.) 2025). Qwen3 (Yang et al.) (025). and OpenAl's ol (Jaech et al. 2024)
Central these advancements Proximal Policy Optimization (PPO) (Schulman et al. 2017). foundational RL algorithm. Building on PPO, Group Relative Policy Optimization (GRPO) (Shao al. 2024) simplifies the training pipeline by incorporating verifiable rule-based rewards. Subse- and dynamic sampling. boosting both training efficiency and performance. In parallel, SFT is now standard prac- tice for initializing RL training (Cui et al. 2025a), motivating further research into hybrid paradigms
quently, DAPO (Yu et al. (025) further refines GRPO with techniques like
that optimize the synergy between SFT and RL (Yan et al.) 2025
Ma et al. 2025).
CONCLUSION
We introduce STAR, a framework combining constrained knowledge distillation (CKD) and a similarity-driven RL mechanism Sim-RL to transfer LLMs' capabilities to super-tiny models for efficient, low-latency deployment. Empirically, STAR establishes a new performance benchmark for this model class, rivaling and surpassing some larger models. Our analysis demonstrates that our training curriculum is superior to conventional paradigms for low-capacity models, effec- tively transferring teacher competence into a generalizable student policy. We position STAR promising approach for principled small-model specialization. We hope this work catalyzes further research on compact, reliable agents- exxpoorig multi-teacher strategies, richer reward designs, and deployment-aware constraints-to make capable models accessible where they are most needed.
7 LIMITATION & FUTURE WORK
While STAR demonstrates strong performance on function calling, several limitations warrant fur- ther investigation. First, our current work is validated on function calling, yet the underlying frame- work shows promising potential for generalization to other tasks (c.g., SQL generation, mathemat- ical reasoning), which presents promising avenue for future work. Second, we have explored some similarity-guider rewards to improve the training process. While this initial approach has proven effective, more comprehensive investigation into alternative and potentially more sophisti- cated similarity measures is left for future work. Such an exploration could help in designing more granular feedback, although the potential performance gains remain to be quantified.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Our research is conducted in full alignment with the ICLR Code of Ethics. This study did not in- volve any human participants animal experimentation. The datasets employed, namely ToolACE, XLAM, LLMeeirelevaance and cool-use-synthetic, were all procured strict accordance with their respective usage policies, ensuring no infringement on privacy. Throughout our methodology, we have diligently worked to prevent the introduction of bias and to avoid producing discriminatory re- sults. Furthermore, no personally identifiable information (PII) was processed, and our experimental procedures were designed to pose no risks to privacy or security. We uphold a steadfast commitment to the principles of transparency and integrity in all aspects of our work.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our work the STAR framework, we provide detailed descriptions of our methodology and experimental setup. Our core methods, Similarity-guided Reinforcement Learning (Sim-RL) and Constrained Knowledge Distillation (CKD), are described in Section 3.2 and Section 3.1 The precise algorithm for our novel similarity-basse reward function is detailed via pseudo-code Appendix AA.7 Our full experimental setup, including the specific models, datasets, and benchmarks used, is outlined in Section 4.1 All training hyperparameters, such as learning rates, batch sizes, and method-specific constants, are comprehensively listed in Appendix AA.)] Fur- thermore, details on the prompt formats used for evaluation are available in Appendix A. The code
is open-sourced and accessible at /attps://githiii com/ com/Owen waann/wennAppeeee wete--pppiicatnti STAR
ACKNOWLEDGMENTS
We thank the anonymous reviewers and the area chair for their valuable feedback, which signifi- cantly improved this paper. We also extend our appreciation to Dakui Wang and Xin Li for their generous assistance with the research infrastructure.
REFERENCES
Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self- generated mistakes. The Twelfth International Conference on Learning Representations, 2024.
URL htt.htttss://ppeeeeee htt.pp/nnrrrreeeeeeeeeooorrmaama
net Eorum? wwt.iiddddtttaatthhtoe
Anshumann, Mohd Abbas Zaidi, Akhil Kedia, Jinwoo Ahn, Taehwak Kwon, Kangwook Lee, Hae- jun Lee, and Joohyung Lee. Sparse logit sampling: Accelerating knowledge distillation in llms. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Pro- ceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria, July 27 August 1, 2025, pp. 18085-18108. Asso- ciation for Computational Linguistics, 2025. URL http:s///aallatthoooogoo wrgp:o25..r./.00.......
acl acl-long. 885/
Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang, Xingshan Zeng, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Yuefeng Huang, et al. Acebench: Who wins the match point in tool learning?
arXiv preprint arXi:2250112855. 2025.
Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang. Wendi Li, Bingxiang Hc, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. Process reinforcement through implicit rewards. arXiv preprint 250.0:5502.01456.. 2025a.
Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang. Yuxin Zuo. Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, Zhiyuan Liu, Hao Peng. Lei Bai, Wanli Ouyang, Yu Cheng, Bowen Zhou, and Ning Ding. The entropy mechanism of reinforcement learning for reasoning language models, 2025b. htl/:tttpp::/axxl..orrgg
.26117
Quy-Anh Dang and Chris Ngo. Reinforcement learning for reasoning in small 1ms: What works and what doesn't, it, 2025. URL
11

---

Published as a conference paper at ICLR 2026
DeepSeek-AL, Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, Xiaokang Zhang. Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng. Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangho Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao. Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang. Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen. Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng. Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang. W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xi- aosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang. Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang. Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan. Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang. Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li. Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepscek-rl: Incentivizing reasoning capability in llms via reinforce- ment learning, 2025. htttst::: /ttpss:////xvvv.// org/ abs/ 2501 12948
Jia Deng, Jie Chen, Zhipeng Chen, Daixuan Cheng. Fei Bai, Beichen Zhang, Yingian Min, Yanzipeng Gao, Wayne Xin Zhao, and Ji-Rong Wen. From trial-and-error to improvement: A systematic analysis of Ilm exploration mechanisms in rivr, 2025. htlphttss:///aiiv abs/2508. 07534
Lutfi Eren Erdogan, Nicholas Lee, Siddharth Jha, Sehoon Kim, Ryan Tabrizi, Suhong Moon, Cole- man Richard Charles Hooper, Gopala Anumanchipalli, Kurt Keutzer, and Amir Gholami. TinyA- gent: Function calling at the edge. In Delia Irazu Hernandez Farias, Tom Hope, and Man- ling Li (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 80-88, Miami, Florida, USA, November 2024. Asso- ciation for Computational Linguistics. doi: 10. 1111666///20444mmlpm-moee //aclanthology.co org/ emnlp-mnmmmmmme demo.97 97
URL https:
Anna Goldie, Azalia Mirhoseini, Hao Zhou, Irene Cai, and Christopher D Manning. Synthetic data generation and multi-step reinforcement learning for reasoning and tool use. In Second Conference on Language Modeling, 2025. URL cN9STRYQVa
net net/forummio
Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. Minillm: Knowledge distillation of large lan- guage models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL httts::/
net furt/foru??2 id= 5h0qt IBZZZ
Wenzhe Guo, Joyjit Kundu, Uras Tos, Weijiang Kong, Giuliano Sisto, Timon Evenblij, and Manu Perumkunnil. System-perrormaance and cost modeling of large language model training and in- ference, 2025.
Bingguang Hao, Maolin Wang. Zengzhuang Xu, Yicheng Chen, Cunyin Peng, Jinjie Gu, and Chenyi Zhuang. Exploring superior function calls via reinforcement learning, 2025. URL https:
//axiv.org//bb//// 2ttpa///.02000000.11..1 05118v3
12

---

Published as a conference paper at ICLR 2026
Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in neural network. org/ wwg/abse abs/1503, 02531
CoRR, abs/1503.02531, 2015. URL
Jian Hu, Xibin Wu, Wei Shen, Jason Klein Liu, Weixun Wang, Songlin Jiang, Haoran Wang, Hao Chen, Bin Chen, Wenkai Fang. Xianyu, Yu Cao, Haotian Xu, and Yiming Liu. OpenRLHF: A ray-based easy-to-use, scalable and high-perffrmance RLHF framework. In Ivan Habernal, Peter Schulam, and Jörg Tiedemann (eds.), Proceedings of the 2025 Conference on Empiri- cal Methods in Natural Language Processing: System Demonstrations, pp. 656-666, Suzhou, China, November 2025a. Association for Computational Linguistics. ISBN 979-8-89176-334- 0. doi: 10. 10.11863/111225555 216663///2205..mmmp-mmmmoo// URL httph:ttt///ttt hattps://aclantnollogg. wttp://oooooooor//222200000 org/2025. emn demos. 487]
Jingeheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang. and Heung-Yeune Shum Xiangyu Zhang. Open- reasoner-zero: An open source approach to scaling reinforcement learning on the base model. https://githubs wwm.ttth..ccommeeeeneeeeee
Zero//pen Reasoner- Zero 2025b.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Camey,, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, Andre Saraiva, Andrea Vallone, Andrew Duberstein, Andrew Kondrich, An- drey Mishchenko, Andy Applebaum, Angela Jiang, Ashvin Nair, Barret Zoph, Behrooz Ghor- bani, Ben Rossen, Benjamin Sokolowsky, Boaz Barak, Bob McGrew, Borys Minaiev, Botao Hao, Bowen Baker, Brandon Houghton, Brandon McKinzie, Brydon Eastman, Camillo Lu- garesi, Cary Bassin, Cary Hudson, Chak Ming Li, Charles de Bourcy, Chelsea Voss, Chen Shen, Chong Zhang, Chris Koch, Chris Orsinger, Christopher Hesse, Claudia Fischer, Clive Chan, Dan Roberts, Daniel Kappler, Daniel Levy, Daniel Selsam, David Dohan, David Farhi, David Mely, David Robinson, Dimitris Tsipras, Doug Li, Dragos Oprica, Eben Freeman, Eddie Zhang, Ed- mund Wong. Elizabeth Prochl, Enoch Cheung, Eric Mitchell, Eric Wallace, Erik Ritter, Evan Mays, Fan Wang, Felipe Petroski Such, Filippo Raso, Florencia Leoni, Foivos Tsimpourlas, Francis Song, Fred von Lohmann, Freddie Sulit, Geoff Salmon, Giambattista Parascandolo, Gildas Chabot, Grace Zhao, Greg Brockman, Guillaume Leclerc, Hadi Salman, Haiming Bao, Hao Sheng, Hart Andrin, Hessam Bagherinezhad, Hongyu Ren, Hunter Lightman, Hyung Won Chung. Ian Kivlichan, Ian O' O'Connell, Ian Osband, Ignasi Clavera Gilaberte, and Ilge Akkaya. Openai ol system card. CoRR, abs/2412.16720,, 2024. doi: 1048550/AXXX.2412..6222 URL https::/ddo /00 48550 /0100//arxvvvv. 2412 16720
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training 1lms to reason and leverage search engines with reinforcement learning, 2025. URL htttts:t///aattteeee wwwrxiiiorr//....
Feiyang Kang, Michael Kuchnik, Karthik Padthe, Marin Vlastelica, Ruoxi Jia, Carole-Jean Wu, and Newsha Ardalani. Quagmires in sft-rl post-training: When high sft scores mislead and what to use instead, 2025. URL BtRhhttps:// h/tp:::::///xxvv..// org/ abs/ 02////11100001
Gyeongman Kim, Doohyuk Jang, and Eunho Yang. Promptkd: Distilling student-friendly knowl- edge for generative language models via prompt tuning, 2024. URL https::/aaxxi.orgg/ abs 02522242.222 12842
Minsoo Kim, Sihwa Lee, Janghwan Lee, Sukjin Hong, Du-Seong Chang, Wonyong Sung, and Jung- wook Choi. Token-scaled logit distillation for ternary weight generative language models, 2023.
URL httphhttp:/// /tstps://arxivv org/ abs/ /2t.s/230... 06744
Yoon Kim and Alexander M. Rush. Sequence-level knowledge distillation. In Jian Su, Xavier Carreras, and Kevin Duh (eds.). Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pp. 1317- 1327. The Association for Computational Linguistics, 2016. doi: 10. 10.18653/V1/D166 1139. URL https: wow.rorgg.........oooooooo C. 30653/wl/dle 1139
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brah- man, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Ma- lik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang. Ronan Le Bras, Oyvind Tafjord, Chris
13

---

Published as a conference paper at ICLR 2026
Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi, and Hannaneh Ha- jishirzi. Tulu 3: Pushing frontiers in open language model post-training, 2025. URL https:
//axxiv. arreiiiorg/aaa abs///b///// 211./2211111111111
Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song. Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. API-bank: A comprehensive benchmark for tool-augmented LLMs. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023. URL httpss://spt http://///ppeeeeeee.....m net. Forum?id=o2HBf http://nnraeeeee....eeooooraaaaa
Yixing Li, Yuxian Gu, Li Dong, Dequan Wang, Yu Cheng, and Furu Wei. Direct preference knowl- edge distillation for large language models. arXiv preprint arXiv:2406. 19774, 2024.
Chin-Yew Lin, ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 74-81, Barcelona, Spain, July 2004. Association for Computational Linguis- tics. URL httpss: / www.aantooooo..oooooooooooo-.--.
Qiqiang Lin, Muning Wen, Qiuying Peng, Guanyu Nie, Junwei Liao, Jun Wang, Xiaoyun Mo, Jiamu Zhou, Cheng Cheng, Yin Zhao, Jun Wang, and Weinan Zhang. Robust function-calling for on-device language model via function masking. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. net/ forum? URL http::: htw.aaaeweeeeeeeeeeeeeooooraaaro ii
Weiwen Liu, Xu Huang, Xingshan Zeng, Xinlong Hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, Zezhong Wang. Yuxian Wang, Wu Ning, Yutai Hou, Bin Wang, Chuban Wu, Xinzhi Wang. Yong Liu, Yasheng Wang, Duyu Tang, Dandan Tu, Lifeng Shang, Xin Jiang, Ruiming Tang, Defu Lian, Qun Liu, and Enhong Chen. Toolace: Winning the points of LLM function calling. In The Thirteenth International Conference Learn- ing Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. URL
net/
Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, Rithesh R. N., Liangwei Yang, Silvio Savarese, Juan Carlos Nicbles, Huan Wang. Shelby Heinecke, and Caiming Xiong. Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024. URL http: wwwppappppraeaa nips cc cttccc///peeee htte/paper_fiiie fire__fiees___ paa.e///ppe/22247 Benchmarks Track html
Abst- Daatttt-Daasettsts and
Lu Ma, Hao Liang, Meiyi Qiang, Lexiang Tang, Xiaochen Ma, Zhen Hao Wong, Junbo Niu, Chengyu Shen, Runming He, Bin Cui, and Wentao Zhang. Learning what reinforcement learning can't: Interleaved online fine-tuning for hardest questions, 2025. Utt.https::/arrxi..or
abs/25055 07527
Jaay Pan, Junjie Zhang, Xingyao Wang, Lifan Yuan, Hao Peng, and Alane Suhr. Tinyzero. wwwpa/hhooooaaiiiaaaaaanaaae
2025. Accessed: 2025-01-24.
Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. Gorilla: Large lan- guage model connected with massive apis. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.). Ad- vances in Neural Information Processing Systems 38: Annual Conference on Neural Infor- mation Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024. URL httlttttp///peeeeeeee nips. cc/paper files/ filess/ppprr/224/haa Ttt.L//2200011111111100000000ae0
html
Shishir G. Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. The berkeley function calling leaderboard (bfel): From tool use to agen- tic evaluation of large language models. In Forty-second International Conference on Machine Learning, 2025.
14

---

Published as a conference paper at ICLR 2026
Hao Peng, Xin Lv. Yushi Bai, Zijun Yao, Jiajie Zhang, Lei Hou, and Juanzi Li. Pre-training distil- lation for large language models: A design space exploration. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilchvar (eds.), Proceedings of the 63rd Annual Meet- ing of the Association for Computational Linguistics (Volume I: Long Pupers), ACI, Vienna, Austria, July 27- August 1, 2025, pp. 1600-3618 Association for Computational Linguisties, 2025. URL https: https::/aaaaaatthooooooo
org 2025 acl- 1ooeellooo..180
Cheng Qian, Emre Can Acikgoz, Qi He, Hongru WANG, Xiusi Chen, Dilek Hakkani- Tür, Gokhan Tur, and Heng Ji. ToolRI. Reward all tool learning needs. In The Thirty-ninth Annual Confer- ence on Neural Information Processing Systems, 2025. URL https::/ .ettnreeiieeeee////e//ee net/ -odiiniddeOLGbXX
Victor Sanh, Lysandre Dehut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter, 2020. URL httLl/tss///axii.ooo//b///////// 01108
Sneheel Sarangi and Hanan Salam. Small Ilms do not learn generalizable theory of mind via reinforcement learning, 2025. URL httpp:://////cc.. /ttp//rxvvv....////////////..a org/ http//bb/2207...2. //////507...1111
Timo Schick, Jane Dwivedi-Yis Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Ham- bro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Lan- guage models can teach themselves to use tools. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances Neu- ral Information Processing Systems 36: Annual Conference on Neural Information Pro- cessing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL ots wttp::sppppp.cpp....ie.e.. d842425e 4b wtte///bb0332200f65358880 http://esoeoobbaattaaaoooooooeea -bstrractt-oooffeerre.. html
www..
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. CoRR, abs/1707.0633. 06347, 2017. URL http://arxiv. wwazxiv.org/gb....
1707.0634
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathe- matical reasoning in open language models, 2024. URL
03300
Sharath Turuvekere Sreenivas, Saurav Muralidharan, Raviraj Joshi, Marcin Chochowski, Ameya Sunil Mahabaleshwarkar, Gerald Shen, Jiaqi Zeng. Zijia Chen, Yoshi Suhara, Shizhe Diao, Chenhan Yu, Wei-Chun Chen, Hayley Ross, Oluwatobi Olabiyi, Ashwath Aithal, Olek- sii Kuchaiev, Daniel Korzekwa, Pavlo Molchanov, Mostofa Patwary, Mohammad Shoeybi, Jan Kautz, and Bryan Catanzaro. Llm pruning and distillation in practice: The minitron approach, 2024. URL
11796
Richard S. Sutton. Learning to predict by the methods of temporal differencee. Mach. Learn., 3:9- 44, 1988. doi: 10. 10.100//F000 15009.
Chenxing Wei, Jiarui Yu, Ying Tiffany He, Hande Dong, Yao Shu, and Fei Yu. Redit: Reward dithering for improved I1m policy optimization, 2025. URL wtt.://///..01100000000000011111 2506. 18631
Taiqiang Wu, Chaofan Tao, Jiahao Wang, Runming Yang, Zhe Zhao and Ngai Wong. Re- thinking kullback-leibler divergence in knowledge distillation for large language models. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend AI-Khalifa, Barbara Di Eugenio, and Steven Schockaert (eds.), Proceedings of the 31st International Conference on Computational Linguistics, COLING 2025, Abu Dhabi, UAE, Jamuary 19-24, 2025, pp. 5737 Associ- ation for Computational Linguistics, 2025. URL htttt:://aclaaaneeee /ttp:///cllaan http:///acaaaathhcccccoc wttp:/00....g/002....../
col wg-maiaaaa 383/
Tian Xie, Zitian Gao. Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou. Kai Qiu, Zhirong Wu, and Chong Luo. Logic-rl: Unleashing Ilm reasoning with rule-based reinforcement learning, 2025. URL //tpx/aaavvvv/// abs 14768
15

---

Published as a conference paper at ICLR 2026
Jianhao Yan, Yafu Li, Zican Hu, Zhi Wang, Ganqu Cui, Xiaoye Qu Yu Cheng, and Yue Zhang. Learning to reason under off-policy guidance. In The Thirty-ninth Anmual Conference on Neu- ral Information Processing Systems, 2025. BRLtttts://oo /ttp:///eeeeeeiieeeeeeee hetereviewwweeeeeeeeeeee htw.://wwwee//ormmiiee VOLLLoW
/
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge. Haoran Wei, Huan Lin, Jialong Tang, Jian Yang. Jianhong Tu, Jianwei Zhang. Jianxin Yang. Jiaxi Yang. Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang. Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng. Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang. Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin. Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yugiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. URL https: //apxii org/ at///bbb////5// 09388
Chuanguang Yang, Xinqiang Yu, Zhulin An, and Yongjun Xu. Categories of response-based, feature-based, and relation-based knowledge distillation, 2023, URL http::t/aaxxi..ooot
abs/2306. 10687
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guang- ming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei- Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingx- uan Wang. Dapo: An open-source Ilm reinforcement learning system at scale, 2025. URL
16

---

Published as a conference paper at ICLR 2026
A
APPENDIX
A.1 PROMPT
Our training data is organized according to the Qwen chat template. On BFCL, we employed the QwenHandler with customized system prompt (see Figure Conversely, to adhere to the strict evaluation protocol ACEBench, we used its official, unmodified prompt template
A.2 TRAINING DETAILS
All experiments were conducted using the OpenRLHF framework (Hu al. 2025a) on single server equipped with 8 NVIDIA H20 GPUs. For the various training schemes in our experiments, we employed the following hyperparameter settings:
Reinforcement Learning (RL): For RL training, employed GRPO for fine-tuning. We set constant learning rate 3e-7, with both rollout and training batch sizes 128. The KL-divergence constraint managed via the k2 approximation, with an initial KL. coefficient of le-3. For each prompt, 8 response rollouts were generated.
Knowledge Distillation (KD): For KD training, model was optimized with a learning rate of 3e-6 and batch size of 128. For the tail-suppression term, Aun was set to 10, and both and m were fixed at 100.
Supervised Fine-tuning(SFT) For SFT, the learning rate is fixed 2c-5, while the batch size is set to 128.
A.3 ANALYSIS OF TOP-K FKL AND RKL
This work dissects the gradient dynamics top-k knowledge distillation to provide a principled ex- planation for the contrasting empirical performance of Forward KL (FKL) and Reverse KL (RKL) divergences. We reveal that FKL's success stems from stable, bounded gradient, whereas RKL is prone to instability due potentially unbounded gradient signal, thereby elucidating the funda- mental mechanism behind their differing behaviors.
Notation:
A teacher model produces logits E R,C, yielding a probability distribution p softmax(27).
A student model produces logits 2s R,C, yielding probability distribution g softmax(zs).
We denote 4; = top-k-indices(p) as the index set the the largest probabilities in the teacher distribution As this set is determined solely by the teacher, it is treated constant in the gradient computation with respect to the student's 's parameters.
A.3.1 ANALYSIS OF TOP-K FKL
The top-K FKL loss is defined as:
LOKI.TOpK
Σ setx
Pi log
Σ rEL2
Pi(logy: - log 4:)
(13)
Zi
2The official ACEBench prompt is available at: (CHHench/bblcb//ai////oooo https///nnonddnnieffeeeeeeeerrre
C3/
17

---

Published as a conference paper at ICLR 2026
iimsstartt >assistant
# You are a helpful assistant. The assistant first thinks about the reasoning process in the mind and then provides the user with the answer. The reasoning process are enclosed within <think>explain why the user's question be answered without calling a function or why you should ask the user for more information one or more functions and your plan to solve the user's s senttiinn</ttiik,,.. and then give the answer. You can call the tool by <tool.call> </tool.call> tag.
why you should call
If the user's s question can answered without calling any function, please answer the user's question directly. In this situation, you should return your thought and answer the user's s question directly.
# If the user cannot be answered without calling any function, and the user does not provide enough information to call functions, please ask the user for more information. In this situation, you should return your thought and ask the user for more information.
# If the user's question cannot be answered without calling any function, and the user has provided enough information to call functions to solve it, you should call the functions. In this situation, the assistant should return your thought and call the functions.
# Tools
You may call one or more functions to assist with the user query.
You are provided with function signatures within <tools></tools> XML tags: <tools>
("name": "earnings "earringssgetbymonttt" "description": "Fetches earning data for a specific month and year using the RapidAPI service.", "parameters"; ("month": ("description": "The month for which to fetch earnings data.", "type": "str", "default"; "05"), "year": ("description": "The year for which to fetch earnings data.", "type": "str", "default"; "2022"]]} ("name": "creditcard. sew.addd.eeerrttecc.mmeeee
"description": "Generates a fake credit card number using the specified brand and API key.", "parameters"; ("brand": ("description"; "The desired card brand (e.g., 'Visa', 'Mastercard'). Defaults to None.", "type": "str, optional", "default"; ""]]} </tools>
For each function call, return json object with function name and arguments within stool_callx</tool.co XML tags: <tool_call>
{"name": <function-name: "arguments"; <args-json-objett>) </tool_callx(/mmendd)
<|im_start />user >user
I want to analyze the market for Apple (AAPL) First, give the most recent Minus Directional Indicator (MINUS.DI) for AAPL using daily intervals, then interpret what that value implies for the stock's s short-term price movement. < iimendd <lim start assistant <think>
Figure 5: Customized system prompt example on BFCL evaluation.
18

---

Published as a conference paper at ICLR 2026
The partial derivative of the loss with respect student logit 2g, is found by applying the chain rule with the softmax derivative JEs, qi(ōis - qj), yielding:
LFKL-TopK
OC dq: Σ
Pj
0q:
Ozs,
dq: dzs; iET
dzs,
Σ
(6i)
93)
q
iEh
4j
Σ iEJ=
Pi
Pj
1jEh
(14)
To elucidate the underlying training dynamics, we decompose the FKL-TopK gradient by analyzing its components for logits within the top-k set versus non-top-k set:
For a non-top-k logit (j & Ig):
TLKI 0zs,
4j
Σ i
(15)
For a top-k logit (j E Tx): i DLL Topk Øj Σ
0zs,
Pi Pj T
(16)
This formulation induces learning dynamic where logits for top-k and non-top-k items receive fundamentally different treatments. The gradient for top-k logit is strictly smaller than the gradient for any non-top-k logit (since Py > 0). This creates a clear, stable dynamic: the logits non-top-k items strongly suppressed, while the logits of top-K items either encouraged (if the gradient negative) or suppressed much more weakly. The model learns to focus its probability mass on the teacher's 's chosen top-K candidates.
A.3.2 ANALYSIS TOP-K RKL
The RKL objective presents a fundamental issue. we define a proper probability distribution p' from the teacher's top-k logits by padding with zeros (i.e., = 0 for i T&), the RKL Dxu (glle) becomes ill-defined. Any student probability qi > for an index E I would result in a term qi log(g./O). causing the loss to diverge infinity, which is impossible to optimize.
The only viable alternative a masked RKL, which is not true KL divergence over the full vocab- ulary:
Σ q log gi
CRKL-Topk
(17)
iets
The gradient of this loss with respect to a student logit 29, is:
ÖLRII-TopK
O(gi log %:)
0q.
02s,
Oqp
Ozs,
REl:
Σ REl:
log li
+
(:(0s - -9))
pi
qj
log
Tj
+
1
1jet,
Σ gi
qi log
+ 11]
(18)
pi
sEle
Let's analyze the dynamics by defining the summation term S Eier If (log For a non-top-k logit (j $ Ix):
Is
1).
p:
ToRK S
(19)
02g,
19

---

Published as a conference paper at ICLR 2026
For top-k logit (j Ig):
@LRLL-TopK
Li + 8) log Pj
(20)
Ozs,
This structure, however, can induce undesirable optimization dynamics. Specifically, when the teacher assigns negligible probabilities (pj 0) to certain top-k items, when the student be- comes over-confident (i.e., its probability mass is highly concentrated), the gradients for some top-k logits become smaller than those for non-top-k logits. In this regime, the model is para- doxically incentivized to promote non-top-k items over some within the top-k set, irrespective of an external signal S. This behavior often leads to poor convergence and, in extreme cases, training collapse.
A.4 STABLE VARIANT OF TOP-K RKL AND AKL
To remedy the instability of top-k RKL and Adaptive KL divergence (AKL), we introduce tail suppression term, analogous to the one used in CKD. Let Jm = top-m-indices(y) be the indices of the student's 's top-m predictions, and J = Jm \ I4 be the set "confident but wrong" predictions. The stabilized top-K RKL loss is defined as:
SstaillizedRRK RKI. Topk = LRKI.TTpK + Ltait Σ iEL
qi
log q + Σ9
(21)
Pi
JEJ
The gradient of Least with respect to student logit 2s, is:
OC
((q:)) 0q:
Ozs;
3g: Ōzs,
Σ
(gi
45)
SEJ
>aj
15E
Maj Σ qi
(22)
iE
Let's analyze the the new gradients with the tail suppression term
For a top-k logit (j In):
@CStabillettRKLL-Tpx Qj
log li
S+A
1- Σ go
(23)
Ozs;
sE
For a confident-but-wrrog logit (j E J(.):
d Rdui = Gi 1
Σ qi - iE
(24)
Ozs,
The tail suppression term introduces positive component Ag; Σiejs 4)). For sufficiently large X, the gradient for a confident- buttt-but--oooot out-wrong logit is larger than that for top-k logit. This restores a stable learning dynamic by ensuring that the student is penalized for confidently predicting classes outside the teacher's top-k set.
A.5 GRADIENT ANALYSIS FOR CKD
Our proposed CKD method combines top-k FKL with the same tail suppression mechanism (see Equation 3) Unlike with RKL, the goal here is not to fix instability but to refine the already stable FKI. dynamics prevent over-confidence. The gradient with respect 23, is:
20

---

Published as a conference paper at ICLR 2026
For top-k logit (j Ix):
dCCDD
EDi
P
E
(25)
0zs;
Ve s'd
For a confident-but-wrongg logit (j E J'm):
OLCKD dzs,
qj
Σ iELn
Pi +
1- Σ qb
(26)
IEV
For other non-top-k logits (j $ I U.m::
0CCKD Ozs,
Σ iE
pt ΣP-ク q
(27)
iEJ
Compared to the standard FKL gradient in Equation 14 CKD strategically re-balances the learning signals:
1. Targeted Suppression:
The gradient for ccoffidentbut-wrrorg but--rong"" logits (j E J' is signif- icantly increased. This focuses the suppressive force on the most likely sources of error, penalizing the student for being confident in incorrect predictions.
2. Relaxed Suppression: The gradient for other non-top-k logits (j & la Jy, is reduced. This tells the model not waste capacity aggressively suppressing classes already assigns low probability to.
This re-balancing mechanism prevents the model from collapsing probability mass entirely onto the top-k set Ig. By forcing the student to specifically avoid confident mistakes outside of /x. CKD encourages healthier, less peaky student distribution, which translates to improved generalization and robustness, thus addressing the primary limitation of top-k FKL.
A.6 SENSITIVITY ANALYSIS
Table 6: Sensitivity analysis on hyperparameters and Atail for CKD.
k
Atail
BFCL v3 Overall
AceBench Normal
w/o RL
w/ RL
w/o RL
w/ RL
10
49.58
51.48
43.20
49.20
100
10
49.56
51.70
39.00
53,00
1000
49.84
51.59
36.70
52.20
50.12
51.11
38.00
48.20
48.78
50.62
39.10
50.10
100
10
49.56
51.70
39.00
53.00
30
49.82
51.80
41.70
47.80
100
48.85
51.83
42.10
48.20
We investigate the sensitivity of our Constrained Knowledge Distillation (CKD) method to its two key hyperparameters: & and Apal-- Table 6 presents the results on both BFCLv3 and ACEBench- Normal benchmarks, with and without subsequent Sim-RL refinement. For this analysis, m is fixed at 100, value large enough to capture the student's 's most probable and potentially erroneous outputs.
First, observe that across wide range of hyperparameter settings, CKD maintains strong per- formance, frequently surpassing the results of competing methods shown Tables [ and This demonstrates the robustness of our proposed approach.
21

---

Published as a conference paper at ICLR 2026
Analysis of k. The hyperparameter k defines the size of the trusted vocabulary set from the teacher model. A very small & (c.g., h = 10) overly constrains the student, forcing it to mimic narrow distribution, which can harm generalization as reflected by the lower performance post-RL.. Conversely, very large l (e.g.. h = 1000) makes the LFKLa term approximate the stan- dard forward KI. divergence and reduces the impact of the tail penalty, offering diminishing returns while still performing well. Our chosen value of A 100 strikes an effective balance, providing suf- ficient guidance from the teacher without excessively restricting the student's distribution, proving beneficial for both initial distillation and subsequent RL adaptation.
ACEBench
Analysis of Atal The weight Auil controls the strength of the tail suppression penalty. A small weight (e.g., Amil = 1) is insufficient to suppress the student's 's tendency to assign probability to irrelevant tokens, leading to suboptimal performance after RI. As Aual increases, performance im- proves, peaking at Aual = 10, especially on ACEBench. However, excessively large values (e.g., Auil 30) can overly punitive. This may excessively suppress the student's output probabilities, making the distribution too sharp and hindering the exploratory capacity that is crucial for effective RL fine-tuning, as reflected by the performance drop on ACEBench. Thus, Auil = 10 provides an optimal balance that effectively regularizes the tail distribution while preserving healthy capacity for exploration.
A.7 PESUDO CODE OF THE REWARD
The total reward R is calculated using a composite function that first evaluates the syntactic for- mat and then, if the format is correct, the accuracy of the tool calls the textual response. The framework defined by the main function Ca cu ateoota Reward(see Algorithm ] and its subroutines.
Algorithm Total Reward Calculation
function CALCULATETOTAL
GroundTruth, ToolSchema)
2: Input:
3:
Generation: The full string output from the model.
4:
GroundTruth: The label string containing the correct output.
5:
ToolSchema: A definition of available tools and their parameters.
6:
Output:
7:
R: The final reward score the range [-1, 1].
8:
Reomat +- BAL.CUUATEFOORAATREE.ROOOOO
Tool ToolSchema)
9:
if Rioomat 0 then
10:
return-1
end if
12:
Poallsi Presponse +- PARSE( PARSE(GGnnraatoon Generation)
13:
Gealls; Gresponse +- PARSE (aaoundr
14:
Biool <-
15:
Bresponse f- 0
16:
if Gealls is not empty then
17:
Riool + CALCULLAEETOORRWWWADDD Gealls)
> See Algorithm
else
19:
Rresponse f- CALCULLTERRESPONSFEEVAAA
(P response Gresponse
end if
21:
R 4- Rggol + Rresponse
22:
return R
23:
end function
A.8 THE USE OF LARGE LANGUAGE MODELS (LLMs)
During the preparation of this manuscript, employed Large Language Model (LLM), specif- ically Gemini 2.5 Pro as a tool for linguistic enhancement and technical formatting, to assist in polishing the language in certain sections. The model's contributions were directed towards improv-
22

---

Published as a conference paper at ICLR 2026
Algorithm 2 Tool Call Reward Calculation (Rion)
1: function CALCULATTETOORREEVVVDDPDD. Gealls)
2: total similarryy f-
3: Gaalis f mutable copy of Gealls
4:
for each predicted call p E Paalls do best_mateh mest_ best match g + null for each ground-truth call G calls do
5:
-1
6:
7:
8:
p.name- = g.name then
9:
$ AAGUUEETTSMMAAARRY(aarruuuun
wtw.:raaaguuetssgggaggmeeeeeeee g.arguments)
See Algorithm
10:
if > best_match_score then
best_match_ssor score <- $
12:
best metchha f-
end if end if end for
14:
16:
if best_match_gg is not null then similariyy +- total similarity + best mest__matc_sson score Remove best.match.g from Gealls
17:
18:
19:
end
20:
end for
21:
union size <- Pealts + +Gaallls
22:
if union_sizz 0 then return 1
23:
elsereturn similariyl union_size
end if
25:
end function
Algorithm Argument-level Similarity (sim)
1: function ARGUMENTSIMILARTTN
Gargs)
2: intersection_key _keys <- keys( P args) keys Cargs)
3: unyon_keys + keys(PPaaaa Pargs)) U keys( (Gargs)
4: weighted sum 0
5:
for each key E intersection keys do
6:
Pk < Pars [k), O Gangs[Ki
7:
if Pk, Sb are Strings then score +- (OUGE-L.FF
8:
9:
else if Pk: 9k Numeric/Boolean then score + (1 if pk = Gk else 0)
10:
11:
else
12:
score +- (1 if str(pk) = str(gk) else 0)
13:
end if weighted_sum + weighted.sumt + score
14:
end for
16:
if |union = 0 then return
else return weighted_sum) hwtp////nineessss
18:
end if
end function
ing readability, clarity, and the overall flow of the text through tasks such sentence rephrasing and grammar correction. Furthermore, was used for assistance with LaTeX syntax to ensure proper formatting.
must be emphasized that the LLM's function was strictly supportive and limited to the aspects mentioned above. The model had no role in the core intellectual work, which includes the ideation, research methodology, experimental design, data analysis, or interpretation results. All scientific ideas, concepts, and analyses presented herein exclusively conducted by the authors.
23

---

Published as a conference paper at ICLR 2026
The authors have thoroughly reviewed and verified the entire manuscript and take full responsibility for its final content, including all text polished or formatted with the aid of the LLM.
A.9 DETAILS OF EVALUATION METRICS
Evaluation Metrics for BFCLv3. The evaluation metrics for BFCLv3 are listed below:
Overall Acc: This metric represents the comprehensive performance of the model on the entire BFCLv3 benchmark. It is calculated weighted average of the accuracies from various specific evaluation categories, providing a single, overarching score to rank differ- ent methods.
Non-Live Ace: This metric assesses model performance primarily on the static BFCL VI dataset. This dataset was curated by the benchmark creators and includes single-tur scenarios like simple, multiple, and parallel function calls. As noted in the documentation, this portion of the benchmark may be susceptible data contamination for models trained public datasets.
Live Acc: This metric measures model performance on the BFCL V2 live dataset. This dataset is composed of live, user-contributed function documentation and queries, designed to tackle issues of data contamination and bias. It aims to faithfully evaluate model's ability to generalize and perform effectively in diverse, real-world tool-use scenarios that has not seen before.
Multi Turn Acc: Introduced with the BFCL V3 dataset, this metric specifically evalu- ates the model's proficiency in handling multi-turn and multi-step function calling tasks. It tests the model's 's ability to maintain conversational context over several exchanges, cor- rectly interpret user follow-up requests, and make appropriate function calls based the accumulated dialogue history.
Evaluation Metrics for ACEBench-Normal. The evaluation metrics for ACEBench-Normal listed below:
Summary: This is summary score that aggregates the performance across all sub- categories within the Normal dataset to provide a single, comprehensive measure of the model's general tool-use capability in standard scenarios.
Atom: This metric evaluates the model's performance on atomic cases, with specific focus on ability to handle different parameter types. It involves the precise assessment of the model's handling of data types such as enums, numbers, lists, booleans, and objects.
Single-Turn: This metric assesses the model's 's basic tool-calling competence in scenarios that resolved within single conversational turn.
Multi-Ture This metric measures the model's capability in multi-turn dialogue flows. It assesses whether the model can perform context-Sensitive orchestration of tool calls and maintain state memory across several conversational turns to fulfill the user's goal.
Similar API: This metric tests the model's 's ability to distinguish between nearly identical tool specifications. The model must select the correct API based subtle differences in the user's query and the API documentation.
Preference: This metric evaluates the model can incorporate contextual user information for API selection. The model must make preference- based selection by taking the user's history or profile into account.
A. STATISTICAL SIGNIFICANCE
To ensure the statistical significance of our findings, we conduct experiments with 3 different random seeds. The results are presented as mean + standard deviation. Tables 7 and detail the comprehen- sive performance of our methods against various baselines the BFCLv3 and ACEBench Normal benchmarks, respectively, showing that our CKD+Sim-B approach consistently achieves strong re- sults across most evaluation metrics. We further validate architectural choices through ablation studies: coblemJo the superiority of our proposed CKD strategy over other knowledge distil- lation techniques, and Table 10 demonstrates the significant advantage of Sim-RL. reward design
24

---

Published as a conference paper at ICLR 2026
compared alternative reward functions. The consistently low variance and superior performance of our proposed components across all these detailed tables underscore that our method is robust and its effectiveness is statistically well-supported.
Table 7: Performance comparison of different fine-tuning methods on Qwen3-0.66l -.,6B, evaluated on the BFCLv3 benchmark. Values are presented as mean + std.
Method
Overall Acc Non-Live Ace
Live
Multi Turn Acc
Standard methods
Base-mon modee
47.33
71.81 65.40 + 0.33
65.66 61.67 + 0.05
1.88 1,45 + 0.15
SFT
44.19 + 0.17
SFT-think
47.14 0.20
69.82 0.81
64.59 + 0.33
3.94 + 0.31
FKI
49.68 0.12
76.05 + 0.26
66.16 + 0.41
5.41 + 0.25
Recent methods
ToolRL
47.78 0.13
66.27 = 0.57
66.57 = 0.14
6,95 = 0.52
LUFFY
48.81 = 0.18
75.68 = 0.30
64,80 = 0.29
4.62 = 0,44
GKD
47.71 0.20
67.50 = 0.74
67.89 = 0.17
3.95 = 0.42
Our methods
CKD
49.61 0.31
75.44 + 0.38
66.09 + 0.20
5.12 + 0.54 0.54
Sim-RL
49.56 0.15
75.51
67.52 = 0.20
3.33 0.11
0.11
SFT+Sim-RL."
50.30 0.14
76.78 + 0.35
66.81 + 0.20
5.62 + 0.30
CKD+Sim RL
51.80 0.12
78.98 = 0.29
68.41 = 0.30
6.93 = 0.48
Results the official benchmark; error not provided. It refers to Sim-RI. on SFT-think.
Table 8: Performance comparison of different fine-tuning methods on Qwen3-0.6B, evaluated on ACEBench Normal benchmark. Values are presented as mean + std.
Method
Summary
Atom
Single-Turn
Multi-Turn Similar API
Preference
Standard methods
Base-model"
27.20
37.70
19.50
10.00
36.00
6.00
SFT
2,03 - 0.16
1.23 0.41
0.16 0.23
0.00 = 0.00
16.67 + 1.88
0.00 + 0.00
SFT-think
30.17 = 1.28
44.47 + 1.67
14,62 + 1.63
12.75 + 2.16
29.00 + 4.12
11.49 = 2.59
FKL
37.76 = 1.52
53.55 + 3.03
19.43 2.06
18.31 = 3.03
42.37
14.00 = 3.84
Recent methods
ToolRL
27.77 = 1.00
41.00 2.35
12.25
1.03
10.00 + 0.70
3
7.50 + 2.17
LUFFY
44.83 0.97
60.56
1.56
25.50
2.48
22.99
2.16
56.00 4.32
22.00 +
GKD
37.36 + 1.17
49.43
2.04
21.50
1.08
18.66
1 0.94
48.00 1 4.32
23.33 1 4.98
Our methods
CKD
39.71 + 1.20
55.94 + 2.09
20.51
2.24
20.00 + 2.77
43.06 + 4.87
16.87 + 4.12
Sim-RL
40.63 1 0.83
54.76 1 1.11
25.83
1 1.02
22.99
1 4.08
50.66
1 3.39
10.66
1 2.49
Si-+-
38.35
0.84
$2.80
1.43
22.59
2.17
19.56
3.11
44.43
4.54
14.43
1 3.59
CKD+Sim-RL
52.11 1 0.91
69.76 1 1.51
33.67
1.62
30.66
1 3.09
57.67
4.10
21.33 1 1.94
Resultt from the official benchmark; error bars not provided. "It refers Sim-RL on SFT-think.
25

---

Published as a conference paper at ICLR 2026
Table 9: Model performance on function calling benchmarks with different KD strategies. Values presented mean + std.
BFCLv3 Overall
ACEBench Normal
Method
w/o RL
w/ RL
w/o RL
w/ RL
CE
47.14 + 0.20
50.36 + 0.10
30.17 + 1.28
38.35 + 0.84
FKL
49.68 + 0.12
51.16 0.14
37.76 + 1.52
49,96 + 1.04
RSKD
49.34 + 0,29
50.50 + 0.11
38.10 + 0.92
50.53 + 0.54
RKL'
49.29 + 0.26
50.58 0.06
36.19 1.07
40.56 + 0.37
AKL
49.59 + 0.39
50.40 0.10
43.80 = 1.23
49.33 = 0.77
CKD
49.61 + 0.31
51.80 + 0.12
39.71 = 1.20
52.11 = 0.91
The stable variant.
Table 10: Ablation study on reward designs. Values are presented as mean + std.
Method
BFCLv3 Overall
ACEBench Normal
CKD+Binary Reward
50.83 + 0.09
37.43 + 0.90
CKD+TooIRL
49.01 + 0.13
43.00 + 1.51
CKD+SwiRL
50.96 + 0.09
37.63 + 2.05
CKD+Sim-Re RL
51.80 + 0.12
52.11 + 0.91
A.11
CASE STUDY
This appendix provides qualitative analysis illustrating how Sim-RL addresses critical failure modes present in other reward designs.
demonstrates the rigidity of binary rewards. Functionally correct outputs- as a func- tion call missing an optional argument or containing a trivial formatting difference-are incorrectly assigned score of O. This provides no useful learning signal. Sim-RL resolves this by assign- ing partial credit for correct function and primary argument (0.5) and full score for semantically equivalent outputs (1.0), thus rewarding genuine progress.
Table 11
Table 12 highlights that even some well well-performing RL methods can still encounter the issue of "reward hacking." For instance, tool call may receive perfect score (1.0) because it is syntacti- cally correct for the user's immediate question. However, by ignoring the conversation history, this rewards an inefficient and redundant action if the model already has the answer. The model then learns to game the system by making simple, unnecessary tool calls, exacerbating this inefficient behavior. Sim-RL avoids this by comparing the action against the optimal context-aware response (a direct answer) and correctly assigns a score of 0.0 to penalize the suboptimal action.
In summary, Sim-RL combines semantic flexibility handle near-correctness with contextual grounding to prevent reward hacking, resulting in more robust and reliable reward signal for train- ing agents.
26

---

Published as a conference paper at ICLR 2026
Table Binary Reward vs. Sim-RL for Partially Correct Tool Calls.
Example 1: Missing Default Argument
Example 2: Trivial Formatting Difference
Function
Check.wordpress
label.tepllteebbe e.aands
Query
"Can you check if
"Can you list the brands available for A4 size blank label sheets?"
https: /taps://exeee /wwemxxapee..ee/eeeee com is running WordPress?"
Model Rollout
("name": "check.wordpresss, "azguments"; ("url": com")
{ ("name": "Label.template hrrandddtebbradd".......
"arguments"; {"format": "a4"}]
Ground Truth
("name": "checkk.wordpress",
{ ("name":
"azguments"; {"url":
"arguments": {"format": "A4"]}
"https://exampl.. com", "user.agent" (tozilla/5.00) 0"}}
Binary RL Score
(Mismatch)
O (Mismatch)
Sim-RL Score
0.5 (Partial credit for correct
1.0 (ROUGE-L is case-insensitive)
function and primary argument)
Table 12: Example of Reward Hacking via Redundant Tool Call
Example: Redundant Tool Call (Reward Hacking)
Context
In previous turn, the model already looked up information for "SFO" airport. "What is the ICAO code for SFO airport, and how many runways does it have?"
Query
Model Rollout
<tool.call> ("name": "airportstatistics"""
"aruumenss: {"iata": "SFO"}} stool.ccall
Ground Truth
"The ICAO code for SFO is KSFO, and it has runways."
SwiRL Score
1.0 (Rewards the valid-looking tool call, ignoring context)
Sim-RL Rin-RLSah Score
0.0 (Penalizes the unnecessary call compared to the optimal response)
27

---

Published as a conference paper at ICLR 2026
A.12 ADDITIONAL RESULTS
To strengthen our claims and provide deeper insights, we have conducted the additional experiments detailed analysis.
and incorporated
Comparison on a Larger Student Model: We apply our method to a 1.7B student model and compare its performance against the SFT+Sim-RL baseline. As Table I3 shows, CKD continues to outperform SFT, confirming the scalability and effectiveness of our approach on larger models.
We also conduct an ablation study on the teacher model's size, using a Qwen3-14B model as the teacher. The results in Table 14 below show that our method remains effective, demonstrating its robustness to the choice of teacher model size.
Ablation on Teacher Model Size:
Ablation on Teacher Refinement: We run an ablation study comparing Qwen3-0.6B students dis- tilled from the base teacher vs. the refined teacher, which is shown Table [3] The results show that while better teacher indeed leads to better student, this does not affect the overall validity of our method. With the un-refined teacher, our core STAR method (CKD Sim-RL. still clearly outperforms the standard SFT+Sim-RL baseline. Furthermore, without refinement, the suboptimal base teacher model leads to the comparatively lower performance of the student model after only the CKD stage. However, the subsequent Sim-RL to this student model results in a substantial per- formance gain. These observations are crucial as they substantiate the robustness our framework, demonstrating its effectiveness even when initialized with less capable teacher model.
Table 13:
Performance comparison of CKD and SFT, followed by Sim-RL. The teacher model is the refined Qwen3-8B, and the student model is Qwen3-1.7B.
Method
BFCLv3 Overall
ACEBench Normal
SFT to Qwen3-1.70 + Sim-RL CKD Qwen3-1.78 Sim-RL
55,54
56.20
56.05
60.90
Table 14: Ablation study on teacher model size. The student model is Qwen3-0.6B.
BFCLv3 Overall
ACEBench Normal
Method
w/o RL
w/ RL
w/o RL
w/ RL
CKD from Qwen3-8B Sim-RL
49.56
51.70 50.75
39.00 38.50
53,00
CKD from Qwen3-14B- Sim-RL
50.12
54.30
Table 15: Ablation study on teacher refinement. The teacher model is Qwen3-8B and the student model is Qwen3-0.6B.
BFCLx3 Overall
ACEBench Normal
Method
w/o RL
w/ RL
w/o RL
w/ RL
SFT+Sim-RL
47.59
50.41
28.70
38.90
CKD from Qwen3-8B (im-s)
47.13
51.35
31.40
47.20
CKD from Qwen3-8B (Retine))+Sim-RL.
49.56
51.70
39.00
53.00
28


Reviews:
[{"review_id": "usEFckFLk8", "reviewer": "Reviewer_ovcd", "summary": "This paper proposes a two-stage training framework—a knowledge distillation phase followed by a reinforcement learning (RL) phase—to enable a compact 0.6B Qwen model to achieve strong tool-calling capabilities. For each phase, the authors introduce targeted improvements. In the distillation phase, they use a forward KL divergence variant that augments the standard top-k forward KL to suppress confidently incorrect predictions. In the RL phase, they design a heuristic reward function that evaluates the similarity between rollouts and ground truth with more fine-grained criteria, providing richer reward feedback compared to conventional binary rewards. By combining these techniques, the authors demonstrate that their method effectively trains a small model with strong tool-calling performance.", "questions": "NA", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The paper provides targeted and practical improvements for enhancing tool-calling capability, offering insights that could be useful for others working in this area. The performance gains are solid and well-demonstrated through experiments", "weaknesses": "* The framework is sound and the empirical results are solid; however, the methodological contribution is not particularly significant.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2sHLYzqN6N", "reviewer": "Reviewer_949b", "summary": "This paper proposed a framework to effectively imbue tiny-LMs with tool calling capabilities of their larger models. The proposed method consists of two stages: (1) Constrained Knowledge Distillation to prevent highly confident incorrect predictions by the student model, and (2) a similarity-based RL refinement on top of the trained student that computes a fine-grained similarity-based reward between the ground-truth functional calls and the model prediction (more suitable for multi solutions problems). The authors verify their approach by comparing it to several baselines evaluated on BFCL and Acebench (for testing generalization). Results showed improved performance across all benchmarks with the more profound boost on Acebench showing better generalization to unseen function call formats.", "questions": "1- Line 057: typo: the -> the\n\n2- Minor suggestion for structuring Sec 3: chronologically, it would have made sense to start with distillation and then talk about refinement (sim-RL)\n\n3- [line 106] Add citation for RLVR: Lambert, Nathan, et al. 2025. “Tulu 3: Pushing Frontiers in Open Language Model Post-Training.” In Proceedings of the Second Conference on Language Modeling\n\n4- [Line 270]: If I understand correctly, you have two iterations of Sim-RL? you refine both the teacher and the student using Sim-RL.  It would make sense to try to clarify this both in text and figure to avoid confusion.\n\n5- Baslines: which baseline is representing a simple binary RL reward? This is specially important and relevant to you analysis section explaining your reward design", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is generally well-written and well motivated\n- Experimental results are promising and show strong generalization compared to baselines\n- The method seems to simple and effective at mitigating overfitting problem especially when compared to conventional approach of SFT+RL\n- Results on closing the performance gap with much stronger models in Table 3 is pretty interesting\n- Also appreciate additional theoretical explanation for their Top-K truncation with FK", "weaknesses": "- There’s no direct comparison with existing RL-based methods. It’s not clear how the proposed reward is different from those proposed in related prior works for example one in [1]. In general more comparison with existing RL rewards would be nice. (see more in questions)\n\n[1] Anna Goldie, Azalia Mirhoseini, Hao Zhou, Irene Cai, and Christopher D Manning. Synthetic\ndata generation and multi-step reinforcement learning for reasoning and tool use.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gB62MtJ8WL", "reviewer": "Reviewer_u15k", "summary": "The paper introduces a training pipeline to adapt tiny-scale language models for function calling. Specifically, the introduced pipeline is composed of a RL method, Sim-RL, with rewards crafted based on the similarity between generated function calls against the ground truth and a knowledge distillation method, Constrained Knowledge Distillation (CKD), that uses forward KL-divergence as the divergence metrics with an additional component to penalize the confident-but-wrong tokens. The combination of these two approaches, STAR, shows the best performance among models under 1B and closes the performance gap with larger models on two benchmarks.", "questions": "- It seems that the methodology of CKD is not specific to the task of generating function calls. Has the author considered applying this method to other tasks? If not, what constrains CKD to this specific task?\n- The paper lacks some explanations on the categories of the benchmarks shown in Table 1 and 2.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper is relatively well-written and easy to follow.\n- The performance gains on the 0.6B model scale is consistent over most metrics in the two evaluation benchmarks, and the performance gap with larger is significantly reduced.\n- The paper includes discussion on the comparison between KD+RL and SFT+RL besides the empirical results that might be insightful for future work.\n- The paper includes analysis on the comparison among different KD methods.", "weaknesses": "- Sim-RL looks highly dependent on the generated function calls’ format that the author defines based on the Qwen tool calling template. It might be important to show the generalizability of this method for alternative formats. \n- STAR requires RL training on the teacher model, which introduces additional non-trivial compute cost compared to SFT+RL.\n- More potential analysis studies could improve the persuasiveness of the paper in showing the advantages of CKD over SFT. For example, a comparison between these methods in a larger model (e.g. 1.7B), an ablation on the teacher model’s size, etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XUzH28Y2PZ", "reviewer": "Reviewer_Qjs5", "summary": "This paper explores training extremely small language models for function calling. The authors introduce the STAR framework, which extends the traditional Knowledge Distillation (KD) + Reinforcement Learning (RL) pipeline with two innovations. First, Constrained Knowledge Distillation (CKD) mitigates key shortcomings of existing KD methods. Second, during the RL phase, SimRL employs a similarity-based reward function aligned with ground-truth outputs. Through evaluations on standard benchmarks, the authors demonstrate that STAR models surpass state-of-the-art KD and RL approaches within the super-tiny model regime.", "questions": "1. Is the improvement of CWD on baselines (such as AKL) statistically significant?\n\n2. How does the Sim-RL reward function compare to other reward functions in the literature (ex: ToolRL)?\n\n3. What is the impact of using the teacher’s generations D^T v/s D?\n\n4. In KD, the asymmetry of the divergence seems to be leading to poor performance. What if you replace it with a symmetric divergence? Ex: Jensen Shannon", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "Strengths:\n\n1. The paper is well written and easy to follow, the experimental results are also well presented.\n\n2. The CKD loss is well motivated, and the gradient analysis is particularly insightful.\n\n3. The final performance of the STAR models are impressive.\n\n4. The paper shows that current SOTA  RL / distillation methods struggle in the super tiny models regime, which is interesting.", "weaknesses": "The two main contributions of this paper are 1) the CKD loss function and 2) the sim-rl reward function, and the impact of both has not been well studied.\n\n1. In Table 4, CKD’s performance does not appear significantly stronger than other loss functions. For instance, CKD outperforms AKL by only a small margin (~0.1–0.5) on the BFCLv3 benchmark but performs worse than AKL on ACEBench without RL. Were the improvements of CKD tested for statistical significance?\n\n2. The proposed reward function has not been evaluated against prior approaches—for instance, the similarity-based reward function in ToolRL or other variants based on ASTs and PRMs. As a result, its effectiveness remains unvalidated.\n\n3. There is no ablation on the effect of distilling from a teacher; what if STAR is applied to D directly?\n\n4. (Line 418) “standard metrics are unreliable..” – what are the standard metrics? I also didn’t follow how sim-rl is better suited to handling the stochasticity of the teacher?\n\n5. (Minor) Since CKD is applied first in the pipeline, the paper reads better if it’s introduced first.\n\n6. (Minor) A lot of citations are missing a space after the text, for ex. Line 38: “function calling(Patil et al., 2024; Jin et al., 2025)”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)