Language Ranker: A Lightweight Ranking framework for LLM Decoding
Authors: Chenheng Zhang
Tianqi Du
Jizhe Zhang
Mingqing Xiao
Yifei Wang
Yisen Wang
Zhouchen Lin
Venue: NeurIPS 2025 poster ()
Keywords: language model
recommender system
inference-time computing
Primary area: deep_learning
TL;DR: We rethink LLMs from the perspective of recommender systems and propose Language System, which exploits the model’s output distribution more efficiently and effectively.

Abstract: Conventional research on large language models (LLMs) has primarily focused on refining output distributions, while paying less attention to the decoding process that transforms these distributions into final responses. Recent advances, such as scaling the computation of inference time with reward models, have underscored the importance of decoding, but these methods often suffer from high computational costs and limited applicability.
In this paper, we revisit LLM generation through the lens of recommender systems, conceptualizing the decoding process as analogous to the ranking stage in recommendation pipelines. From this perspective, we observe that both traditional decoding methods and reward models exhibit clear limitations such as redundancy.
Motivated by this insight, we propose Language Ranker, a novel framework that introduces a lightweight module to rerank candidate responses using features extracted by the base model. Experiments across a wide range of tasks show that Language Ranker achieves performance comparable to large-scale reward models, while requiring only <0.5M additional parameters, significantly reducing the computational overhead during both training and inference stages.  This highlights the efficiency and effectiveness of our method, showcasing its potential to fully unlock the capabilities of LLMs.

Language Ranker: A Lightweight Ranking framework for LLM Decoding
Chenheng Zhang
Tianqi Du
Jizhe Zhang'
Mingging Xiao 1.4 1.2
Yifei Wang
Yisen Wang"
,22
Zhouchen Lin
State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University
Institute for Artificial Intelligence. Peking University MIT CSAIL, MA, USA
4 Miirosooft Research Asia
Abstract
Conventional research large language models (LLMs) has primarily focused refining output distributions, while paying less attention to the decoding process that transforms these distributions into final responses. Recent advances, such scaling the computation of inference time with reward models, have underscored the importance of decoding, but these methods often suffer from high computa- tional costs and limited applicability. In this paper, we revisit LLM generation through the lens of recommender systems, conceptualizing the decoding process analogous to the ranking stage in recommendation pipelines. From this per- spective, we observe that both traditional decoding methods and reward models exhibit clear limitations such as redundancy. Motivated by this insight, we propose Language Ranker, a novel framework that introduces a lightweight module rerank candidate responses using features extracted by the base model. Experiments across a wide range of tasks show that Language Ranker achieves performance comparable to large-scale reward models, while requiring only <0.5M additional parameters, significantly reducing the computational overhead during both train- ing and inference stages. This highlights the efficiency and effectiveness of our method, showcasing its potential to fully unlock the capabilities of LLMs. The implementation is released at https://gihhbb.com/heeeeeeeen www.///hhneenngg///annneeeeeeeee _ankerrrraerr
Introduction
Traditional research on enhancing the capabilities of large language models (LLMs) has primarily focused improving the quality of output distributions through approaches such as scaling up model sizes [1, fine-tuning for specific tasks (SFT) 2 and reinforcement learning with human feedback (RLHF) []) However, the decoding process, which converts the output distributions into final responses, has not received sufficient attention. Current decoding strategies, including top-k sampling 7, self-consistency [ and contrastive decoding (9, are largely aule-based and speciffc,,, limiting their ability to fully exploit the potential of LLMs' powerful output distributions.
is found that if oracle could select the best response from multiple samples generated by model, the performance of 7B model could even surpass that of a 70B model as the sample number increases
10 This finding highlights the tremendous potential of the decoding process in maximizing model performance. To approximate the oracle, recent studies on inference-time computing 11
13
"Equal contribution.
Conresponding authors: Zhouchen Lin (zlin@ pku.edu.ca) and Yisen Wang (yisen. wang @ pku.edu.cn).
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Decoding Strategy
Recommender System
Englinerring
No Shared Featuuee
Shared Feature Eng Ingineering
Fieeure
Cardkcate Resperons
Allee-waad Method
Pesponse
Embedding
Gandidade Embediingg
Faarkor
Heem
Learnigg-brred
LM Backbone
Feailue Enginerring
Language Ronker
Reword Model
No Shared Feature Engineerinng
Shared Feature Englisering
Instuntion
Feature
Reacome
Response
Рasporen
Learning-Oasedd
Ausillary Reward Model
LM
IM
The comparison among existing methods, recommender system and our Language Ranker. The two charts on the left highlight the limitations of existing decoding strategies and reward models, in contrast the recommender system pipeline shown in the top-right. Language Ranker addresses these limitations by incorporating feature-shared, learnable, and lightweight ranker.
Figure 1:
Table I:
Comparison of parameters and performance between the Language Ranker and reward models, using 1-8ma3.--8B-Insteee as the base model.
Training Stage
Inference Stage
Method
Performance
Trainable
GPU-Loaded
Sampling
Ranking
Language Ranker
<0.5M
<0.5M
8.2B
<0.5M
RM (Jama8B-LoRA)
176M
8.2B
8.2B
8.4B
RM (gp(2)
137M
137M
8.2B
137M
have introduced reward models select the best response. While these methods demonstrate strong performance across various tasks, the reliance on auxiliary reward models significantly increases computational and time overhead during both training and inference, thereby limiting their scalability and applicability in broader contexts.
To address these limitations, we rethink LLMs through the lens of recommender systems. As shown in Figure I each L.M can viewed as special recommender system, where the input serves the user information, and the model's role is to recommend the most appropriate response as the "item" tailored to the user's needs. Therefore, the model backbone, language head, and decoding process correspond directly to the feature engineering, retriever, and ranker in traditional recommender system [14]. When a user provides an input, the model backbone first extracts user features, represented the hidden states of the last token. The language head then generates coarse response distribution. Finally, a predefined decoding strategy samples several candidate responses from this distribution and selects the most suitable among them.
In this analogy, the limitations both existing decoding strategies and reward models become evident. As illustrated in Figure current decoding strategies are typically simple and rule-based, neglecting the crucial role of reranking model responses. Mcanwhile, reward models, though effective as rankers, introduce substantial computational overhead both in training and inference. From the perspective of recommender systems, these methods essentially redo the feature engineering for ranking from scratch, ignoring the features already extracted during the recall stage that could have been shared. This redundancy leads to significant unnecessary computations and inefficiency.
In this paper, we propose Language Ranker, inspired by recommender systems, to address aforemen- tioned shortcomings of existing methods. The Language Ranker incorporates a carefully designed lightweight ranker to rerank candidate responses generated by the base model, which constitutes the only trainable component in our framework. As illustrated in Figure the earlier layers of the base model can be viewed shared feature engineering for both the retriever and ranker, similar to

---

Condidates
AU
Jusaa 5140.
ChatBot
L
Hoad
low-resource
Listwiise
Paltihiiee
2. Get Hidden States sure for pntvuation and each canciden suposien
Math
b
API Agent
Solver
148 1.51-115.
LLM Server
Hicban
highhressurcee
Resporses
be sitond lyye
Evuluzao the candosko and bes reporos taxed
Engineering
L
(ilg lis ins decend Bilm Day atar deccca)
CodeBot
Figure 2:
The Framework of Language Ranker. The base model first generates mul- tiple candidate responses. The hidden states cor- responding the final tokens of both the in- struction and each candidate response are then extracted from predefined layer and used features, Finally, the ranker selects the most ap- propriate response based these features.
Figure 3: Personalized Language Ranker, We pair single base model with different rankers to enable personalized adaptation for diverse
needs simultaneously. The base model runs on high-resource central nodes, while rankers can be deployed on edge devices local user devices. The CPU-trainability allows each user's ranker to perform continual learning with behavioral data, paving the way for deeper personalization.
recommender systems. Once the candidate responses are sampled, the ranker utilizes the extracted features to rerank the candidates and identify the most appropriate response.
As shown in Table T by leveraging the representations of base models, method achieves per- formance comparable to that of the LlamaSB-based reward model, while requiring only <0.5M additional parameters, significantly reducing the computational overhead during both training and inference stages. Table [ independently on CPUs, demonstrating the potential to build a personalized Language Ranker. As illustrated in Figure 3 the base model can be paired with different rankers to enhance capabilities across various dimensions. The base model is expected to run on high-resource central nodes, while the rankers can be deployed on edge nodes or even on users' local devices, allowing continual learning and personalized adaptation diverse user needs.
shows that the lightweight ranker supports both training and inference
In conclusion, the main contributions of this paper are:
We reinterpret LLMs through the lens of recommender systems, revealing the limitations of existing decoding strategies and reward models while highlighting the potential of the decoding process.
We propose Language Ranker, a novel and lightweight ranking framework for LLMs that is both efficient and effective. It allows single base model to be flexibly paired with different rankers, allowing personalized adaptation to diverse user needs.
We conduct extensive experiments across diverse tasks and multiple base models, demon- strating that our framework achieves performance comparable to large-scale reward models while requiring <0.5M additional parameters, thereby substantially reducing computational overhead during both training and inference.
2 Decoding Ranking Mechanisms Design
In this section, we introduce Language Ranker, our lightweight ranking framework for LLMs inspired by recommender system. This framework addresses the limitations of existing decoding strategies and reward models by incorporating efficient and effective ranking mechanism.

---

2.1 Language Ranker
As shown in Figure 2 we employ a lightweight yet effective ranker to rerank candidate responses generated by language models. Specifically, hyperparameter is defined to select specific layer in the model, and the hidden states of this layer are used as features for the ranker. _ Before inference, the hidden states of the selected layer corresponding to the final token of the given instruction is recorded as the instruction feature, denoted as i. The model then begins the inference process, sampling K candidate responses. Once each candidate response is fully generated, the hidden state of the chosen layer corresponding to the final token is recorded as its feature. These features, representing the candidate responses, are denoted {TR (r These instruction and response features are then fed into the ranker to identify the most suitable response.
Following common practices in recommender systems 14, we design both a listwise ranker and pointwise ranker refine candidate responses. Both rankers first project the input features into low-dimensional space, compressing information while significantly reducing the parameter count for subsequent processing. They then process the projected features using their respective blocks and compute the relevance between each response and the instruction features, which is subsequently used rerank the responses.
Specifically, the listwise ranker processes all candidates simultancously, enabling direct comparisons between them:
[, 12, rk] Trans (Proj
(1,, 82, 82, ... [K] Rele (7, f2, ,rk))
As illustrated in Figure 2 after projection, the instruction feature and the response features {rx )== interact within Transformer block. Subsequently, relevance scores between the instruction and each candidate response are computed, and the candidate with the highest score selected as the final output.
The pointwise ranker, in contrast, evaluates each candidate response individually based on the given instruction feature:
E, rk] = (MLP(PP(((),MMMMPPPPPo)))))
(3)
= Rele (i,
(4)
Each projected feature is independently processed using a shared MLP block. The ranker then computes relevance score between the instruction feature and each response feature. The response with the highest relevance score is selected as the final result.
Notably, the choice of relevance function for both the listwise and pointwise rankers depends on the form of the response labels. If the labels are binary (0 or 1), cosine similarity applied. framing the task as classification problem. In contrast, if each response is assigned a specific score, a learnable relevance function is employed fit these scores:
8k = Rele (i, fz):
(5)
= W * concat (i, rk))
(6)
The Transformer block in the listwise ranker and the MLP block in the pointwise ranker can both be extended to multiple blocks. For efficiency, we use a single block in all main experiments, and the impact of the block number is analyzed in Subsection 3.2
2.2 Dataset Construction and Ranker Training
The training dataset for our ranker is constructed in a manner similar to that of reward model datasets 15]. introducing virtually additional computational or time overhead. For each task, allow the base model to perform sampling and generate 100 responses for each instruction in the training set. During this process, the corresponding instruction features and response features recorded. These features are then used to train the ranker effectively. After collecting all responses, we assign labels
'The final layer of the model backbone is often suboptimal for feature extraction; instead, layers located around 60% from the bottom of the model typically yield better representations, as shown in Subsection 3.2

---

depending on the characteristics of the task. These details will be discussed within the context of specific tasks Section
The listwise ranker processes a list of candidates simultaneously. allowing for direct comparisons among them. To prepare the training data, K candidate responses randomly sampled for each query from the previously constructed dataset. This process is repeated multiple times, and groups that do not contain both positive and negative responses are filtered out. Ultimately, V data groups per query, along with their corresponding hidden states, are collected for training, formally represented
(rlm) 11 (
(rkk) yK )
For training process, the loss function is selected based on n=1
the form of the labels. If v/ah) {0, 1), the task is framed as classification problem, where s(") is computed using cosine similarity. We optimize the ranker using the following KL divergence loss:
(nc)
Vi
(n)
(m)
exp(se
(m)
9
K
K
Σ
(m)
exp(s (n))
k=1
N
Tlist
1
DKL
(
(8)
N
If Vi (n) I. the task is treated as regression problem, where si (n) is computed by the learnable relevance function previously introduced. We apply the mean squared error (MSE) loss:
K
Thiel
K
k
3k (m)
(9)
The pointwise ranker is much simpler than the listwise ranker. For each query, it indepen- dently pairs each candidate response with its corresponding instruction, formally represented as: (ú, (1(*), ( >>> p=1² The choice of loss function also depends on the form of the labels. Following the discussion for the listwise ranker, summarize the corresponding loss functions for the two forms of labels below:
exp(s(0))
(n)
Pk
1 + exp(s(n)))
(10)
1 N
Jpoint
(m) log + +(1 - (1) log() - (n))
(11)
N
(n) - yy()) )m
Jpoint
(12)
Experiments
In this section, conduct experiments on three representative LLM tasks: mathematics, coding, and function calling. We further perform detailed analyses and ablation studies, as well as evaluate the transferability of our method. To additionally demonstrate the generality of our approach, we assess its general instruction-folloingg capability, as described in Appendix B. The full set of hyperparameters is provided in Appendix
3.1 Main Experiments
Baselines For each task, we train two reward models of different scales for comparison. The first is based on GPT-2 [16], relatively small model that nevertheless has over 100 times more parameters than our ranker. The second reward model is trained from the corresponding base model using LoRA. Although the number of trainable parameters is similar to GPT-2, the full model must be loaded into GPU memory for both training and inference, resulting substantially greater computational overhead. In addition, we use the first sampled response from the base model simple baseline
5

---

Table 2: The total performance across the three tasks compares our methods with reward models and
common decoding strategies. The RM means reward model. In the Parameter column, we report the number of trainable parameters for each method. For reward models trained with LoRA, we additionally report the number of GPU-loaded parameters.
Method
Parameter
MATH
MBPP
xLAM
Llama3. 1--BaB...-BB.... 1-8B-Instruce
ListRanker (ours)
0.30M
46.3
54.5
32.6
PointRanker (ours)
0.28M
45.8
55.1
30.4
RM (gpt2)
137M
42.9
47.7
29.4
RM (Llama8B)
176M / 8.2B
45.1
52.9
32.8
Beam Search
40.3
42.3
27.0
First Sample
25.1
41.9
10.6
Srren2.5-7B-Insse
ListRanker (ours)
0.27M
74.8
63.2
71.0
PointRanker (ours)
0.25M
75.2
62.7
70.4
RM (gpt2)
137M
71.9
60.2
65.4
RM (Qwen7B)
161M 7.6B
74.6
62.9
70.2
Beam Search
67.9
62.2
68.0
First Sample
68.7
60.6
57.0
Qwen2.5-322 -2s--uutt
ListRanker (ours)
0.36M
81.1
74.2
72.8
PointRanker (ours)
0.34M
81.3
74.6
72.4
RM (gpt2)
137M
78.8
70.6
68.8
RM (Qwen32B)
537M / 32.8B
80.7
75.9
73.6
Beam Search
78.1
71.4
70.6
First Sample
75.9
68.2
65.2
and adopt deterministic beam search as a representative decoding strategy. A detailed analysis and comparison with other decoding strategies are provided in Appendix B.3
Ranker Settings In all experiments, the rankers implemented using either single Transformer block or a single MLP block, and they operate on features extracted from approximately the bottom 60% of the base model's layers. During both training and evaluation, each data group consists of 10 candidate responses. The ranker is trained to classify each response as correct or incorrect, formulating the task as binary classification problem. Cosine similarity is used compute the final logits, and the training objective is defined by the classification loss Jels as specified in Equations or
Models We evaluate our method on 1-AMAA..--B-InSSE Swen2.5-32B-Insttee Instructt [18] demonstrate its generality across different model architectures and scales. To further validate our approach on a broader range of models, we also conduct experiments Gemma3-4B-it [19), presented in Appendix B.2 To ensure fairness, all models evaluated under the zero-shot setting, with prompts detailed in Appendix D
17 - Instrucct,, and
Datasets For the mathematics task, we use the MATH dataset 20 which contains 12,500 competition-level problems spanning seven topics and five difficulty levels. To ensure coverage while maintaining efficiency, different topics and difficulty levels. For the coding task, we use the complete MBPP dataset (2I), which consists of short Python programming problems, 374 for training and 500 for testing, each paired with test cases to evaluate the correctness of the generated solutions. For the function calling task, adopt the function calling problems and answers. We randomly sample 1,500 more challenging problems with more than three APIs, and split them into 1,000 training and 500 testing examples.
uniformly sample 1,000 problems each for training and testing across
function-callingg calling- 160ling--60k dataset (22), which comprises 60,000 high-quality
Metrics For the mathematics, coding, and function calling tasks, we design task-specific labeling criteria to ensure consistent and fair evaluation. For the mathematics task, extract the final answer

---

MATT
933
NEST
54
$2.5
50.5
00.3
20
44.
saoettc Rarker
Labela Ranbe
Snnta Rarder
Purroose Rarhot
423
Ponterse Nartec
50
Namber Cancatalas
Number Candaaaiis
Candiaata Number
(a) Mathematics Task
(b) Coding Task
(c) Function-Calle Calling Task
Figure 4: The performance of the Language Ranker built on Llama3.1 improves consistently across all three tasks as the number of candidate responses increases.
Table 3:
The total training time on the MBPP dataset for both CPU and GPU settings, includ- ing data loading stages.
Table 4: Comparison of performance and param- eter on MATH under different ranker architecture ablations, using Llama3. 1-B as the base model.
Method
CPU
A100
Ranker Setting
Accuracy Parameter
Listwise Ranker
67s
44s
Listwise Ranker
46.3
0.30M
Pointwise Ranker
71s
42s
remove projection
46.4
192M
remove instruction
44.2
0.30M
RM (gpt2)
>Ih
72s
RM (Llama8b)
too long
24min
Pointwise Ranker
45.8
0,28M
remove projection
46.0
128M
remove instruction
44.1
0.28M
remove MLP block
42.5
0.25M
from each response and compare with the ground-truth answer. A response is labeled as correct the extracted answer matches exactly; responses that are incorrect or from which the answer cannot be extracted (c.g., due to formatting errors) are labeled as incorrect. For the coding task, we extract the generated code segment and execute it on set of predefined test cases. A response is labeled as correct only all test cases pass; otherwise, it is labeled as incorrect. For the function calling task, we extract the function calls from each response and use regular expressions to parse the function names and parameter values. The response is labeled as correct only if both the function and all arguments match the moound-truth function calls.
Results Both the listwise and pointwise rankers significantly improve model performance across all tasks. Our lightweight method consistently outperforms the reward model (gpt2), despite being over 100 times smaller scale, and even achieves performance comparable to reward models trained from the base model. Specifically, for Llama3. 1-8B Instruct, approach improves over the first-sample baseline by more than 20% on MATH and 12% on MBPP, substantially outperforming both larger reward models. On the function calling task, it trails the Llama&B-based reward model by only 0.2%. For Qwen2.5-75 7-- Instruct, the Language Ranker outperforms all baselines. For Qwen2.5-32B- Instruct, rankers with fewer than 0.5M parameters achieve performance comparable to 32B-scale reward models, demonstrating remarkable potential. This suggests that rankers can adapt to even larger base models, the extracted features they rely on become inereasingly expressive-alloos allowing them to stand on the shoulders of giants.
3.2 Analysis and Ablation Study
Ranker Scaling Law Figure 4 illustrates the relationship between the performance the Language Ranker and the number of candidate responses provided to the ranker. We found that performance improves across diverse tasks as the number of candidates increases, demonstrating the Ranker Scaling Law. A key open problem in current research is how to effectively scale inference-time computation large language models to enhance their performance. Most existing work has focused on optimizing inference configurations during the sampling stage, often relying traditional reward models for
7

---

Table Performance comparison across different ranker configurations in MATH for Llama3. 1--BB
Instruct.
Hidden States Layer
Block Number
Ranker Type
0.1
0.3
0.6
1.0
2
3
Llama3.
Listwise Ranker
41.2
44.6
46.3
44.9
46.3
46.7
46.6
46.9
Pointwise Ranker
40.6
43.6
45.8
44.0
45.8
46.2
46.4
46.3
Qwen2.5-7B
Instructt
Listwise Ranker
70.6
72.7
74.8
73.6
74.8
74.9
75.2
75.4
Pointwise Ranker
71.4
73.1
75.2
73.9
75.2
75.1
75.6
75.5
Table 6: Performance across different hyperparameter configurations for Llama3.1 1-8B-Instruct the MATH dataset. Green indicates the best resuls, while red indicates the worst results.
Optimizer
SGD
Adam W
Optimizer
AdamW
Learning Rate
0.05 0.1 0.5 1.0
le-5 le-4
Learning Rate
5e-5 le-4 2e-4 5e-4
Batch Size=256
46.2
46.1
45.8
45.7
46.2 46.1
Batch Size=64
41.2 42.2 45.1 43.6
Batch Size= 1024
46.1 45,9 46.3 45.9
45.9 46.1
Batch Size=256
43.2 41.9 42.8 44.7
(a) Listwise Ranker
(b) Reward Model (Llama8B)
In contrast, our approach focuses on the ranking stage, providing an efficient, effective, and scalable alternative. This distinction underscores the complementarity between our method and sampling-bast techniques, suggesting that they can be integrated to further improve model performance.
response reranking 112 [ 23
CPU Trainability As shown in Table 3 the lightweight rankers can be efficiently run on CPUs, demonstrating the potential of constructing a personalized Language Ranker. As illustrated in Table 3 the base model can be paired with different rankers to enhance capabilities across various dimensions. The hidden states of the final token ( 8KB) are compact enough to be transmitted over the internet. In this setup, the base model runs high-resource central nodes, while rankers can be deployed on edge devices or even local user devices, enabling flexible adaptation to diverse user needs. Moreover, the CPU-trainability allows each user's ranker to perform continual learning with behavioral data, paving the way for deeper personalization.
Ablation Study To better understand the design of our ranker, we conduct ablation studies on all key components of both the listwise and pointwise architectures. As shown in Table the projection layer compresses sigh-dimensional features into dower-dimensoonal space, playing a critical role in keeping the ranker lightweight. Removing this layer results in much larger ranker with minimal performance gain. Additionally, we examine the role of the instruction feature, which is used to compute relevance scores with each candidate for ranking. Replacing this feature with a learnable vector leads to noticeable drop in performance, underscoring the its importance as form of user information, consistent with our perspective of recommender system.
Ranker Configurations The last layer of the model backbone is often not the best choice for providing features. Since the backbone trained for next-token prediction, the final layers tend to overfit to this specific task. In contrast, intermediate layers typically provide more comprehensive representations of the preceding context, making them better suited for capturing the overall features required for ranking (24). As shown in the Hidden States Layer part of Table 5] the most effective features for the rankers are extracted from the 60% from the bottom of the model layers. As shown in the Block Number part of Table [ increasing the ranker's scale has only a marginal impact. Since the base model has already extracted high-quality features, the ranker's 's task remains relatively simple, making further scaling unnecessary.
Hyperparameter Robustness Table 6 presents the hyperparameter robustness of our method, particularly in comparison with reward models. We uniformly sample a series of hyperparameter con- figurations for both approaches. For the listwise ranker, the accuracy range across 12 configurations only 0.6%, with the best and worst performances at 46.3% and 45. 77,,, respectively. In contrast, the
8

---

Table The Transfer Performance Ranker Trained on Single Task. All results are tested with Llama3. -B--Inttrct on the MATH dataset. For task types, we use abbreviations in the table due to space constraints: Prealgebra (PA), Algebra (A), Number Theory (NT), Counting and Probability
(CP), Geometry (G), Intermediate Algebra (IA), Precalculus (PC).
Target Task
Source Task
PA
A
NT
CP
G
IA
PC
67.5
61.3
38.2
43.7
33.4
21.9
32.2
PA
0.0
-0.4
-0.5
-0,2
0.0
-0.8
-1.7
66.0
61.7
38.5
42.0
34.7
22.3
31.1
A
-1.5
0.0
-0,2
-1.9
-4.0
-0,4
-28
64.9
60.2
38.7
41.4
35.7
20.7
31.0
NT
-2.6
-1.5
0.0
-2.5
-2.7
-2.0
-2.9
66.5
61.3
37.4
43.9
35.3
22.2
32.6
CP
-1.0
-0.4
-1.3
0.0
-2.1
-0.5
-1.3
66.0
60.5
36.7
41.4
37.4
22.4
31.1
G
-1.5
-1.2
-1.8
-2.5
0.0
-0.3
-2.8
64.3
58.8
35.7
38.6
32,4
22.7
31.3
IA
-3.2
-2.9
-2.8
-5.3
-5.0
0.0
-2.6
63.0
59.1
35.6
41.1
34.5
22.3
33.9
PC
-4.5
-2.6
-2.9
-2.9
-2.9
-0,4
0.0
reward model is much more sensitive to hyperparameters, showing larger variation of 3.9% across configurations (best: 45.1% %, worst: 41.2%).
3.3 Cross-Domain and Cross-Task Transfer
To assess the generalization capability of the Language Ranker, we conduct transfer experiments on the MATH dataset, which contains seven distinct problem types. We train the ranker on a single problem type and evaluate its generalization to the remaining domains (see Table (7). The results show that rankers trained on any individual domain maintain robust performance across all others. Remarkably, in some cases, the transfer performance approaches that of domain-specific rankers, demonstrating the system's adaptability to unseen domains.
To further evaluate its transferability across broader task domains, we perform cross-task transfer experiments, following similar experimental setup. Specifically, we train the ranker on a single task type (math code) and evaluate its generalization ability on the other task. Table shows that rankers trained on any individual task maintain robust performance on the other one. Notably, the transfer performance even surpasses in-domain performance of GPT-2-based reward model (43.4 vs. 42.9 on MATH and 51.2 vs. 47.7 on MBPP), further highlighting our adaptability to unseen domains.
Moreover, unlike recent general reward models, our ranker is highly efficient and easy to train. Its lightweight design allows a single base model be paired with multiple task-specific rankers, enabling flexible deployment with minimal overhead. This makes the proposed Language Ranker framework practical and scalable solution for adapting diverse downstream tasks under real-world resource constraints.
4 Related Work
Decoding Methods A variety rule-based decoding methods have been proposed to improve language model performance, including top-k sampling 6 7 temperature- sampling 225 and nucleus sampling [26]. Beyond these, more refined algorithms have been developed for specific tasks. Self-consistency has been introduced as method to improve Chain-of-Thougee (CoT) reasoning by

---

Table 8: Cross-task generalization between math and code tasks. Each ranker is trained on one task and evaluated on both assess transferability.
Method
To MATH
To MBPP
Ranker From MATH
46.3
51.2 (-3.3)
Ranker From MBPP
43.4 (-2.9)
54,5
RM (gpt2)
42.9 (-3.4)
47.7 (-6.8)
majority voting 8 Other approaches leverage auxiliary model either selected or fine-tuned to assist in generating responses that better align with desired requirements [28] [9]. However, these methods are typically rule-based task-specific, which limits both their performance ceiling and application scope. We propose more general ranking framework to address these limitations.
Reward Models Reward models have been widely adopted for the enhancements of LLMs. They serve as learned proxies for human preferences in RLHF [30] 31, and have also been applied to guide multi-step reasoning processes [32 B3] While effective in a range of scenarios, reward models typically introduce significant computational overhead, limiting their practical deployment in real- world systems. To address this issue, some efforts aim to teach models to act as self-critics [34] (5)) but their performance remains suboptimal, simplify reward model training [36], but it primarily targets RLHF settings and still requires additional forward pass during both training and inference to extract embeddings. Another line of work proposes scoring all candidate tokens simultaneously to reduce call frequency, but relying more heavily on large-scale models [37].
embedding-
alternative has been proposed to
Inference-Tinee Computing Recently, there has been growing interest in scaling inference-time computation to improve the performance of LLMs. Most existing approaches focus on optimizing configurations at the sampling stage, often relying on traditional reward models to evaluate and rerank
generated responses [3. 23 15 39,, or specific design on extending CoT length 40 42 In
contrast, our method shifts the focus to the ranking stage and introduces lightweight architecture that operates directly on features already extracted by the base model. This design eliminates the need for additional forward passes, providing scalable, efficient, and effective alternative. We believe our approach complements sampling-bant strategies and can be combined with them to further improve model performance.
5 Conclusion and Discussion
In this paper, we have introduced the Language Ranker, a novel lightweight ranking framework for enhancing the LLMs. By rethinking LLMs through the lens of recommender systems, we identified the limitations of existing decoding strategies and reward models, particularly their limitations on rule-based methods and computationally expensive reranking. Our approach integrates lightweight ranker that leverages features extracted by the base model, enabling more efficient, effective and scalable reranking with minimal computational overhead.
Moreover, the separability of the ranker from the base model further enhances the flexibility of our approach. This decoupling allows for the independent optimization of the ranker, enabling a base model to be paired with multiple rankers that enhance different aspects of its capabilities, making adaptable various domains. We hope that approach can offer perspectives for the future of language model inference and contribute to the development of more resource-efficient Al systems.
Acknowledge
Zhouchen Lin supported by the NSF China (No. 62276004), the Beijing Major Science and Technology Project under Contract no. Z251 100008425006 and the State Key Laboratory of General Artificial Intelligence. Yisen Wang supported by Beijing Natural Science Foundation (L257007), Beijing Major Science and Technology Project under Contract no. 72511000000 125100000420066 National Natural Science Foundation of China (92370129, 62376010), Beijing Nova Program (20230484344, 20240484642), and State Key Laboratory of General Artificial Intelligence.
10

---

References
[1] Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. In Advances in Neural Information Processing Systems, 2022.
[2] Ziqi Gao, Qichao Wang, Aochuan Chen, Zijing Liu, Bingzhe Wu, Liang Chen, and Jia Li. Parameter-efficient fine-tuning with discrete fourier transform. arXiv preprint arXiv:2405.03003 2024.
[3] Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D. Lee, Danqi Chen, and Sanjeev Arora. Fine-tuning language models with just forward passes. In Advances Neural Information Processing Systems, 2023.
[4] Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. In Advances in Neural Information Processing Systems, 2023.
[5] Long Ouyang, Jeff Wu, Xu Jiang. Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, 2022.
[6] Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. In Proceed- ings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), 2018.
[7] Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, and Yejin Choi. Learning to write with cooperative discriminators. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2018
[8] Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations, 2023.
[9] Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023.
[10] Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoe V. Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787 2024.
[11] Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute opti- mally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.033: 03314, 2024.
[12] Yangzhen Wu, Zhiqing Sun. Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for prolemm-olvvigg with language models. arXiv preprint arXiv:2408.000 00724, 2024.
[13] Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. Rewarding progress: Scaling automated process verifiers for Im reasoning. arXiv preprint arXiv:24 20.0::244008///// 10.08/46, 2024.
[14] Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys, 2020.
[15] Luong Trung Xinbo Zhang, Zhanming Jie, Peng Sun, Xiaoran Jin, and Hang Li. Reft: Reason- ing with reinforced fine-tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), 2024.
11

---

[16] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019.
[17] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The Ilama 3 herd of models. arXiv preprint arXiv:2407.2178. 2024.
[18] An Yang. Baosong Yang. Beichen Zhang. Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. 2024.
[19] Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295, 2024.
[20] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. In Advances in Neural Information Processing Systems, 2021.
[21] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang. Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv: 108.0732,, 2021.
[22] Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, et al. APIGen: Automated pipeline for generating verifiable and diverse function-calling datasets, arXiv preprint urXiv:24066 18518, 2024.
[23] Kexun Zhang, Shang Zhou, Danqing Wang. William Yang Wang, and Lei Li. Scaling Ilm inference efficiently with optimized sample compute allocation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7959-7973, 2025.
[24] Oscar Skean, Md Rifat Arefin, Yann LeCun, and Ravid Shwartz-Ziv. Does representation matter? exploring intermediate layers in large language models. arXiv preprint arXiv:2412.09956 2024.
[25] Jessica Fieler and Yoav Goldberg. Controlling linguistic style aspects in neural language generation. In Proceedings of the Workshop on Stylistic Variation, 2017.
[26] Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In International Conference on Learning Representations, 2020.
[27] Xuezhi Wang and Denny Zhou, Chain-of-thought reasoning without prompting. arXiv preprint arXiv:2402. 10200, 2024.
[28] Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, and Radha Pooven- dran. SafeDecoding: Defending against jailbreak attacks via safety-aware decoding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024.
[29] Yue Zhang, Leyang Cui, Wei Bi, and Shuming Shi. Alleviating hallucinations of large language models through induced hallucinations. arXiv preprint arXiv:2312.15710, 2024
[30] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models follow instructions with human feedback. In Advances in neural information processing systems, 2022.
[31] Yifan Zhang, Ge Zhang, Yue Wu, Kangping Xu, and Quanquan Gu. General preference modeling with preference representations for aligning language models. arXiv preprint arXiv:2400.02197, 2024.
[32] Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. rstar-math: Small Imss can master math reasoning with self-evolved deep thinking. arXiv preprint 250.0
12

---

[33] Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, al. Process reinforcement through implicit rewards. arXiv preprini arXiv:2502.01450, 2025.
[34] Yuancheng Xu, Udari Madhushani Sehwag, Alec Koppel, Sicheng Zhu, Bang An, Furong Huang, and Sumitra Ganesh. Genarm: Reward guided generation with autoregressive reward model for test-time alignment. arXiv preprint arXiv:2410.08/99 2024.
[35] Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction. arXiv preprint arXiv:2408. 15240, 2024.
[36] Hao Sun, Yunyi Shen, Tean-Francois Ton, and Mihaela van der Schaar. Reusing embeddings: Reproducible reward model research in large language model alignment without gpus. arXiv preprint arX45 2025.
[37] Ahmad Rashid, Ruotian Wu, Rongqi Fan, Hongliang Li, Agustinus Kristiadi, and Pascal Poupart. Towards cost-effective reward guided text generation. arXiv preprint arXiv:2502.04517. 2025.
[38] Haoxiang Wang. Wei Xiong, Tengyang Xie, Han Zhao, and Tong Zhang. Interpretable prefer- ences via multi-bjeective reward modeling and mixture-of-experts. In Findings of the Associa- fion for Computational Linguistics: EMNLP 2024, 2024.
[39] Tianqi Du, Zeming Wei, Quan Chen, Chenheng Zhang, and Yisen Wang. Advancing Ilm safe alignment with safety representation ranking. arXiv preprint arXiv:2505. 15710, 2025.
[40] Yuntian Deng, Yejin Choi, and Stuart Shieber, From explicit cot to implicit cot: Learning to internalize cot step by step. arXiv preprint arXiv:2405. 2024.
[41] Yingcong Li, Kartik Sreenivasan, Angeliki Giannou, Dimitris Papailiopoulos, and Samet Oymak. Dissecting chain-of-thought: A study on compositional in-context learning of mips. arXiv preprint arXiv:2305. 18869, 2023.
[42] Tianqi Du, Haotian Huang, Yifei Wang, and Yisen Wang. Long-short alignment for effective long-context modeling in lims. In Proceedings of the 52nd International Conference on Machine Learning, 2025.
[43] Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free dolly: Introducing the world's first truly open instruction-tuned 1lm, 2023.
[44] Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori Hashimoto. Alpacafarm: simulation framework for methods that learn from human feedback. In Advances in Neural Information Processing Systems, 2024.
[45] Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. Length-controde alpacaeval: A simple way debias automatic evaluators. arXiv preprint arXiv:2404.04477. 2024.
[46] Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. In The Tweifth International Conference on Learning Representations, 2023.
13

---

A Hyperparameter settings
In sampling process, we temperature 1.5 for diverse responses and max new tokens new_tokens as 1024 to make sure completed answers. We sample 100 responses for each problem. During training, we perform grid search over the parameter ranges specified in Table 9
Table 9: The hyperparameter list
Hyperparameter
Value
Sampling
Sampling Temperature
1.5 1024
Sampling Max New Tokens
Ranker Training
Batch Size
[256, 1024]
Epoch
Optimizer
[SGD, AdamW] [0.05, 0.1, 0.5, 1.0]
SGD LR
SGD Momentum
[0.0, 0.9]
AdamW LR
[1e-5, le-4]
AdamW Betas
(0.9, 0.999) Ic-4 [Constant, Cosine Decay] 64
Weight Decay
LR Schedule
Projection Dimension
Reward Model Training
Batch Size
[64, 256]
Epoch
Optimizer
AdamW
AdamW LR
[5e-5, 5e-4]
AdamW W Betas
(0.9, 0.999)
Weight Decay
1e-4 [Constant, Cosine, Decay] 64 [64, 128]
LR Schedule
LoRA r
LoRA alpha
B Additional Experimental Results
B.1 Enstruction-Folloviig Task
Our framework performs well on the three tasks presented Section 3 To further demonstrate its general applicability, we also evaluate it on mixed instruction-following task.
Considering that instruct models specifically fine-tuned for cnttruction-folloiinn tasks, we conduct evaluations on this task with Llama3. 1-----8BBass [17] and Qwen2.5-7B 5-B-BBBse [18]. For
Models
fairess, all models are evaluated using zero-shot prompts, shown in Appendix D
Datasets We use the first 1,000 queries from the Databricks- Saabricks-Dolly-1544 dataset
43 for training.
For evaluation, we adopt AlpacaEval 44. widely recognized benchmark for assessing instruction- following capabilities in LLMs. consists diverse test queries sourced from Self-Instruct, OASST, Anthropic's s Helpful dataset, Vicuna, and Koala.
Unlike tasks such mathematics, instruction-following lacks objective ground-truth answers, making rule-based evaluation infeasible. To address this, we follow the AlpacaFarm 44 method and prompt DeepSeek- V3 to simulate human judgment by assigning scores from 5 to all sampled responses. These responses are then used to train both our ranker and reward models. Win Rate metric [45), relative measure based on reference model. For each base model, we use its corresponding instruct variant as the reference for example, Ilama3. 1-8B- Instructt for I. Llama3.1-
Metrics
For evaluation, we adopt the official AlaacaEval evaluator to compute the Length-Controlle
8B-Basc.
14

---

Table 10: The evaluation on general instruction-follonee tasks compares our method against reward models and common decoding strategies. We conduct experiments on Llama3. 1--Ba3.--B-BBar and Qwen2.5-7B-Base reporting the win rate using the corresponding Instruct model as the reference. RM (base) refers to reward models trained from the respective base model.
Method
Parameter
Llama3.1 1amma.1--B-Ball 1-8B-Base
Qwen2.5-7B-Base
ListRanker (ours)
<0.3M
30.7
46.3
PointRanker (ours)
<0.3M
27.1
45.8
RM (gpt2)
137M
27.1
42.9
RM (base)
-1 1770//8B
31.6
45.3
First Sample
19.0
25.1
Beam Search
20.4
40.3
Table 11: Comparison between methods and reward models math and code tasks. The RM denotes reward model. The Parameter column reports the number of trainable parameters for each method. For reward models trained with LoRA, we also report the number of GPU-loaded parameters.
Method
Parameter
MATH
MBPP
Gemma3 4B-it
ListRanker (ours)
0.20M
72.2
51.4
PointRanker (ours)
0.19M
72.4
51.7
RM (gpt2)
137M
69.2
50.3
RM (Gemma3)
161M 7.6B
69.1
50.9
Beam Search
67.4
49.2
First Sample
63.5
48.8
As shown in Table the performance of our methods far exceeds that of vanilla decoding strategies and is comparable to the reward models trained from base models. Notably, with the assistance of 0.3M-level ranker, Owen2.5-7B-BB. achieves 46.3% win rate compared to Qwen2.5- 7B-Instruct, which has undergone extensive fine-tuning on various instruction-following tasks.
Results
B.2 Experimental Results on Gemma3-4B-it
We conduct experiments across wide range of tasks using both 7B and 32B base models, covering two prevalent architectures: LLaMA and Qwen. To further validate the generality of our approach, we additionally include results on Gemma3-4B-in [19]. thereby expanding the evaluation across both model scales and backbone architectures. The results for the math and code tasks are presented in Table
Consistent with the findings in Section 3 both the listwise and pointwise rankers substantially improve performance the math and code tasks with Gemma3-4B-it. These experiments further confirm the effectiveness and robustness our method across different model sizes and architectures.
B.3 Detailed Comparison with Existing Decoding Methods
We clarify that decoding methods generally fall into two broad categories: reranking- based methods, which operate on sampled responses as our method does, and (ii) methods that modify the model's output probability distribution during generation.
The first category often relies on auxiliary reward models like our baselines, or is task-specific. For example, self consistency [8] improves reasoning performance by aggregating multiple sampled answers via majority voting. This technique particularly effective in tasks like math, where the final answer is often a short, well-defined value-making consensus across candidates meaningful.
However, tasks such code generation, function calling, or even general cotrucction-followinn the outputs tend to be long, diverse, and semantically equivalent in multiple ways. In these settings,
15

---

Table 12: Comparison between our ListRanker and self consistency methods across different tasks. Self consistency improves reasoning tasks like math but performs weakly on code, function calling. and custruction-followee tasks.
Method
MATH
MBPP
xLAM
AlpacaEval
ListRanker (ours)
46.3
54.5
32.6
30.7
Cel--Consistency
44.9
41.9
24.6
20,4
First Sample
25.1
41.9
10.6
20.4
candidate responses often differ significantly in surface form, making majority voting unreliable and diminishing the effectiveness self consistency. As shown in Table [2, self consistency indeed improves performance on MATH, but shows much weaker performance on function calling, and negligible gains on code and prrrucction-ollowing tasks.
The second category includes methods such as Contrastive Decoding 9 and DoLa 46). which refine the model's output distribution during generation. These approaches orthogonal to ours, which focuses on post-sampling ranking. They are complementary to our method and baselines, and can be integrated independently; therefore, we do not include them direct comparisons in our evaluation.
C Limitations
The proposed ranker demonstrates strong effectiveness, efficiency, and transferability. However, it operates on the hidden states of the base model, requiring access to the final-token representations during inference. This requirement introduces no computational overhead theoretically, and storing the representation of only the final token incurs minimal memory usage. Nevertheless, fully supported by widely used inference frameworks such vLLM. We believe that, with the rapid progress of representation-bat based methods, this limitation will be gradually alleviated the near future.
not yet
16

---

D Prompts for Each Task
Prompt for mathematics task
system:
You are math expert.
user:
Please solve the given math problem step by step and present the answer in the following format: "boxed(X]"" where X is the answer.
[Question]
Prompt for coding task
system:
You are an expert Python programmer.
user:
Write a Python function based on the following instructions and test example. Please ensure that the function is clearly marked with a start and end can easily extract from your output.
Instructions:
{question]
Test Example: list[]]st([]]
Please provide your code with clear start and end markers, like so:
#START OF CODE
def [function (attooon__mmm(mt))
function code
return result
#END OF CODE
Prompt for function calling task
system:
You are function-calling assistant. Your role is to complete tasks solely through correct function calls, without generating any additional text. For each task, directly output the func- tion call(s) required to complete it. If the task involves multiple steps. you may issue multiple function calls sequentially. Each function call must be formatted as a JSON object. For example: [[ "n"amee "functionA", "arguments": ("paraml": "valuel". "param2": "valuc2"] ). ("name": "functionB", "arguments":: : "param "valuel". "param2": "value2")]] The following the available functions: (function_list)
Now, use the appropriate function(s) complete the given task.
user:
[Question]
Please directly output the function call(s) solve the task without any other text.
17

---

Prompt for instmuctionfoolooiinng task
system:
You are an assistant.
user:
Below instruction that describes task, paired with an input that provides further context.
Write a response that appropriately completes the request.
[Instruction] } Begin!
Scoring criteria cestruction-follwiiin task
Review the user's question and the corresponding response using the additive 5-point scoring system described below
The user's question is between <question>and </question>The response of the Al Assistant between <response>and </response>
Points are accumulated based on the satisfaction of each criterion: Add point if the response is relevant and provides some information related to the user's inquiry, even if it is incomplete or contains some irrelevant content. Add another point if the response addresses substantial portion of the user's question, but does not completely resolve the query provide a direct answer. Award a third point the response answers the basic elements of the user's question in useful way, regardless of whether it seems to have been written by an Al Assistant or if it has elements typically found in blogs or search results. Grant fourth point if the response is clearly written from AI Assistant's 's perspective, addressing the user's 's question directly and comprehensively, and well-organized and helpful, even there is slight room for improvement clarity, conciseness focus. Bestow fifth point for a response that is impeccably tailored to the user's question by an AI Assistant, without extraneous information, reflecting expert knowledge, and demonstrating high-quality, engaging, and insightful answer. If the response repeats itself or is not concise and to the point, score the response O.
<quesioon-rromps<qqussiiioo <reppnseeresonse<rssspsss
After examining the user's instruction and the response: output the score of the evaluation using this exact format: "score: <total points>", where <total points>is between 0 and 5 Briefly justify your total score, up to 100 words.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See Abstract and Introduction, where enumerates the contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: see Appendix
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
19

---

Justification:
No theory
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have provided sufficient details and instructions for the repreduction the main experimental results.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
20

---

Answer: [No]
Justification: We promise to release the code after publication. We have provided sufficient details and instructions for the repreduction of the main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.c
public/guiidss/Cdeeeiiieee Swt.eeuuummsssoooliiiii siiclllicy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (ht
/ fttips.cc/pbbl.. http:ubllc/gguie http:/uuudde//ooeuuubbeee wwt.ssuumissioiniii ssionsolicyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: see Experiments and Appendix Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We tune the learning rate for most our experiments. Due to limited computing resources, we are not able run the experiments many times derive the error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
21

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: see Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics wtt.ap///aarriss.../ooooo///tttr
Question:
Answer: [Yes]
Justification: This paper only include experiments on public open datasets.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
No societal impact
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
22

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses such risks
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: experiment setup
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, |ppper swi thcode. com/ cona/datsets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
23

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification: We introduce new assets. However, we do not release them at submission time. We commit to releasing the codebase upon publication.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification: the paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: the paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: We only use LLM for writing and formatting purposes.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer our LLM policy httpss///eerrrrssccccoorrrrrrres for what should or should not be described.
25


Reviews:
[{"review_id": "pgchHmmh1t", "reviewer": "Reviewer_RWdC", "summary": "This paper addresses the output decoding process of large language models from a novel perspective. It introduces an additional ranking module based on Transformer architecture to replace components like the reward module in RL, aiming to enhance the quality of final model outputs. Experimental results demonstrate that the proposed method achieves comparable performance to existing approaches while significantly reducing model size.", "questions": "Please refer to the weaknesses part.", "limitations": "1. The model design and experimental setup are inconsistent with the stated motivation.\n\n2. The experimental comparisons are insufficient, with inadequate validation across different scales and backbone models.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. This paper offers a unique perspective by viewing final model output from the standpoint of recommender systems. This opens up new directions for optimizing language models from alternative angles.\n\n2. This paper is well-organized and clearly written, making it easy to follow and understand.\n\n3. Experimental results show that the proposed method achieves comparable performance with substantially reduced computational cost and model size.\n\nWeaknesses:\n1. Although the paper emphasizes the motivation of adopting a recommender system perspective, the actual implementation uses list-wise/point-wise prediction with a Transformer/MLP architecture. The authors are suggested to emphasize the task and scenario adaptation of the ranker at the end of Section 2.\n\n2. Compared to methods that use GPT-2 or similar models to compute rewards, this approach adopts a structurally simpler method. However, the paper lacks validation on complex, large-scale tasks, making it difficult to determine whether the promising results are due to the simplicity of the tasks or limited data diversity, which might allow a small model to perform well.\n\n3. The method performs well on smaller models (e.g., 7B), but its performance on larger models (e.g., 32B) is less convincing. Further experiments are needed to demonstrate the effectiveness of the approach at scale.", "was_revised": true, "final_justification": "I have revisited the paper and realized that I had misunderstood the application of the point-wise score, so I no longer have questions about W1. Additionally, I suggest placing the explanation of when to use each ranker at the end of Section 2, as this may help readers better understand the content. For W3, I agree that this approach can save resources while improving efficiency. After consideration, I will update my rating to \"4: Borderline accept\".", "rebuttal": "We sincerely appreciate your recognition of our paper’s **writing quality**, the **novelty** of the recommender-system perspective, and the **strong performance achieved under limited resource constraints**. We believe that some of the concerns raised may arise from misunderstandings, which we clarify below with further explanations and additional experimental results.\n\n---\n\n**W1.** Although the paper emphasizes the motivation of adopting a recommender system perspective, the actual implementation uses point-wise prediction with a Transformer architecture. As the authors themselves note in the experimental setup, the model essentially performs a binary classification task, functioning more like a rating predictor than a true ranker. By the way, what if two results are classified as positive?\n\nWe would like to clarify two key points: **(1)** we do **not** use only point-wise prediction with a Transformer architecture, and **(2)** our ranker does **not** essentially perform a binary classification task.\n\nAs described in Section 2 and illustrated in Figure 3, our framework includes **two distinct rankers** following common recommender system paradigms: a listwise ranker and a pointwise ranker [1-2]. Both rankers compute the similarity between the instruction feature and each candidate response feature. The **listwise ranker** processes all candidate responses jointly through a Transformer block, enabling it to model inter-response interactions and calculate similarity scores in a holistic manner. In contrast, the **pointwise ranker** computes the similarity between the instruction and each response independently via an MLP block and ranks the responses based on their predicted similarity scores. Therefore, it is inaccurate to characterize our method as solely point-wise.\n\n**During inference**, both rankers sort the candidate responses based on their similarity scores and **select the one with the highest score**. As shown in Section 2, we design task-specific training objectives to ensure that the similarity scores effectively reflect response quality. For example:\n\n- For **math**, **code**, and **tool-use** tasks, where responses can be clearly labeled as correct or incorrect, we apply a classification loss that increases the scores of correct responses and decreases those of incorrect ones.\n- For general **instruction-following tasks** (Appendix B), where no absolute correctness labels are available and only coarse-grained user ratings exist, we use a regression loss to fit the observed ratings directly.\n\nTherefore, the binary classification is merely a proxy training objective used on certain tasks to help the ranker learn meaningful similarity scores—this is a common and well-established practice in recommender systems. The concern about “two results being classified as positive” does not apply. Our system does not rely on hard classification; instead, it ranks all responses based on their real-valued similarity scores.\n\n[1] Zehlike, M., et al. Fairness in ranking, part ii: Learning-to-rank and recommender systems.\n\n[2] Liu, W., et al. Neural re-ranking in multi-stage recommender systems: A review.\n\n---\n\n**W2.** Compared to methods that use GPT-2 or similar models to compute rewards, this approach adopts a structurally simpler method. However, the paper lacks validation on complex, large-scale tasks, making it difficult to determine whether the promising results are due to the simplicity of the tasks or limited data diversity, which might allow a small model to perform well.\n\nWe respectfully disagree with this comment and believe that our evaluation setup is sufficient to support the conclusions. As noted by reviewers **L4gf** and **gdjF**, the **strong empirical results across various tasks** is one of the **strengths** of our work.\n\n**Regarding data diversity**, we evaluate four representative task categories: **math**, **code**, **tool use**, and **instruction following**. The first three reflect core LLM capabilities of broad interest [6-8], while the last assesses general instruction-following ability. In particular, the instruction-following task leverages **AlpacaEval** [1], a large-scale benchmark that aggregates diverse instructions from sources such as Self-Instruct [2], OASST, Anthropic’s Helpful dataset [3], Vicuna [4], and Koala [5], offering a comprehensive and challenging test of general LLM capabilities.\n\n**Regarding benchmark complexity and credibility**, all benchmarks used in our experiments are widely adopted and authoritative within their respective domains:\n\n- **MATH** and **MBPP** are standard benchmarks for mathematical reasoning and code generation. They are featured in nearly all major LLM technical reports [6–8].\n- **Tool use** is evaluated using **xLAM**, a representative dataset in tool-augmented language modeling that involves large-scale tool library with over 3,600 tools [9].\n- **Instruction following** is evaluated using **AlpacaEval**, which is specifically designed to test general-purpose language understanding and generation, and is also frequently reported in major LLM technical reports [6].\n\nThese benchmarks provide a broad and credible testbed for evaluating models across both narrow and general tasks, and we believe they sufficiently validate the effectiveness of our approach.\n\nThe strong performance of our ranker with only <5M parameters across diverse and challenging tasks is not due to task simplicity, but rather a result of its efficient design: it effectively leverages the rich features already extracted by the base models, enabling effective ranking with minimal additional overhead. \n\n[1] Dubois, Y., et al. AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback.\n\n[2] Wang, Y., et al. Self-instruct: Aligning language models with self-generated instructions.\n\n[3] Bai, Y., et al. Training a helpful and harmless assistant with reinforcement learning from human feedback.\n\n[4] Chiang, W., et al. Vicuna: An open-source chatbot impressing gpt-4 with 90% chatgpt quality.\n\n[5] Geng, X., et al. Koala: A dialogue model for academic research, March.\n\n[6] DeepSeek-AI. DeepSeek-V3 Technical Report.\n\n[7] Gemini Team, Google. Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities.\n\n[8] Qwen Team, Qwen3 Technical Report.\n\n[9] Liu, Z., et al. APIGen: Automated PIpeline for Generating Verifiable and Diverse Function-Calling Datasets.\n\n---\n\n**W3.** The method performs well on smaller models (e.g., 7B), but its performance on larger models (e.g., 32B) is less convincing. Further experiments are needed to demonstrate the effectiveness of the approach at scale.\n\nWe would like to clarify that the performance of our method on the 32B model is also strong, for the following reasons:\n\n1. **The average accuracy is competible to that of a trained 32B reward model (76.1% vs. 76.7%)**, demonstrating that our approach remains effective even at large scale.\n2. **Our method introduces only ~0.35M additional parameters during both training and inference.** The ranker is lightweight enough to be trained and deployed even on CPU. In contrast, using a separate 32B reward model requires training an additional large model and computing reward scores for each response at inference stage—resulting in significant time and computational overhead that limits practical usability.\n\nThe strength of our approach lies not only in its **effectiveness**, but also in its **efficiency**. From 7B to 32B base models, our method achieves consistently strong performance with similarly lightweight rankers (0.30M → 0.35M). This demonstrates its scalability and general applicability across model sizes.\n\nMoreover, the low overhead of our ranker brings many benefits. For example, a single base model can be paired with different rankers to enable personalized adaptation for diverse user needs simultaneously, with minimal additional overhead.\n\n---\n\n**L1.** The model design and experimental setup are inconsistent with the stated motivation.\n\nWe would like to clarify that our model design and experimental setup are aligned with the stated motivation. This concern may stem from a misunderstanding of our approach, which we have clarified in detail in our **response to W1.** \n\n---\n\n**L2.** The experimental comparisons are insufficient, with inadequate validation across different scales and backbone models.\n\nThank you for your suggestion. We have conducted experiments across a wide range of tasks using both 7B and 32B base models, covering two prevalent architectures: LLaMA and Qwen. To further validate the generality of our method, we additionally include results on **Gemma3-4B-it**, thereby expanding the coverage in both model scale and backbone diversity. The results on the MATH and the CODE task are shown in the table below.\n\n| Method | Parameter | MATH | CODE |\n| --- | --- | --- | --- |\n| ListRanker (ours) | **0.20M** | 72.2 | 51.4 |\n| PointRanker (ours) | **0.19M** | **72.4** | **51.7** |\n| RM (gpt2) | 137M | 69.2 | 50.3 |\n| RM (Qwen7B) | 161M/7.6B | 69.1 | 50.9 |\n| Beam Search | — | 67.4 | 49.2 |\n| First Sample | — | 63.5 | 48.8 |\n\nSimilar to the results in our paper, both the listwise and the pointwise rankers significantly improve model performance on both the MATH task and the CODE task with Gemma3-4B-it. These experiments furthur prove the effectiveness of our proposed ranker across different scales and backbone models."}, {"review_id": "XA7D8QNpzS", "reviewer": "Reviewer_L4gf", "summary": "This paper introduces Language System, a lightweight ranking framework inspired by recommender systems to improve the decoding phase of large language models. The authors propose to treat the decoding process as a ranking problem, akin to recommender system pipelines, and design a learnable, compact ranker to rerank sampled responses using features from intermediate LLM layers. The method requires <0.5M parameters and eliminates the need for expensive reward model inference. Experiments across math, coding, function-calling, and instruction-following tasks demonstrate that Language System achieves comparable or superior performance to much larger reward model-based approaches.", "questions": "Q1: How would the performance be affected if only 10–20 candidate responses were sampled instead of 100? Can the ranker still outperform reward models under realistic latency budgets?\n\nQ2: The paper mentions CPU trainability. Are there any preliminary results or benchmarks on on-device adaptation?\n\nQ3: Would it be feasible to integrate sampling and ranking into a unified training loop, where the ranker could guide sampling directly (e.g., via rejection sampling or contrastive decoding)? Or it is possible that such \"ranker\" is also beneficial in pretraining data selection?", "limitations": "The method still requires storing and retrieving large intermediate hidden states, which incurs I/O overhead. While the ranker is light, the pipeline depends on sampling a large number of outputs (K=100), which could challenge real-time or low-resource applications.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strength.\n\n- **Novel perspective.**  The paper offers a fresh conceptualization of LLM decoding as a recommendation-ranking problem, which is both intuitive and practical.\n- **Efficient architecture.** The proposed rankers (listwise and pointwise) are extremely lightweight (<0.5M parameters), enabling fast and cost-effective reranking.\n- **Strong empirical results.** Across multiple tasks and model sizes (LLaMA3, Qwen2.5), the method consistently matches or outperforms larger reward models like GPT-2 and LoRA-trained variants.\n\n---\n\n## Weaknesses.\n\n- **W1: Insufficient baselines**\n\nThe paper does not compare the proposed method against several state-of-the-art open-source reward models, such as generative reward models. Additionally, inference-time scaling techniques like self-consistency are only briefly mentioned and not empirically evaluated. Without these comparisons, it is difficult to assess where the method stands relative to the current best practices in decoding optimization.\n\n- **W2: Limited model coverage.**\n\nAlthough the method is validated on LLaMA and Qwen series models, it remains unclear whether it generalizes across different architecture families (e.g., Gemma, Mistral) or performs well on smaller-scale models (<7B). This restricts the generalizability claims, especially for edge or low-resource deployment scenarios.\n\n\n- **W3: Narrow scope of transferability.**\n\nThe transferability experiments are limited to different subdomains within the math task. A more compelling demonstration would involve testing cross-domain transfer—e.g., training on math and evaluating on code—to align with recent trends in general reward models that aim to serve multiple downstream tasks without retraining.\n\n- **W4: Missing ablation on ranker scale.**\n\nThe paper does not investigate whether increasing the size or complexity of the ranker (e.g., using larger projection dimensions or more Transformer blocks) leads to consistent performance gains. Or does other architecture also effective? Understanding the scaling behavior of the ranker is essential for practitioners seeking the best trade-off between efficiency and performance.", "was_revised": true, "final_justification": "I believe the authors' rebuttal has addressed most of my concerns. After carefully reviewing the interactions between the authors and other reviewers, I think a score of 4 is fair, so I will maintain my initial score.", "rebuttal": "We sincerely appreciate your recognition of the **novelty of our approach**, as well as its **strong empirical results** and **efficient design**. The concerns raised—mainly regarding evaluation scope and generalizability, with some minor misunderstandings—are addressed below through additional experiments and clarifications.\n\n---\n\n**W1.** **Insufficient baselines.**\n\nThank you for your constructive suggestion. We address the concerns below with further experiments and analysis.\n\n**Generative Reward Model (GRM)**\n\nWe include additional results on **LLaMA3.1–8B-Instruct** to compare our method with strong GRMs. GRM performance remains heavily constrained by their training data. For example, on the **math** task—where **RM-R1** is enhanced via RL—it slightly outperforms our method by **1.5%**. However, on out-of-domain tasks like **tool use**, its performance drops significantly, lagging behind ours by **5.8%**.\n\nMoreover, GRMs are **computationally expensive** to train [1], and continual fine-tuning for new domains is often impractical. This makes them less flexible and scalable than our **lightweight** approach.\n\n|  | Parameter | MATH | xLAM |\n| --- | --- | --- | --- |\n| ListRanker (ours) | **0.3M** | 46.3 (-1.5) | **32.6** (+5.8) |\n| RM-R1-DS-7B | 7.6B | **47.8** | 26.8 |\n| RM-R1-Qwen-7B | 7.6B | 45.2 | 26.4 |\n| First Sample | — | 25.1 | 10.6 |\n\nIn terms of ranking-time overhead, GRMs are also nearly impractical. Take the **math** task as an example: our method introduces **negligible latency** during ranking, whereas GRMs require generating over **16K tokens per query** to compute scores—resulting in **over one minute of delay**, which is virtually unacceptable in real-world applications.\n\n|  | Parameter | Ranking Tokens | Ranking Latency |\n| --- | --- | --- | --- |\n| ListRanker (ours) | **0.3M** | — | **< 1s** |\n| RM-R1-DS-7B | 7.6B | 16.1K | > 1m |\n| RM-R1-Qwen-7B | 7.6B | 18.3K | > 1m |\n\n**Decoding Optimization**\n\nWe clarify that decoding methods generally fall into two broad categories: (i) **reranking-based methods**, which operate on sampled responses as our method does, and (ii) methods that **modify the model’s output probability distribution** during generation.\n\nThe first category often relies on **auxiliary reward models** like our baselines, or is **task-specific**. For example, **self consistency** [2] improves reasoning performance by aggregating multiple sampled answers via majority voting. This technique is particularly effective in tasks like **math**, where the final answer is often a short, well-defined value—making consensus across candidates meaningful.\n\nHowever, for tasks such as **code generation**, **tool use**, or even **general instruction-following** (see Appendix B), the outputs tend to be long, diverse, and semantically equivalent in multiple ways. In these settings, candidate responses often differ significantly in surface form, making majority voting unreliable and diminishing the effectiveness of self consistency.\n\nAs shown in the table below, self consistency indeed improves performance on **MATH**, but shows **much weaker performance on tool use**, and **negligible gains on code and instruction-following** tasks.\n\n|  | MATH | MBPP | xLAM | AlpacaEval |\n| --- | --- | --- | --- | --- |\n| ListRanker (ours) | **46.3** | **54.5** | **32.6** | **30.7** |\n| Self-Consistency | 44.9 | 41.9 | 24.6 | 20.4 |\n| First Sample | 25.1 | 41.9 | 10.6 | 20.4 |\n\nThe second category includes methods such as Contrastive Decoding [3] and DoLa [4], which refine the model’s output distribution during generation. These approaches are **orthogonal to ours**, which focuses on post-sampling ranking. They are complementary to our method and baselines, and can be integrated independently; therefore, we do not include them as direct comparisons in our evaluation.\n\n[1] Chen,X., et al, RM-R1: Reward Modeling as Reasoning.\n\n[2] Wang, X., el al. Self-Consistency Improves Chain of Thought Reasoning in Language Models.\n\n[3] Li, X., et al. Contrastive Decoding: Open-ended Text Generation as Optimization.\n\n[4] Chuang, Y., et al. DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models.\n\n---\n\n**W2. Limited model coverage.** \n\nThanks for your suggestions. To further validate the generality of our method, we additionally include results on **Gemma3-4B-it**, thereby expanding the coverage in both model scale and backbone diversity. The results on the MATH and the CODE task are shown in the table below.\n\n| Method | Parameter | MATH | CODE |\n| --- | --- | --- | --- |\n| ListRanker (ours) | **0.20M** | 72.2 | 51.4 |\n| PointRanker (ours) | **0.19M** | **72.4** | **51.7** |\n| RM (gpt2) | 137M | 69.2 | 50.3 |\n| RM (Qwen7B) | 161M/7.6B | 69.1 | 50.9 |\n| Beam Search | — | 67.4 | 49.2 |\n| First Sample | — | 63.5 | 48.8 |\n\nSimilar to the results in our paper, both the listwise and the pointwise rankers **significantly improve model performance** on both the MATH task and the CODE task. These experiments furthur prove the effectiveness of our proposed ranker across different scales and backbone models.\n\n---\n\n**W3. Narrow scope of transferability.**\n\nAccording to your suggestions, we conduct cross-domain transfer experiments using Llama similarly to the transfer experiements in the paper. We train the ranker on a single task type (MATH or CODE) and evaluate its generalization on the other task. \n\n|  | To MATH | To CODE |\n| --- | --- | --- |\n| Ranker From MATH | **46.3** | 51.2 (-3.3) |\n| Ranker From CODE | 43.4 (-2.9) | **54.5** |\n| GPT2 | 42.9 (-3.4) | 47.7 (-6.8) |\n\nThe Table shows that rankers trained on any individual task maintain **robust performance on the other one**. Notably, the transfer performance is even better than the reward model using GPT2 (43.4 v.s. 42.9 on MATH and 51.2 v.s. 47.7 on CODE). This furthur highlights the system’s adaptability to unseen domains.\n\nFurthermore, unlike recent general reward models, our ranker is **highly efficient and easy to train**. Its lightweight design supports pairing **one base model with multiple task-specific rankers**, enabling flexible deployment with minimal overhead. This makes our Language System framework a practical and scalable solution for adapting to diverse downstream tasks under real-world resource constraints.\n\n---\n\n**W4. Missing ablation on ranker scale.** \n\nThank you for the insightful comment. We have already analyzed **ranker scaling** in **Table 5 (Sec. 4)**. Increasing Transformer blocks from 1 to 4 yields only marginal gains (e.g., 46.3 → 46.9 on MATH), while computational cost grows linearly. This supports our default use of a **single block** for optimal efficiency–performance trade-off.\n\nWe also compare two architectural variants: (i) a **listwise** ranker with Transformer blocks and (ii) a **pointwise** ranker with MLP blocks. As shown in **Tables 2 and 5**, both perform well, suggesting that the primary benefit comes from **reusing base model features**, rather than increasing ranker complexity.\n\nAdditionally, **Table 4** presents fine-grained ablations on the ranker architecture, verifying the contribution of each component. For example, removing **instruction features** (i.e., user intent) leads to a clear performance drop, highlighting the importance of modeling instruction–response alignment and further supports our recommender-system-inspired design perspective.\n\n---\n\n**Q1.** The performance with 10–20 candidate responses.\n\nWe believe there may be a misunderstanding. **At inference time, we only sample 10 candidate responses**, which aligns with realistic latency budgets. The setting with 100 candidates is used **only during ranker training** to construct a diverse and informative training set—this is a common practice and is also used when training reward models. All results reported in Tables 2–6 are obtained using **10 candidates during inference**, under which our ranker consistently **outperforms or matches** the performance of reward models.\n\nTo further assess generality, we also evaluate the ranker’s performance across different numbers of candidate responses (from 1 to 100), as shown in **Figure 4**. The results indicate that performance generally improves as the number of samples increases, but our method remains effective even with small candidate sets.\n\n---\n\n**Q2.** CPU benchmarks.\n\nTo our knowledge, **there are no prior benchmarks on on-device adaptation for LLM ranking**, as previous approaches typically rely on large reward models that are impractical to train or deploy on CPU devices. In contrast, our ranker is the **first lightweight design (<0.5M parameters)** that can be efficiently trained and run on CPUs (see Table 3), making **on-device adaptation both feasible and practical**. \n\n---\n\n**Q3.** Integrate sampling and ranking into a unified training loop.\n\nThank you for this great suggestion. We conducted a preliminary experiment where the **ranker guides sampling in a loop**:\n\n1. train a pointwise ranker on part of the data,\n2. use it to select “hard” responses (misclassified ones) from new data,\n3. retrain the ranker on these, and repeat 3 times.\n\nOn **Qwen2.5‑7B‑Instruct (MATH)** this improves accuracy **from 75.2  to  76.4**. This shows that integrating sampling and ranking can further boost performance. Exploring this unified training paradigm and extensions such as data selection during pretraining is an exciting direction for future work.\n\n---\n\n**L1.**   I/O overhead with large number of outputs (K=100).\n\nThis concern appears to stem from a misunderstanding regarding the output number. As clarified in **Response to Q1**, our method uses only **10 candidate responses** during **inference**, not 100. With 10 candidates, the total size of hidden states is approximately **100KB**, resulting in **minimal I/O overhead**. Combined with the ranker’s lightweight architecture (<0.5M parameters), our method remains highly efficient and is well-suited for **real-time** or **low-resource** deployment scenarios."}, {"review_id": "vBjU0XfcWT", "reviewer": "Reviewer_gdjF", "summary": "This paper introduces “Language System”,  a novel and lightweight ranking framework designed to enhance Large Language Models (LLMs) by reinterpreting the decoding process through the lens of recommender systems. The authors argue that existing decoding strategies and auxiliary reward models suffer from high computational costs and redundancy. Language System addresses this by employing a lightweight ranker that leverages features extracted directly from the base model to rerank candidate responses. The framework aims to provide comparable performance to large-scale reward models while significantly reducing additional parameters and computational overhead during training and inference.", "questions": "1.Can the authors provide a more rigorous theoretical analysis or a deeper conceptual justification for why the recommender system analogy is particularly effective for LLM decoding, beyond just an empirical observation of efficiency? What specific properties of recommender systems are being leveraged that are uniquely beneficial here?\n2.Could the authors provide more specific details on the dataset construction for each task, especially regarding how the labels for candidate responses are generated?\n3.The method involves generating K candidate responses. How robust is the performance of Language System to the size of this candidate set (K)? Is there an optimal K, and how does varying K affect both performance and efficiency?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:   The paper introduces a compelling analogy between LLM decoding and recommender systems, offering a fresh perspective on the problem of efficient LLM inference.  The proposed Language System is highly parameter-efficient, requiring only <0.5M additional parameters, which significantly reduces computational overhead during both training and inference compared to large reward models. The experimental results demonstrate that Language System achieves performance comparable to large-scale reward models across various tasks.\n\n\nWeaknesses: While the recommender system analogy is interesting, the core technical contribution of employing a lightweight ranker on extracted intermediate features, while efficient, may not be sufficiently novel. Lack of Deep Theoretical Analysis: The paper primarily presents an analogy and empirical results without delving into a deeper theoretical justification for why this specific recommender system perspective offers unique advantages beyond empirical efficiency.The paper primarily compares against reward models and simple decoding strategies. A more comprehensive comparison against other state-of-the-art efficient decoding or reranking methods that do not rely on auxiliary reward models would be crucial to fully assess the novelty and superiority of the proposed approach.", "was_revised": true, "final_justification": "The improvement in methodological rigor is commendable: The authors provided detailed explanations of the dataset construction and response labeling process, which increases my confidence in the rigor of this work. Their analysis of the candidate set size K also shows they considered the method's practical application. These are positive improvements that enhance the overall quality of the paper.", "rebuttal": "We appreciate your recognition of the **fresh perspective, strong empirical performance, and high efficiency of our proposed framework**. The concerns raised—particularly regarding theoretical justification and evaluation breadth—are valuable and well taken. We address each of these points in detail below with further clarifications, analysis, and new experimental evidence.\n\n---\n\n**W1.** Lack of novelty. \n\nWe respectfully disagree with the concern regarding novelty. As noted by reviewers **RWdC** and **L4gf**, **novelty is one of the key strengths** of our work. Specifically, our method introduces **a series of novel contributions** that clearly distinguish it from existing approaches:\n\n1. **Purpose of Feature Reuse:** We are the **first to use LLM hidden states explicitly for response reranking** as an **efficient and effective inference-time method**.\n2. **Alignment Between Instruction and Response Features:** Inspired by the recommender system perspective, we treat the final token of the **instruction** as a **user feature** and compute its relevance to each candidate response’s feature. This is the **first work that applies recommender system principles to LLMs.** As shown in **Table 4**, our ablation study demonstrates that this design significantly improves performance, highlighting the importance of modeling user–response relevance explicitly.\n3. **Personalized Language System:** Our framework allows a single base model to be paired with **multiple lightweight rankers**, each tailored to different user preferences or task requirements — analogous to personalized ranking in recommender systems [1]**.** This enables a novel form of **personalized LLM system**, which prior approaches do not efficiently achieve, especially under real-world resource constraints.\n\n[1] Pei, C., et al. Personalized Re-ranking for Recommendation.\n\n---\n\n**W2.** Lack of Deep Theoretical Analysis.\n\nThank you for this constructive suggestion. We agree that a theoretical perspective is valuable, and we have developed a formal analysis that explains **why the two‑stage Language System, inspired by recommender systems, offers unique advantages beyond empirical results**. Below, we present the core insights of this analysis.\n\n### **Modelling LLM Response Generation as Ultra-Large Classification**\n\nLet $\\mathcal{V}$ be the vocabulary ($|\\mathcal{V}|=V$) and $L$ the maximum response length. A language model $h_{\\theta}$ induces a distribution: \n\n$h_{\\theta}(y\\mid x)\\;\\mathcal{X}\\longrightarrow\\Delta\\bigl(\\mathcal{Y}\\bigr), \\quad\\text{with } \\mathcal{Y}= \\mathcal{V}^{L}, |\\mathcal{Y}|=V^{L}.$\n\nGenerating a single response can therefore be viewed as a $|\\mathcal{Y}|$-way classification task. Suppose the base model currently has an error rate $\\varepsilon\\in(0,1)$, and our target is to achive a stricter error $\\tau<\\varepsilon$.\n\nThere are two strategies:\n\n1. **Scheme I:** further fine-tuning $h_{\\theta}$;\n2. **Scheme II (ours):** drawing $K$ independent samples from $h_{\\theta}$ and training a $K$-way ranker $g_{\\phi}$.\n\n### **Theorem 1 (Sample-Efficiency of Top-$K$ Reranking)**\n\nLet $d_1=\\Theta(L\\log V)$ denote the VC dimensions of the hypothesis classes for Scheme I and $d_{2}=\\Theta(\\log K)$ for Scheme II.\n\nIf $\\varepsilon^{K}<\\tfrac{\\tau}{2}\\;$ and $d_{2}=o\\bigl(d_1\\bigr) \\tag{A}$,\n\nthen for any confidence $\\delta\\in(0,1)$ the additional labeled samples required satisfy\n\n$m_{\\text{II}} = \\tilde{\\mathcal{O}}\\Bigl(\\tfrac{d_2+\\ln(1/\\delta)}{(\\tau-\\varepsilon^{K})^{2}}\\Bigr) = o\\Bigl( \\tfrac{d_{1}+\\ln(1/\\delta)}{(\\varepsilon-\\tau)^{2}} \\Bigr) = o\\bigl(m_{\\text{I}}\\bigr).$\n\nThat is, Scheme II **requires strictly fewer labeled samples than Scheme I** to reach the same target error.\n\n### **Proof Sketch**\n\n- **Scheme I:** Standard PAC–VC bounds yield\n    \n    $m_{\\text{I}} = \\tilde{\\mathcal{O}}\\left(\\frac{d_{1} + \\ln \\frac{1}{\\delta}}{(\\varepsilon - \\tau)^{2}}\\right).$\n    \n- **Scheme II:** Top-$K$ sampling misses the correct label with probability $\\varepsilon^{K}$. If the ranker errs with probability $\\varepsilon_{2}$, the overall error is\n    \n    $E_{\\text{II}} = \\varepsilon^{K} + (1 - \\varepsilon^{K})\\varepsilon_{2}.$\n    \n    Imposing $E_{\\text{II}} \\le \\tau$ gives $\\varepsilon_{2} \\le  \\frac{\\tau - \\varepsilon^{K}}{1 - \\varepsilon^{K}} $.\n    \n    Applying PAC-VC bounds for the ranker:\n    \n    $m_{\\text{II}} = \\tilde{\\mathcal{O}}\\left(\\frac{d_{2} + \\ln \\frac{1}{\\delta}}{\\varepsilon_2^{2}}\\right) = \\tilde{\\mathcal{O}}\\left(\\frac{d_{2} + \\ln \\frac{1}{\\delta}}{(\\tau - \\varepsilon^{K})^{2}}\\right).$\n    \n    Since $d_{2} \\ll d_{1}$, this implies $m_{\\text{II}} \\ll m_{\\text{I}}$.\n    \n\n### **Remarks**\n\n- **Typical LLM Regime:** For realistic setting $V \\sim 10^{5}$, $L \\approx 100$, and moderate $K \\le 10$, we have $d_{2} \\ll d_{1}$. Pre-trained LLMs often achieve $\\varepsilon < 0.4$; with $K = 5$ the coverage condition $\\varepsilon^{K} < \\tau/2$ is easily met for practical $\\tau$ (e.g., 5%).\n- **Theoretical Justification:** Given that $d_{2} << d_{1}$ and $m_{\\text{II}} \\ll m_{\\text{I}}$. **Scheme II** requires significantly fewer training and inference parameters, as well as substantially less training data. This provides a strong theoretical justification for the efficiency and practicality of our proposed method.\n\n\n---\n\n**W3.**  Incomplete comparison with decoding methods.\n\nThank you for the suggestion. We clarify that decoding methods generally fall into two broad categories: (i) **reranking-based methods**, which operate on sampled responses as our method does, and (ii) methods that **modify the model’s output probability distribution** during generation.\n\nThe first category often relies on **auxiliary reward models** like our baselines, or is **task-specific**. For example, **self consistency** [2] improves reasoning performance by aggregating multiple sampled answers via majority voting. This technique is particularly effective in tasks like **math**, where the final answer is often a short, well-defined value—making consensus across candidates meaningful.\n\nHowever, for tasks such as **code generation**, **tool use**, or even **general instruction-following** (see Appendix B), the outputs tend to be long, diverse, and semantically equivalent in multiple ways. In these settings, candidate responses often differ significantly in surface form, making majority voting unreliable and diminishing the effectiveness of self consistency.\n\nAs shown in the table below, self consistency indeed improves performance on **MATH**, but shows **much weaker performance on tool use**, and **negligible gains on code and instruction-following** tasks.\n\n|  | MATH | MBPP | xLAM | AlpacaEval |\n| --- | --- | --- | --- | --- |\n| ListRanker (ours) | **46.3** | **54.5** | **32.6** | **30.7** |\n| Self-Consistency | 44.9 | 41.9 | 24.6 | 20.4 |\n| First Sample | 25.1 | 41.9 | 10.6 | 20.4 |\n\nThe second category includes methods such as Contrastive Decoding [3] and DoLa [4], which refine the model’s output distribution during generation. These approaches are **orthogonal to ours**, which focuses on post-sampling ranking. They are complementary to our method and baselines, and can be integrated independently; therefore, we do not include them as direct comparisons in our evaluation.\n\n[1] Chen,X., et al, RM-R1: Reward Modeling as Reasoning.\n\n[2] Wang, X., el al. Self-Consistency Improves Chain of Thought Reasoning in Language Models.\n\n[3] Li, X., et al. Contrastive Decoding: Open-ended Text Generation as Optimization.\n\n[4] Chuang, Y., et al. DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models.\n\n---\n\n**Q1.** Provide a theoretical analysis.\n\nThank you for your suggestion. We provide the theoretical analysis in our response to **W2**. In summary, under the setting of enhancing LLM performance on a specific downstream task, we show that our **two-stage Language System** can achieve comparable or better target performance with **significantly lower sample complexity and training cost** compared to directly fine-tuning the LLM.\n\n---\n\n**Q2.** Provide details on the dataset construction, especially response labeling. \n\nThank you for the question. The construction of training datasets for each task involves two steps:\n\n1. **Response Sampling:** We prompt the base language model to generate **100 candidate responses** for each input query.\n2. **Response Labeling:** Each response is labeled based on task-specific criteria.\n\nSpecifically, the labeling criteria for different tasks are as follows:\n\n- **Math:** We extract the final answer from each response and compare it with the ground-truth answer. A response is labeled as correct if the extracted answer matches exactly. Responses that are incorrect or from which the answer cannot be extracted (due to formatting errors) are labeled as incorrect.\n- **Code:** We extract the code segment from each response and execute it on a set of predefined test cases. A response is labeled as correct only if **all test cases pass**; otherwise, it is labeled as incorrect.\n- **Tool Use:** We extract the tool call from each response and use regular expressions to parse the **tool name** and **parameter values**. The response is labeled as correct only if **both the tool and all arguments** match the ground-truth tool calls.\n\n---\n\n**Q3.** How robust is the performance of Language System to the size of this candidate set (K)? \n\nAs shown in **Figure 4**, we evaluate the ranker’s performance across different numbers of candidate responses (ranging from 1 to 100). We observe that performance generally improves as the number of candidates increases, but our method remains effective even with a **small candidate set**.\n\nTo balance effectiveness and efficiency, we set K = 10 as the default configuration for our main experiments. This setting achieves strong performance while keeping computational overhead low, making it suitable for practical deployment."}, {"review_id": "6dp9T9EBSX", "reviewer": "Reviewer_pdDQ", "summary": "This paper proposes Language System (LS), a lightweight, modular framework that sits between a base language model and the environment to enable structured, goal-directed communication among multiple LLM agents. Rather than relying on free-form prompting or hard-coded coordination mechanisms, the LS introduces a function call abstraction that wraps natural language into semantically meaningful units (e.g., ask(), query(), decide(), speak()), supporting flexible role-based delegation and inter-agent messaging.\n\nThe core idea is to define a declarative communication layer through which each agent can invoke high-level language functions in a cooperative setting, such as collaborative QA, debate, or strategic planning. The system is designed to be lightweight and model-agnostic, operating without changes to the underlying LLMs.\n\nEmpirical results on synthetic tasks and cooperative reasoning benchmarks demonstrate that agents using the LS framework achieve better task success rates, more interpretable dialogues, and improved compositional reasoning compared to vanilla prompting or chain-of-thought baselines.", "questions": "1. Can the proposed Language System be applied beyond synthetic multi-agent reasoning tasks? How does it perform in open-domain, real-world applications such as collaborative dialogue, assistant planning, or decision support?\n2. How do you measure the quality or efficiency of communication between agents? Are there improvements in message compactness, convergence steps, or semantic coherence due to the LS layer?\n3. What are the latency and compute implications of introducing the LS layer, especially as the number of agents or roles increases?", "limitations": "No.\n\nWhile the paper presents a well-motivated and technically sound framework for multi-agent coordination via the Language System (LS), it does not adequately discuss potential limitations or societal risks, which is important for a system designed to mediate communication between autonomous language agents.\nOvercoordination and Emergent Misuse\nThe structured communication layer could be used to enable collusive behavior, manipulation, or spam generation by coordinating LLM agents in malicious contexts. The authors should acknowledge this risk and discuss how the system might be constrained or audited to prevent misuse.\n\nOver-Reliance on Cooperative Assumptions\nThe experiments assume fully aligned, cooperative agents. However, many real-world deployments involve partial alignment or adversarial interactions (e.g., debate systems, multi-agent negotiation). The authors should discuss whether LS can gracefully handle such non-cooperative or competitive dynamics, or clarify that it is out of scope.\n\nLack of Discussion on Bias and Amplification\nSince LS wraps LLM outputs into structured prompts, there is a risk of amplifying biases or toxic behaviors if the underlying model is not well-aligned. This should be addressed explicitly in the limitations section.\n\nScalability and Real-Time Use\nThe system’s latency and cost at scale are not discussed. While the LS is lightweight in design, scaling to many agents or complex environments may incur bottlenecks. Clarifying these limitations would help practitioners plan deployments responsibly.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Quality\nThe proposed Language System (LS) is a well-engineered abstraction that enables structured multi-agent communication via semantically meaningful language functions.\n\nThe system is modular and lightweight, requiring no changes to the underlying language models, making it broadly applicable.\n\nThe authors provide experiments across multiple synthetic environments and reasoning tasks, comparing LS-enhanced agents against standard prompting baselines and chain-of-thought methods.\n\nAblations demonstrate the utility of the system’s key components (e.g., modular roles, function types).\n\nClarity\nThe paper is clearly written and well-structured. Diagrams effectively illustrate the architecture and agent-message interactions.\n\nExamples are intuitive and support understanding of how the Language System operates in practical settings.\n\nTerminology (e.g., ask(), speak(), decide()) is consistently defined and used, helping clarify roles and responsibilities among agents.\n\nOriginality\nThe idea of introducing a declarative, language-level interface layer for agent communication is novel and distinct from previous works relying on raw prompt engineering or hardcoded workflows.\n\nLS represents a conceptual advance in designing higher-level agent APIs without altering LLM internals.\n\nSignificance\nAs multi-agent LLM systems become more common (e.g., in collaborative assistants, debate agents, or tutoring systems), the proposed LS provides a practical and extensible coordination mechanism.\n\nThe work bridges the gap between general-purpose language models and structured multi-agent orchestration, which is underexplored in current LLM frameworks.\n\nWeaknesses\nQuality\nMost evaluations are limited to synthetic or toy environments. There is limited demonstration of real-world applicability (e.g., open-domain dialogue, decision support, or complex workflows).\n\nThe experiments show modest performance gains in accuracy but lack deeper analysis of why LS improves agent collaboration (e.g., via communication quality metrics or trajectory visualization).\n\nNo results are provided for adversarial, noisy, or uncooperative agents, which are critical for robust multi-agent deployment.\n\nClarity\nWhile the overall presentation is strong, some implementation details are sparse, especially around the mapping between high-level language functions and LLM prompt formatting.\n\nIt's unclear whether the system handles interruptions, fallbacks, or failed communication gracefully—this limits understanding of its robustness.\n\nOriginality\nThe abstraction of function-calls as language macros is clever, but the underlying idea of modular agent messaging has been explored (albeit less formally) in prior agent planning or AutoGPT-style systems.\n\nA more comprehensive comparison to existing multi-agent prompting or tool-use systems (e.g., CAMEL, AutoGen, ReAct) would better situate this work.\n\nSignificance\nWhile the framework is promising, its impact could be diminished if it proves difficult to generalize to diverse, real-world multi-agent use cases.\n\nThe system may be too reliant on cooperative behavior assumptions, with little discussion on scalability to competitive or misaligned agent settings.", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer pdDQ,\n\nThank you for your time and effort in reviewing our paper.\n\nHowever, we noticed a significant mismatch between your comments and the content of our submission — to the extent that we are concerned the review may have been mistakenly attached from a different paper. Specifically, your review describes our method as a **multi-agent system with agent communication and coordination**, whereas our work proposes a lightweight ranking framework for refining responses from a **single language model, without involving any multi-agent interactions.**\n\nIf there are any concerns related to our actual work, we would be more than happy to address them.\n\nThank you again for your consideration.\n\nSincerely,\n\n*The Authors*"}]

Author rebuttal:

Meta-review: **Summary:**
This paper introduces "Language System", a novel framework that re-frames LLM decoding as a recommender system task, using a lightweight ranker to rerank candidate responses. The method demonstrates strong, efficient performance across diverse tasks (math, code, tool use) with minimal parameter overhead (<0.5M). The authors provided a comprehensive rebuttal, adding cross-domain transfer results to address reviewer concerns.

**Pros:**
*  Fresh, well-motivated recommender-system analogy for LLM output selection.
*  Achieves performance comparable to large reward models with a tiny ranker.
*  Comprehensive experiments across multiple domains and model families (LLaMA, Qwen, Gemma) support the claims.

**Cons:**
*   While a theoretical analysis was added, its depth and direct connection to the empirical success remain a point for future work.
*   Scaling laws for the ranker itself are not fully explored.
*   The comparison, though expanded, could still be more exhaustive against the full spectrum of modern decoding-time optimization techniques, such as some confidence-aware inference scaling methods.

Overall, I like the idea of this paper. However, the paper writing could be further optimized. For example, the abstract should clearly state what key conclusions were drawn from the analogy between LLM decoding and RecSys, which led to the identification of limitations. The introduction should explain how the ranker is optimized, as well as address the generalizability of the ranker.
Decision: Accept (poster)