Title: Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations
Venue: NeurIPS 2025
Accepted: yes
Keywords: llm-as-a-judge

llm evaluator
Abstract: The growing scale of evaluation tasks has led to the widespread adoption of automated evaluation using LLMs, a paradigm known as “LLM-as-a-judge”. However, improving its alignment with human preferences without complex prompts or fine-tuning remains challenging. Previous studies mainly optimize based on shallow outputs, overlooking rich cross-layer representations. In this work, motivated by preliminary findings that middle-to-upper layers encode semantically and task-relevant representations that are often more aligned with human judgments than the final layer, we propose LAGER, a post-hoc, plug-and-play framework for improving the alignment of LLM-as-a-Judge point-wise evaluations with human scores by leveraging internal representations. LAGER produces fine-grained judgment scores by aggregating cross-layer score-token logits and computing the expected score from a softmax-based distribution, while keeping the LLM backbone frozen and ensuring no impact on the inference process.
LAGER fully leverages the complementary information across different layers, overcoming the limitations of relying solely on the final layer.
We evaluate our method on the standard alignment benchmarks Flask, HelpSteer, and BIGGen using Spearman correlation, and find that LAGER achieves improvements of up to 7.5% over the best baseline across these benchmarks. Without reasoning steps, LAGER matches or outperforms reasoning-based methods. Experiments on downstream applications, such as data selection and emotional understanding, further show the generalization of LAGER.

Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations

Abstract

The growing scale of evaluation tasks has led to the widespread adoption of automated evaluation using LLMs, a paradigm known as "LLM-as-a-judge". However, improving its alignment with human preferences without complex prompts or finetuning remains challenging. Previous studies mainly optimize based on shallow outputs, overlooking rich cross-layer representations. In this work, motivated by preliminary findings that middle-to-upper layers encode semantically and taskrelevant representations that are often more aligned with human judgments than the final layer, we propose LAGER, a post-hoc, plug-and-play framework for improving the alignment of LLM-as-a-Judge point-wise evaluations with human scores, by leveraging internal representations. LAGER produces fine-grained judgment scores by aggregating cross-layer score-token logits and computing the expected score from a softmax-based distribution, while keeping the LLM backbone frozen and ensuring no impact on the inference process. LAGER fully leverages the complementary information across different layers, overcoming the limitations of relying solely on the final layer. We evaluate our method on the standard alignment benchmarks Flask, HelpSteer, and BIGGen using Spearman correlation, and find that LAGER achieves improvements of up to 7.5% over the best baseline across these benchmarks. Without reasoning steps, LAGER matches or outperforms reasoningbased methods. Experiments on downstream applications, such as data selection and emotional understanding, further show the generalization of LAGER.

1 Introduction

Large language models (LLMs) have witnessed significant progress in various tasks like math reasoning (Ying et al., 2024; Duan & Wang, 2024) and open-domain question answering (Allemang & Sequeda, 2024),and exhibit great generalization and reasoning abilities on unseen tasks and domains (Li et al., 2024c; Ye, 2024; Ni et al., 2024), which makes it possible to evaluate the model generations with an LLM (Bavaresco et al., 2024; Zheng et al., 2023). LLM-as-a-judge (Lin & Chen, 2023; Li et al., 2024a; Wu et al., 2024; Shiwen et al., 2024), an emerging approach to text evaluation, uses large language models to perform automated and scalable assessment of textual responses, reducing the reliance on human annotation. This paradigm has found widespread applications in various scenarios such as model evaluation (Lin & Chen, 2023), data synthesis (Kim et al., 2024a; Cui et al., 2023b; Wang et al., 2024b), and model enhancement via verification and critic agents (Pace et al., 2024; Badshah & Sajjad, 2024).

Researchers have explored various approaches to improve the consistency of judgment with human experts. Prompt-based methods (Lin & Chen, 2023; Liu et al., 2023) improve the judge performance with guidelines encouraging step-by-step reasoning (Chen et al., 2024; Lin & Chen, 2023; Zhuo, 2024)

[𝒔]

(a)

Better evaluation requires going beyond surface-level outputs

(b)

The overview of the overall framework of

LAGER

.

LM_Head

Decoder

Layer 1

𝒉𝑛

1

𝒉𝑛

0

Decoder

Layer 2

· ·

·

Decoder

Layer L

𝒉𝑛

𝐿

Extracting the logits of candidate scores

𝕊

(1 to 9)

𝒛𝟎

𝒛𝟏

𝒛𝒊

𝒛𝑳

Inference process

𝒛 = ෍

𝑙=0

𝐿

𝒘𝑙𝒛𝑙

,

𝑃 𝑠

The probability of each

candidate score token

The probability of each

candidate score token

Decoder-only Model

Embedding

N

×

Decoder Layer

LM_Head

Soft-

7

Score:

generated tokens

the score

token

Score: 7

a scoring

evaluation

prompt

Please evaluate the quality of the corresp-

onding response based on the following

instructions.

Instruction

: How to improve work efficiency?

Response

: Make a clear work plan and

prioritize important tasks.

Please directly give a score from 1 to 9 to

measure the quality of the response. Score 1

means very poor and score 9 means very good.

Scoring Evaluation Prompt A

Please evaluate the quality of the corresp-

onding response based on the following

instructions.

Instruction

: How to improve work efficiency?

Response

: Drink coffee, take more breaks,

and stay focused during work time.

Please directly give a score from 1 to 9 to

measure the quality of the response. Score 1

means very poor and score 9 means very good.

Scoring Evaluation Prompt B

output

input

input

The same score but a

big distribution gap !!!

Score: 7

Score: 7

Reference Score: 7

Reference Score: 5

or carefully curated examples (Song et al., 2025). However, these methods rely on chain-of-thought reasoning, which introduces additional computational cost (Gu et al., 2024). Other researchers propose to finetune a LLM with a specialized dataset (Ye et al., 2024a; Cui et al., 2023a)to adapt the LLM to the judgment task. Although effective, these methods face the generalization problem (Doostmohammadi et al., 2024) as they have been adapted to the domain of the training data.

In this paper, we propose an effective LLM-as-a-judge framework called LAGER, to enhance L LMas-a-Judge a li g nment with human via int e rnal r epresentations. Inspired by prior studies (Wang et al., 2020; Yang et al., 2022; Roh et al., 2024) showing that middle-to-upper layers encode richer semantic and task-specific information, we propose to aggregate the logits of score tokens from different layers. By applying a weighted combination followed by a softmax, we obtain a probability distribution over candidate scores, from which the final score is computed as the expected value. LAGER leverages the semantic diversity across different layers, allowing the final score to integrate both low-level lexical cues and high-level reasoning signals, thereby providing a more comprehensive reflection of the model's understanding of the scoring task. LAGER is plug-and-play, leaving model parameters and next-token predictions unchanged. LAGER outperforms baselines on alignment benchmarks such as Flask, HelpSteer, and BIGGen, with improvements of up to 7.5% in Spearman correlation. Although reasoning-based evaluation methods provide more transparency by exposing intermediate reasoning steps, their reasoning is often shallow and fails to improve reliability, leading to inferior performance. In contrast, without explicit reasoning, LAGER achieves comparable or superior results to these reasoning-based baselines. We further select instruction data with LAGER method and achieve better performance on the AlpacaEval-2.0 benchmark than various baselines. Experiments on more downstream applications show the effectiveness of the LAGER method. 2

2 Preliminary

A decoder-only LLM f consists of an embedding layer f embd, L decoder layers f decoder, and an output projection layer f unembd. Given an input instruction x , the model generates a response y through the

following computation:

where ◦ denotes function composition, i.e., ( f ◦ g )( x ) = f ( g ( x )) . To predict the next token x i , the model typically relies on the hidden state produced by the final decoder layer, denoted as

where x <i = ( x 1 , . . . , x i -1 ) represents the sequence of previous tokens. The output logits for token x i are then computed as:

When incorporating a general LLM as the judge for a model response, the prompt template includes the evaluation task description x d , user instruction x i , the response to be evaluated x r , and the scoring criteria and output format requirement x c (Liu et al., 2023, 2024; Hu et al., 2024a). We denote the full input as X = { x d , x i , x r , x c } , and the set of candidate scores as S ⊂ V , where V is the model's vocabulary. Let h ( L ) n denote the final-layer hidden state at the position where the score token is to be generated (e.g., after 'Score:'). The model produces output logits:

which are then passed through a softmax over the model's vocabulary V :

The score token with the highest probability is the final score, referred to as the vanilla score :

Figure 1(a) shows that vanilla score overlooks informative score distributions by relying solely on the top-probability score token, limiting its ability to distinguish response quality. More reliable and fine-grained evaluation requires the use of deeper internal representations beyond surface output.

3 Methods

3.1 Motivation

Hidden representations across different layers have been widely observed to exhibit distinct characteristics: the bottom layers focus more on local lexical information, while the middle and top layers focus more on semantic and global information(Wang et al., 2020; Yang et al., 2022; Roh et al., 2024; Sun et al., 2025). Previous works (Sun et al., 2025; Alabi et al., 2024) using techniques such as PCA and shared-space analysis have shown that middle-layer hidden states are highly consistent and interchangeable across language adaptations, suggesting a shared rep-

10

20

30

Layer Index

0.0

0.2

0.4

0.6

Spearman Correlation

Flask (Spearman)

HelpSteer (Spearman)

BIGGen (Spearman)

0

10

20

30

Layer Index

LLaMA-3-8B-Instruct

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

resentation space. Thus, they can be directly transformed to logits with the shared LLM output unembedding layer f unembd.

Previous works on LLM-as-a-Judge predominantly rely on the final-layer hidden state, while ignoring the intermediate representations across layers. Hence, we investigate the judge scores derived from hidden representations at different layers to better understand the layer-wise evaluation behavior. Specifically, the logits at the l th ( l < L ) layer can be computed as follows:

Following Equation 4, we can obtain the probability distribution P l ( t | X,y <n ) over the vocabulary for each token, which allows us to compute the final judge score at the l th layer:

Following the above setup, we conduct evaluations of different models' layer-wise scoring performance across multiple benchmarks and analyzed their alignment with human judgments. The results 3 in Figure 2 show a consistent pattern across various benchmarks and models: one or more intermediate layers yield better agreement with human evaluations than the final layer.

In this paper, we focus on point-wise LLM evaluators without reference response. Inspired by our preliminary observations, we explore incorporating the hidden representations from intermediate layers to improve the judge score estimation. We propose the LAGER framework to enhance the consistency with judgment from human experts while maintaining the generalization ability of the LLM evaluator, and achieve better alignment with human score distributions.

3.2 Score Estimation Based on LLM Hidden Representations

For improved formality and clarity, we rewrite the LLM output unembedding layer f unembd as W unembd , so ˆ z l can rewrite as:

Based on the results in Figure 2, identifying consistently well-performing layers is challenging, as the layers exhibit varying levels of alignment with human judgments across different datasets. To address this, we first aggregate the logits corresponding to the set of candidate scores S from different layers using a set of weights w = { w 0 , w 1 , · · · , w L } , where w 0 corresponds to the output of the embedding layer. The above procedure can be represented as:

following Equations 4 and 5, we can calculate a integer judge score. However, as illustrated in Figure 1(b), instead of relying on a single token, the full probability distribution over score candidates can be leveraged to capture richer evaluative information. This yields a probability distribution over the discrete score set, from which we compute a continuous, fine-grained score by taking the expected value. Therefore, we first extract the logits corresponding to all candidate score tokens from the full vocabulary logits and then aggregate them:

where M = { Tokenize( s ) | s ∈ S } denotes the set of tokenized sequences from S . The final probability distribution of S is then obtained by applying a softmax over the aggregated logits:

by taking the expectation over the score distribution, we can obtain the final fine-grained judge score:

Normalizing before aggregation removes the relative scale information of logits and overemphasizes less important layers. We compare both settings in the ablation study (Section 4.3).

3.3 Lightweight Training of Layer Weights

We propose two types of layer weights w . One is to apply average aggregation w l = 1 / ( L + 1) (denoted as LAGER (w.o. tuning) in Table 1), the other is to tune the lightweight L +1 parameters ( L is the number of transformer layers) with a small-scale validation set (denoted as LAGER (w. tuning)). With a frozen backbone and minimal learnable parameters, we refer to the dataset as a validation set to distinguish it from finetuning-based LLM evaluators. To enhance the model's performance while aligning its predictions more closely with the distribution of human scores , we adopt a combination of cross-entropy(CE) loss and mean absolute error (MAE) loss, balanced by a weighting hyperparameter α .

where B is the batch of samples in a training step, and s i truth is the human-annotated score for each sample. Please refer to Appendix E.5 for specific training settings and details. It is important to note that the weights are tuned only once for each backbone and subsequently applied across all benchmarks and downstream tasks. As the LLM remains frozen, no additional judge model is needed for scenarios like self-improvement. Our method is implemented without altering the model's next-token prediction process or logits, thereby offering plug-and-play capability. In contrast, finetuning-based methods require domain-specific judge models and suffer from limited generalization due to scarce labeled data.

3.4 Discussion: Applicability under Restricted Access Settings

While LAGER relies on accessing the intermediate layers of the LLM, it can naturally adapt to more restricted scenarios. For instance, when only final-layer logits are available (e.g., API-based models), we can extract the logits corresponding to score tokens, apply softmax, and compute the expected score (this method is proposed in G-Eval (Liu et al., 2023)). We refer to this setup as expectation score ( E-Score ). The evaluation uses the vanilla score , based on the most likely token, when only the final token prediction is observable. Both settings can be viewed as simplified special cases within our framework. We treat both scores as comparable baselines in our experiments. Nonetheless, powerful open-source models still remain the mainstream for large-scale evaluation due to their lower cost and performance comparable to closed-source models (Lambert et al., 2024; Gu et al., 2024; Malik et al., 2025), making LAGER broadly applicable and easily extensible to future models.

4 Experiments

4.1 Experiments Setup

Benchmarks. Due to potential preference leakage (Li et al., 2025) between models, we evaluate our method on human-annotated datasets for reliable results. We chose three diverse point-wise benchmarks to ensure a comprehensive evaluation: Flask (Ye et al., 2024a), HelpSteer (Wang et al., 2023), and BiGGen Bench 4 (Kim et al., 2024b). In these benchmarks, human-annotated scores range from 1 to 5. Please refer to Appendix E.2 for the details.

Models. We experiment with multiple backbone models of varying sizes and from different families, including Mistral-7B-Instruct-v0.3 (Jiang et al., 2023a), InternLM3-8B-Instruct (Cai et al., 2024), LLaMA3.1-8B-Instruct (Grattafiori et al., 2024), Qwen-2.5-14B-Instruct (Qwen et al., 2025), MistralSmall-24B-Instruct and LLAMA-3.3-70B-Instruct.

Baselines. We compare LAGER with three baselines: GPTScore , vanilla score (VScore) and expectation score (E-Score) . At the same time, we compare the commonly used API-based model GPT-4o-mini . We also experiment with methods that explicitly train the LLM backbones, namely TIGERScore-7B and Prometheus2-7B . While LAGER does not modify the backbone models, these

methods are included solely for reference and are not directly compared. Please refer to Appendix E.3 for specific details about the baselines.

Evaluation Metrics. Following previous works (Gu et al., 2024; Bai et al., 2023; Liu et al., 2025), we use the commonly adopted Pearson and Spearman correlations to measure the consistency between LLM-predicted scores and human annotations. See Appendix E.4 for further details.

Evaluation Details. We adopt greedy decoding for its stability and consistency in evaluation (Lin & Chen, 2023). To comprehensively assess the effectiveness of our method, we conduct evaluations under two different conditions: 1) Direct: Directly providing a score during evaluation without reasoning. 2) Reasoning: Previous work (Liu et al., 2023; Wang et al., 2024a; Hu et al., 2024a) shows that reasoning improves judgment quality, with reasoning-first setups performing best(Chen et al., 2024; Murugadoss et al., 2024; Gu et al., 2024). Thus, we adopt a reasoning-first approach in this study. The prompt template used in our work follows the standard configuration widely adopted in prior studies (Kim et al., 2024b; Liu et al., 2023; Xu et al., 2023), please refer to Appendix G for

the prompt templates. Unless otherwise specified, we highlight the best results in each table using bold , and denote the second-best with underlining.

4.2 Main Results

Tables 1 report Spearman results of LAGER compared to baselines across three benchmarks. Section 5.3 further analyzes the performance variation of LAGER and baseline methods across different model scales, under the setting where models from the Qwen2.5 family are used as frozen backbones, and compares their results on the same set of benchmarks. Specifically, LAGER (w. tuning) achieves near-best performance on these benchmarks and demonstrates strong stability, so it serves as the main approach in our work.

The Effectiveness and Scalability of LAGER. LAGER with tuning consistently outperforms all non-training baselines across diverse LLM backbones on all three benchmarks. This highlights the importance of leveraging internal logits and output distributions for accurate scoring. Although Mistral-Small-24B-Instruct and Qwen2.5-14B-Instruct underperform GPT-4o-mini on Vscore, they achieve comparable results when using LAGER. GPTScore performs poorly due to its reliance on generation probabilities, which overlook the semantic and logical quality. Due to its focus on error detection, the fine-tuned TIGERscore-7B shows relatively limited performance. Compared to the fine-tuned Prometheus2-7B, LAGER also enables InternLM3-8B-Instruct and LLaMA3.1-8B-Instruct, two models of comparable size, to surpass it even though they initially have weaker performance on Vscore. Even without tuning, LAGER outperforms E-Score in 5 out of 6 backbone models, with an average improvement of 4.5%. In the few cases where it underperforms, the performance gap is within 3%. Moreover, it consistently outperforms V-Score.

The Robustness of LAGER. The average E-Score performance of models like Qwen-2.5-14BInstruct, LLAMA-3.3-70B-Instruct, and Mistral-7B-Instruct-v0.3 is sometimes lower than VScore. Specifically, the average E-score performance of LLAMA-3.3-70B-instruct across all benchmarks is 2.4 points lower than Vscore. In contrast, LAGER with tuning achieves almost the best performance across all settings and benchmarks, consistently outperforms Vscore. LAGER can closely match human score distribution and incorporate more fine-grained information. More details are discussed in Section 5. Therefore, we primarily recommend the LAGER with tuning approach and conduct subsequent analyses based on it. The performance gains of LAGER stem not from large-scale finetuning, but from a more efficient and structured utilization of the model's internal representations.

Reasoning Is Not Always Better. The richness of information aggregation increases gradually from Vscore, E-Score to LAGER (with tuning). Figure 6 shows that reasoning leads to an improvement in Vscore performance, a divergence in E-Score performance (with half of the models improving and the other half declining), and consistently weaker results for LAGER (with tuning) under the reasoning setting compared to direct. We speculate that this phenomenon may be due to reasoning potentially causing the model to become overconfident (Yang et al., 2024; Wagner et al., 2024), which in turn leads to inaccurate evaluation scores. This is also consistent with the findings of previous work (Ge et al., 2025; Guo et al., 2024; Li et al., 2024d): As the reasoning steps progress, the model's focus on the original input and the text to be evaluated gradually diminishes, shifting more toward its self-generated reasoning trajectory.

4.3 Ablation Study

To better understand each design choice of LAGER, we conduct a comprehensive ablation study on the direct setup using Mistral-7B-Instruct-v0.3 and InternLM3-8B-Instruct. Table 2 shows the results of the Spearman correlation. The expectation (Exp.) score and the maximum (Max.) score are alternative designs. Given the score distribution of a certain layer, we use Eqn.12 to calculate the expected score or use the score with the highest probability. Logits agg. and prob. agg. are two alternative designs, aggregating information from all hidden layers at the logits level or output distribution level. Removing logits agg. and prob. agg. indicates that we do not use internal states. It is observed that the full configuration (Exp. score + Logits agg. + Tuning) achieves the best performance across all settings, with a maximum Spearman correlation of 0.545; Fine-tuning brings up to +0.10 improvement; expectation scoring outperforms maximum scoring (up to +0.17); and logits aggregation surpasses probability aggregation (up to +0.07). Multi-layer integration is generally effective, especially for Mistral; however, in untuned settings, InternalLM occasionally performs better without aggregation, indicating model-specific differences.

5 Analyses

5.1 Understanding Internal States

0

10

20

30

Layer Index

0.1

0.0

0.1

0.2

0.3

0.4

0.5

Spearman Correlation

(a)

Flask (Spearman)

HelpSteer (Spearman)

0.0

0.2

0.4

0.6

0.8

1.0

0

10

20

30

Layer Index

0.1

0.0

0.1

0.2

0.3

0.4

0.5

Spearman Correlation

(b)

LLaMA-3-8B-Instruct

0.0

0.2

0.4

0.6

0.8

1.0

LLaMA

Mistral

Flask

HelpSteer

(c)

Cosine Similarity Heatmap

0

500

1000

1500

Having observed that internal states contribute significantly to the success of LAGER, we now aim to understand how internal states in each layer contribute to the judgment score. Specifically, we apply LAGER to each layer of the backbone LLM, namely using the expected score from the output distribution of each layer, and report the consistency with human scores. Figure 3 illustrates the results of the Mistral-7B-Instruct-v0.3 and LLaMA-3.1-8B-Instruct backbones. We observe the following: (1) The bottom layers exhibit low or even negative correlation with human scores , indicating their limited utility in evaluation tasks. (2)The middle-upper layers (approximately layers 20 to 30) show the highest correlation with human scores, suggesting they carry the most informative representations for judgment . However, there is a drop in correlation at the topmost layer, implying that certain judgment-relevant signals may be diluted or lost in the final transformation stages. Thus, relying solely on the top layer is insufficient for precise evaluation.

To better understand the differences between internal and top layers, we visualize the cosine similarity between hidden states across layers at the position of the score token using the Flask and HelpSteer datasets. Based on Figure 3(c), the similarity between layers generally decreases as the distance between layers increases. For the bottom layers, the representation changes rapidly with increasing layer number. In contrast, the representation changes relatively slowly for the middle-upper layers, and the representations across these layers remain similar over a broader range. The topmost layer exhibits low similarity to its neighboring layers, displaying a distinct pattern.

Based on the above observations, we believe that: The middle-upper layers exhibit relatively consistent representations and stronger alignment with human scores , indicating that these layers may deserve more attention. Meanwhile, the topmost layer shows comparatively weaker judgment ability, likely because its focus on next-token prediction leads to the loss of fine-grained evaluative information . Instead of searching for optimal layers, which vary across LLM backbones and are hard to identify, LAGER directly learns a weighted combination of all layers. This approach

0.5B

1.5B

3B

7B

14B

32B

72B

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Flask

Vanilla Score

E-Score

LAGER(w, tuning)

0.5B

1.5B

3B

7B

14B

32B

72B

0.0

0.1

0.2

0.3

0.4

0.5

0.6

HelpSteer

0.5B

1.5B

3B

7B

14B

32B

72B

0.4

0.2

0.0

0.2

0.4

0.6

Spearman Correlation Coefficient

is both lightweight and effective, requiring only a small validation set and transferring well across benchmarks (Section 4.2, Appendix D.1, Appendix D.2) and tasks (Section 6.1, Appendix D).

5.2 LAGER Yields More Human-Aligned and Fine-Grained Score Distributions

To better understand how LAGER improves over both the VScore and the E-Score, we evaluate LLaMA-3.1-8BInstruct on the Flask dataset to obtain the Vanilla Score, E-Score, and LAGER, and visualize the min-max normalized scores and human annotations using Gaussian kernel density estimation (KDE). Details of the KDE visualization procedure can be found in Appendix E.1. Figure 4 visualizes the score distributions. Compared to E-Score and VScore, LAGER yields a distribution that more closely matches human judgments, with noticeably higher overlap. LAGER mitigates the high-score bias of VScore and E-Score by shifting the score distribution to the left and producing more evenly distributed scores.

We also quantitatively compare LAGER, E-Score, and VScore by evaluating the Kullback-Leibler (KL) divergence and the mean squared error (MSE). The KL divergence of a candidate score S is computed as D KL (˜ p hs || ˜ p s ) where ˜ p hs and ˜ p s represent the human score distribution and the candidate score distribution, respectively. The results in Table 3, demonstrate that LAGER more accurately approximates the distribution of human-annotated scores.

0.0

0.2

0.4

0.6

0.8

1.0

Normalized Score

0

1

2

3

4

5

Density

KDE Plot of Different Score Distributions

Vanilla Score

E-Score

Human Score

LAGER(w. tuning)

5.3 From Tiny to Huge: LAGER Brings Improvements Across Model Scales

To further investigate whether LAGER can enhance human alignment across a broader range of model scales, we conducted experiments using three benchmarks on instruct models of varying sizes from the Qwen2.5 family, including 0.5B, 1.5B, 3B, 7B, 14B, 32B and 72B.

The Figure 5 and Figure 7 show the alignment effectiveness of Vanilla Score, E-Score, and LAGER (w. tuning) across Qwen2.5 models from 0.5B to 72B. Both direct and reasoning conditions show a mostly upward trend in Spearman correlation coefficients as model size increases across Flask, Helpsteer, and BIGGen datasets, with LAGER leading (e.g., 0.658 in Flask for direct, 0.598 for reasoning). When the model size exceeds 14B, LAGER achieves greater performance gains over the backbone model than the VScore computed using GPT-4o-mini. Notably, on the BIGGen dataset, LAGER shows a significant improvement on the 0.5B model scale, elevating the Spearman correlation from a negative value (-0.4 for the Vanilla Score) to a positive 0.1, marking a substantial gain compared to the baseline. These results demonstrate its effectiveness across different model scales and datasets, and the benefits brought by LAGER are scale-transferable and synergistically amplified as model capacity grows.

6 Applications of LAGER

6.1 Instruction Data Selection

Instruction data selection (Cao et al., 2024; Shen, 2024) involves choosing a subset of high-quality data from a given instruction dataset to enhance both the efficiency and performance of the tuning process. In this experiment, we use LAGER as a scoring metric to select a 10% subset from the alpaca-cleaned-52k dataset 5 (Taori et al., 2023). We compare our subset, Highest LAGER, with four baselines on finetuning the LLaMA3-8B-base model: Longest instructions, Longest responses (Shen, 2024), Highest VScore, and SuperFiltering (Li et al., 2024e). SuperFiltering is an explicitly designed instruction data filtering method. For methods that require a backbone LLM, we utilize LLaMA3.18B-Instruct for a fair comparison. More details are in Appendix F.

As shown in Table 4, our method with tuning achieves the best performance, outperforming the second-best method, the Longest responses, and the explicitly designed method, SuperFiltering, by 0.83 and 1.96 LC win rate, respectively. Even without tuning, our method outperforms most baselines, improving the VScore by a 2.35 LC win rate. And the highest LAGER (w tuning) has relatively shorter average lengths than most baselines. This indicates that our tuning model does not achieve better scores by generating longer responses, but by learning to follow instructions more effectively.

In addition to the application mentioned above, LAGER can be applied to other scoring-based tasks, such as LLMemotional understanding and LLMRecognizing its Knowledge Boundaries . LAGER shows strong transferability, even when there is a mismatch between the task's scoring range and that of the data used for training. Please refer to Appendix D.1 and Appendix D.2 for detailed discussions.

7 Conclusion

In this work, we propose LAGER, an effective LLM-as-a-judge framework to estimate the judge score from the logits information from different LLM layers. LAGER offers training-free and tunable options, allowing flexible integration with various LLMs to deliver fine-grained judgment services. Unlike previous methods that explicitly update LLM backbones, LAGER provides a lightweight tuning solution that only trains weights on a small validation dataset, and the learned weights can be transferred across tasks. Experiments on three comprehensive alignment benchmarks demonstrate the effectiveness of LAGER. Specifically, LAGER outperforms all baselines that do not explicitly train LLMbackbones. Additionally, LAGER demonstrates strong performance in improving alignment with human annotations across models of varying scales and families. Even without explicit reasoning, LAGER outperforms various reasoning-based baselines, including those specifically trained for reasoning tasks, while requiring fewer generation tokens and offering higher efficiency. LAGER is quite general and can be applied to various use cases, including but not limited to instruction data selection, recognizing its knowledge boundaries and emotional understanding.

References

Alabi, J. O., Mosbach, M., Eyal, M., Klakow, D., and Geva, M. The hidden space of transformer language adapters, 2024. URL https://aclanthology.org/2024.acl-long.356 .

Allemang, D. and Sequeda, J. Increasing the llm accuracy for question answering: Ontologies to the rescue!, 2024. URL https://arxiv.org/abs/2405.11706 .

Badshah, S. and Sajjad, H. Reference-guided verdict: Llms-as-judges in automatic evaluation of free-form text, 2024. URL https://arxiv.org/abs/2408.09235 .

Bai, Y., Ying, J., Cao, Y., Lv, X., He, Y., Wang, X., Yu, J., Zeng, K., Xiao, Y., Lyu, H., Zhang, J., Li, J., and Hou, L. Benchmarking foundation models with language-model-as-an-examiner, 2023. URL https://arxiv.org/abs/2306.04181 .

Bavaresco, A., Bernardi, R., Bertolazzi, L., Elliott, D., Fernández, R., Gatt, A., Ghaleb, E., Giulianelli, M., Hanna, M., Koller, A., Martins, A. F. T., Mondorf, P., Neplenbroek, V., Pezzelle, S., Plank, B., Schlangen, D., Suglia, A., Surikuchi, A. K., Takmaz, E., and Testoni, A. Llms instead of human judges? a large scale empirical study across 20 nlp evaluation tasks, 2024. URL https://arxiv.org/abs/2406.18403 .

Cai, Z., Cao, M., Chen, H., Chen, K., Chen, K., Chen, X., Chen, X., Chen, Z., Chen, Z., Chu, P., Dong, X., Duan, H., and et.al. Internlm2 technical report, 2024.

Cao, Y., Kang, Y., Wang, C., and Sun, L. Instruction mining: Instruction data selection for tuning large language models, 2024. URL https://arxiv.org/abs/2307.06290 .

Chen, Y.-P., Chu, K., and Nakayama, H. Llm as a scorer: The impact of output order on dialogue evaluation, 2024. URL https://arxiv.org/abs/2406.02863 .

Chung, J., Kamar, E., and Amershi, S. Increasing diversity while maintaining accuracy: Text data generation with large language models and human interventions. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 575-593, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.acl-long.34. URL https://aclanthology.org/2023.acl-long.34 .

Cui, G., Yuan, L., Ding, N., Yao, G., Zhu, W., Ni, Y., Xie, G., Liu, Z., and Sun, M. Ultrafeedback: Boosting language models with high-quality feedback, 2023a.

Cui, G., Yuan, L., Ding, N., Yao, G., Zhu, W., Ni, Y., Xie, G., Liu, Z., and Sun, M. Ultrafeedback: Boosting language models with high-quality feedback, 2023b.

Dong, Y. R., Hu, T., and Collier, N. Can llm be a personalized judge?, 2024. URL https://arxiv.org/abs/ 2406.11657 .

Doostmohammadi, E., Holmström, O., and Kuhlmann, M. How reliable are automatic evaluation methods for instruction-tuned llms?, 2024. URL https://arxiv.org/abs/2402.10770 .

Duan, Z. and Wang, J. Multi-tool integration application for math reasoning using large language model, 2024. URL https://arxiv.org/abs/2408.12148 .

Fu, J., Ng, S.-K., Jiang, Z., and Liu, P. Gptscore: Evaluate as you desire, 2023. URL https://aclanthology. org/2024.naacl-long.365/ .

Ge, H., Li, Y., Wang, Q., Zhang, Y., and Tang, R. When backdoors speak: Understanding LLM backdoor attacks through model-generated explanations. In Che, W., Nabende, J., Shutova, E., and Pilehvar, M. T. (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 2278-2296, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-251-0. doi: 10.18653/v1/2025.acl-long.114. URL https://aclanthology.org/2025. acl-long.114/ .

Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., Yang, A., Fan, A., and et.al. The llama 3 herd of models, 2024. URL https://arxiv. org/abs/2407.21783 .

Gu, J., Jiang, X., Shi, Z., Tan, H., Zhai, X., Xu, C., Li, W., Shen, Y ., Ma, S., Liu, H., Wang, Y ., and Guo, J. A survey on llm-as-a-judge, 2024. URL https://arxiv.org/abs/2411.15594 .

Guo, T., Pai, D., Bai, Y., Jiao, J., Jordan, M. I., and Mei, S. Active-dormant attention heads: Mechanistically demystifying extreme-token phenomena in llms, 2024. URL https://arxiv.org/abs/2410.13835 .

Hu, X., Gao, M., Hu, S., Zhang, Y., Chen, Y., Xu, T., and Wan, X. Are llm-based evaluators confusing nlg quality criteria?, 2024a. URL https://arxiv.org/abs/2402.12055 .

Hu, X., Lin, L., Gao, M., Yin, X., and Wan, X. Themis: A reference-free NLG evaluation language model with flexibility and interpretability. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pp. 15924-15951, Miami, Florida, USA, November 2024b.

Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., Lavaud, L. R., Lachaux, M.-A., Stock, P., Scao, T. L., Lavril, T., Wang, T., Lacroix, T., and Sayed, W. E. Mistral 7b, 2023a. URL https://arxiv.org/abs/2310.06825 .

Jiang, D., Li, Y., Zhang, G., Huang, W., Lin, B. Y., and Chen, W. Tigerscore: Towards building explainable metric for all text generation tasks. ArXiv , abs/2310.00752, 2023b. URL https://api.semanticscholar. org/CorpusID:263334281 .

Jung, J., Brahman, F., and Choi, Y. Trust or escalate: Llm judges with provable guarantees for human agreement, 2024. URL https://arxiv.org/abs/2407.18370 .

Kim, S., Shin, J., Cho, Y., Jang, J., Longpre, S., Lee, H., Yun, S., Shin, S., Kim, S., Thorne, J., and Seo, M. Prometheus: Inducing fine-grained evaluation capability in language models, 2024a. URL https: //arxiv.org/abs/2310.08491 .

Kim, S., Suk, J., Longpre, S., Lin, B. Y., Shin, J., Welleck, S., Neubig, G., Lee, M., Lee, K., and Seo, M. Prometheus 2: An open source language model specialized in evaluating other language models, 2024b. URL https://arxiv.org/abs/2405.01535 .

Lambert, N., Pyatkin, V., Morrison, J., Miranda, L., Lin, B. Y., Chandu, K., Dziri, N., Kumar, S., Zick, T., Choi, Y., Smith, N. A., and Hajishirzi, H. Rewardbench: Evaluating reward models for language modeling, 2024.

Li, D., Sun, R., Huang, Y., Zhong, M., Jiang, B., Han, J., Zhang, X., Wang, W., and Liu, H. Preference leakage: A contamination problem in llm-as-a-judge, 2025. URL https://arxiv.org/abs/2502.01534 .

Li, H., Dong, Q., Chen, J., Su, H., Zhou, Y., Ai, Q., Ye, Z., and Liu, Y . Llms-as-judges: A comprehensive survey on llm-based evaluation methods, 2024a. URL https://arxiv.org/abs/2412.05579 .

Li, J., Sun, S., Yuan, W., Fan, R.-Z., hai zhao, and Liu, P. Generative judge for evaluating alignment. In The Twelfth International Conference on Learning Representations , 2024b. URL https://openreview.net/ forum?id=gtkFw6sZGS .

Li, J., Yang, Y ., and et.al. Fundamental capabilities of large language models and their applications in domain scenarios: A survey. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 11116-11141, Bangkok, Thailand, August 2024c. Association for Computational Linguistics. doi: 10.18653/v1/2024. acl-long.599. URL https://aclanthology.org/2024.acl-long.599/ .

Li, K., Liu, T., Bashkansky, N., Bau, D., Viégas, F., Pfister, H., and Wattenberg, M. Measuring and controlling instruction (in)stability in language model dialogs. In First Conference on Language Modeling , 2024d. URL https://openreview.net/forum?id=60a1SAtH4e .

Li, M., Zhang, Y., He, S., Li, Z., Zhao, H., Wang, J., Cheng, N., and Zhou, T. Superfiltering: Weak-to-strong data filtering for fast instruction-tuning. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 14255-14273, Bangkok, Thailand, August 2024e. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.769. URL https://aclanthology.org/2024.acl-long.769/ .

Li, Z., Wang, C., Ma, P., Wu, D., Wang, S., Gao, C., and Liu, Y. Split and merge: Aligning position biases in LLM-based evaluators. In Al-Onaizan, Y., Bansal, M., and Chen, Y.-N. (eds.), Proceedings of EMNLP , pp. 11084-11108, Miami, Florida, USA, November 2024f.

Lin, C.-Y. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out , pp. 74-81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https: //aclanthology.org/W04-1013 .

Lin, Y.-T. and Chen, Y.-N. Llm-eval: Unified multi-dimensional automatic evaluation for open-domain conversations with large language models, 2023. URL https://arxiv.org/abs/2305.13711 .

Liu, Y., Iter, D., Xu, Y ., Wang, S., Xu, R., and Zhu, C. G-eval: Nlg evaluation using gpt-4 with better human alignment, 2023. URL https://aclanthology.org/2023.emnlp-main.153/ .

Liu, Y., Yang, T., Huang, S., Zhang, Z., Huang, H., Wei, F., Deng, W., Sun, F., and Zhang, Q. Hd-eval: Aligning large language model evaluators through hierarchical criteria decomposition, 2024. URL https: //aclanthology.org/2024.acl-long.413/ .

Liu, Y., Zhou, H., Guo, Z., Shareghi, E., Vuli´ c, I., Korhonen, A., and Collier, N. Aligning with human judgement: The role of pairwise large language model evaluators in preference aggregation, 2025. URL https://arxiv.org/abs/2403.16950 .

Malik, S., Pyatkin, V., Land, S., Morrison, J., Smith, N. A., Hajishirzi, H., and Lambert, N. Rewardbench 2: Advancing reward model evaluation, 2025. URL https://arxiv.org/abs/2506.01937 .

Murugadoss, B., Poelitz, C., Drosos, I., Le, V., McKenna, N., Negreanu, C. S., Parnin, C., and Sarkar, A. Evaluating the evaluator: Measuring llms' adherence to task evaluation instructions, 2024. URL https://arxiv.org/abs/2408.08781 .

Ni, S., Wu, H., Yang, D., Qu, Q., Alinejad-Rokny, H., and Yang, M. Small language model as data prospector for large language model, 2024. URL https://arxiv.org/abs/2412.09990 .

Pace, A., Mallinson, J., Malmi, E., Krause, S., and Severyn, A. West-of-n: Synthetic preferences for selfimproving reward models, 2024. URL https://arxiv.org/abs/2401.12086 .

Paech, S. J. Eq-bench: An emotional intelligence benchmark for large language models, 2024. URL https: //arxiv.org/abs/2312.06281 .

Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. Bleu: a method for automatic evaluation of machine translation. In Isabelle, P., Charniak, E., and Lin, D. (eds.), Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , pp. 311-318, Philadelphia, Pennsylvania, USA, July 2002. Association for Computational Linguistics. doi: 10.3115/1073083.1073135. URL https://aclanthology. org/P02-1040 .

Qwen, :, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., Lin, R., Li, T., Tang, T., Xia, T., Ren, X., Ren, X., Fan, Y., Su, Y., Zhang, Y., Wan, Y., Liu, Y., Cui, Z., Zhang, Z., and Qiu, Z. Qwen2.5 technical report, 2025. URL https://arxiv.org/abs/2412.15115 .

Roh, Y., Liu, Q., Gui, H., Yuan, Z., Tang, Y., Whang, S. E., Liu, L., Bi, S., Hong, L., Chi, E. H., and Zhao, Z. Levi: generalizable fine-tuning via layer-wise ensemble of different views. In Proceedings of the 41st International Conference on Machine Learning , ICML'24, 2024.

Shen, M. Rethinking data selection for supervised fine-tuning, 2024. URL https://arxiv.org/abs/2402. 06094 .

Shiwen, T., Liang, Z., Liu, C. Y., Zeng, L., and Liu, Y. Skywork critic model series. https://huggingface. co/Skywork , September 2024. URL https://huggingface.co/Skywork .

Song, M., Zheng, M., Luo, X., and Pan, Y. Can many-shot in-context learning help llms as evaluators? a preliminary empirical study, 2025. URL https://aclanthology.org/2025.coling-main.548/ .

Sun, Q., Pickett, M., Nain, A. K., and Jones, L. Transformer layers as painters, 2025. URL https://arxiv. org/abs/2407.09298 .

Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca , 2023.

Wagner, N., Desmond, M., Nair, R., Ashktorab, Z., Daly, E. M., Pan, Q., Cooper, M. S., Johnson, J. M., and Geyer, W. Black-box uncertainty quantification method for llm-as-a-judge, 2024. URL https://arxiv. org/abs/2410.11594 .

Wang, Q., Li, C., Zhang, Y., Xiao, T., and Zhu, J. Layer-wise multi-view learning for neural machine translation. In Proceedings of the 28th International Conference on Computational Linguistics , pp. 4275-4286, Barcelona, Spain (Online), December 2020.

Wang, Y., Yuan, J., Chuang, Y.-N., Wang, Z., Liu, Y., Cusick, M., Kulkarni, P., Ji, Z., Ibrahim, Y., and Hu, X. Dhp benchmark: Are llms good nlg evaluators?, 2024a. URL https://arxiv.org/abs/2408.13704 .

Wang, Z., Dong, Y., Zeng, J., Adams, V., Sreedhar, M. N., Egert, D., Delalleau, O., Scowcroft, J. P., Kant, N., Swope, A., and Kuchaiev, O. Helpsteer: Multi-attribute helpfulness dataset for steerlm, 2023. URL https://aclanthology.org/2024.naacl-long.185/ .

Wang, Z., Bukharin, A., Delalleau, O., Egert, D., Shen, G., Zeng, J., Kuchaiev, O., and Dong, Y. Helpsteer2preference: Complementing ratings with preferences, 2024b. URL https://arxiv.org/abs/2410. 01257 .

Wu, T., Yuan, W., Golovneva, O., Xu, J., Tian, Y., Jiao, J., Weston, J., and Sukhbaatar, S. Meta-rewarding language models: Self-improving alignment with llm-as-a-meta-judge, 2024. URL https://arxiv.org/ abs/2407.19594 .

Xu, S., Hu, J., and Jiang, M. Large language models are active critics in nlg evaluation, 2024. URL https: //arxiv.org/abs/2410.10724 .

Xu, W., Wang, D., Pan, L., Song, Z., Freitag, M., Wang, W., and Li, L. INSTRUCTSCORE: Towards explainable text generation evaluation with automatic feedback. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pp. 5967-5994, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.365. URL https://aclanthology.org/2023.emnlp-main.365 .

Yang, H., Wang, Y., Xu, X., Zhang, H., and Bian, Y. Can we trust llms? mitigate overconfidence bias in llms through knowledge transfer, 2024. URL https://arxiv.org/abs/2405.16856 .

Yang, J., Yin, Y., Yang, L., Ma, S., Huang, H., Zhang, D., Wei, F., and Li, Z. Gtrans: Grouping and fusing transformer layers for neural machine translation. IEEE/ACM Trans. Audio, Speech and Lang. Proc. , 31: 1489-1498, November 2022. ISSN 2329-9290.

Ye, H. and Ng, H. T. Self-judge: Selective instruction following with alignment self-evaluation, 2024. URL https://arxiv.org/abs/2409.00935 .

Ye, Q. Cross-task generalization abilities of large language models. In Cao, Y. T., Papadimitriou, I., Ovalle, A., Zampieri, M., Ferraro, F., and Swayamdipta, S. (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop) , pp. 255-262, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.naacl-srw.27. URL https://aclanthology.org/ 2024.naacl-srw.27/ .

Ye, S., Kim, D., Kim, S., Hwang, H., Kim, S., Jo, Y., Thorne, J., Kim, J., and Seo, M. Flask: Fine-grained language model evaluation based on alignment skill sets, 2024a. URL https://arxiv.org/abs/2307. 10928 .

Ye, Z., Li, X., Li, Q., Ai, Q., Zhou, Y ., Shen, W., Yan, D., and Liu, Y . Beyond scalar reward model: Learning generative judge from preference data, 2024b. URL https://arxiv.org/abs/2410.03742 .

Yin, Z., Sun, Q., Guo, Q., Wu, J., Qiu, X., and Huang, X. Do large language models know what they don't know? In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Findings of the Association for Computational Linguistics: ACL 2023 , pp. 8653-8665, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.findings-acl.551. URL https://aclanthology.org/2023.findings-acl.551/ .

Ying, H., Zhang, S., Li, L., Zhou, Z., Shao, Y., Fei, Z., Ma, Y., Hong, J., Liu, K., Wang, Z., Wang, Y., Wu, Z., Li, S., Zhou, F., Liu, H., Zhang, S., Zhang, W., Yan, H., Qiu, X., Wang, J., Chen, K., and Lin, D. Internlm-math: Open math large language models toward verifiable reasoning, 2024. URL https: //arxiv.org/abs/2402.06332 .

Yuan, W., Neubig, G., and Liu, P. Bartscore: Evaluating generated text as text generation. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems , volume 34, pp. 27263-27277. Curran Associates, Inc., 2021. URL https://proceedings. neurips.cc/paper/2021/file/e4d2b6e6fdeca3e60e0f1a62fee3d9dd-Paper.pdf .

Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y. Bertscore: Evaluating text generation with bert, 2020. URL https://arxiv.org/abs/1904.09675 .

Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., and Stoica, I. Judging llm-as-a-judge with mt-bench and chatbot arena, 2023. URL https://arxiv.org/abs/2306.05685 .

Zhuo, T. Y. Ice-score: Instructing large language models to evaluate code, 2024. URL https://aclanthology. org/2024.findings-eacl.148/ .

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction summarize the paper's three core contributions: proposing LAGER to enhance LLM evaluation, validating its effectiveness on the three benchmarks, and extending it to emotion understanding and uncertainty estimation tasks.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We provide a detailed discussion of the limitations of our method in Appendix A.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The paper does not include theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The paper provides all the necessary information to reproduce the main experimental results that affect the claims and conclusions.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The paper provides open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/public/ guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https://nips.cc/public/ guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The paper specifies all the necessary training and evaluation details, including data splits, hyperparameters, how they were chosen, and the type of optimizer used, to understand the results.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: For the Pearson and Spearman correlation results presented in this paper, we ensure that all reported results are statistically significant.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The paper provides sufficient information on the computer resources.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We provide a detailed discussion of the ethics statement of our method in Appendix A.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We provide a detailed discussion of the future societal consequences of our method in Appendix A.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [Yes]

Justification: We provide a detailed discussion of this in Appendix A.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The data and models used in this paper are all open-source, and we have explicitly complied with the licensing agreements and terms of use.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We have made the code repository for our method publicly available and anonymized, with all details visible in the repository.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [No]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [No]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [No]

Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

A Impact Statement

Limitation Our method relies on obtaining the hidden states from all model layers during the training and evaluation phases. This requirement means full internal access to the model's architecture and intermediate representations is essential for optimization. Consequently, the method is applicable and can be optimized only for open-source models, where such internal states are accessible. In contrast, for closed-source or proprietary models, where access to internal hidden states is typically restricted, our method cannot be deployed to improve performance, limiting its applicability in those scenarios.

Ethics Statement We affirm that this research adheres to the ethical standards in AI and machine learning. No personal, sensitive, or proprietary data was used in the experiments. All datasets employed are publicly available and widely used in the research community. Our code is publicly available, and all experiments are reproducible. We have carefully considered the potential societal impacts and are committed to applying the method responsibly, ensuring it has a positive and controllable impact on society.

Future Societal Consequences Our approach relies on comprehensive access to the internal states of models, which has profound implications for the future ecological landscape of AI development. As opensource models continue to evolve, this method helps drive the democratization of performance optimization, enabling researchers to improve models transparently and collaboratively, fostering innovation, and enhancing the accountability of AI systems. However, it cannot be applied to closed-source or proprietary models, highlighting the growing gap between open and closed AI ecosystems and potentially increasing dependence on a few dominant providers. Furthermore, since this approach is primarily used for improving judgment models, there is a potential risk of misuse, such as interfering with or manipulating evaluation results, reminding us to be mindful of possible negative impacts when promoting its application, ensuring responsible and controlled use.

B Related Work

B.1 Evaluation of Text Generation Tasks

Over the past decades, automatic evaluation metrics, including statistical methods and embedding-based approaches, have played a crucial role in assessing the quality of generated text. The metrics BLEU (Papineni et al., 2002) and ROUGE (Lin, 2004) have long been standard for evaluating generated text. BERTScore (Zhang et al., 2020) evaluates the similarity between two sentences by using a pre-trained BERT model to generate contextual embeddings for each token and calculating the sum of their cosine similarities. BARTscore (Yuan et al., 2021) evaluates the quality of generated text by calculating the weighted logarithmic probability of one text given another text. Chung et al. (2023) found that these metrics cannot be accurately evaluated for semantically equivalent expressions but syntactically different. Therefore, trying to evaluate harder-generated text requires more efficient methods. GPTScore (Fu et al., 2023) evaluates text quality through conditional generation probabilities, enabling customizable assessments using natural language instructions. However, these methods all require reference answers for a stable evaluation.

B.2 LLM-as-a-Judge

The advancement of LLMs has made LLM-as-a-judge possible, where an LLM-based evaluator judges the model responses based on user instructions and the criteria from different aspects of the prompt. The evaluation is flexible as the evaluation criteria can be adjusted dynamically in the prompt instead of retraining the evaluator from scratch. The judgment can be interpretable and detailed criticism as well as feedback are generated before the judge score. The LLM-based evaluator is more efficient and scalable than human experts at much less expense. It is more effective and applicable than statistical evaluation metrics like BLEU and embedding-based metrics like BERTScore.

Now there are three different categories of LLM judges: The Point-wise judge model (Kim et al., 2024a; Lin & Chen, 2023; Dong et al., 2024) evaluates individual responses independently, scoring them based on predefined criteria; The pair-wise judge model(Li et al., 2024b; Kim et al., 2024b; Jung et al., 2024) compares two responses directly, determining which one is superior based on a set of evaluation dimensions; The List-wise judge model(Liu et al., 2025) ranks multiple responses in terms of quality, assigning a rank to each based on the overall evaluation. Researchers explore effective approaches to apply LLMs as judges by prompt-based or finetuning-based methods in various scenarios. The prompt-based approaches aim to enhance judgment via step-by-step instructions as well as multi-turn optimizations. G-Eval (Liu et al., 2023) evaluates NLG output quality using Chain-of-Thought (CoT) and Form-Filling paradigms. Portia (Li et al., 2024f) proposes to evaluate the responses with step-by-step reasoning guidelines. Active-critic (Xu et al., 2024) first generates evaluation criteria from data and iteratively refines them. However, the performance of prompt-based LLM evaluators is unsatisfactory (Gu et al., 2024). The finetuning-based approaches further optimize the LLMs with the specialized dataset to adapt the LLMs to judgment tasks either with instruction tuning (Ye & Ng, 2024; Li et al., 2024b) or

Direct

Reasoning

Task Type

0.25

0.30

0.35

0.40

0.45

0.50

0.55

Spearman Correlation Coefficient

0.327

0.446

0.224

0.304

0.395

0.438

0.475

0.474

0.497

0.499

0.504

0.533

(a)

Direct

Reasoning

Task Type

0.401

0.458

0.240

0.314

0.479

0.457

0.509

0.488

0.535

0.524

0.479

0.511

(b)

E-Score Performance

Direct

Reasoning

Task Type

0.491

0.476

0.369

0.336

0.504

0.472

0.535

0.522

0.546

0.532

0.572

0.567

(c)

LAGER(w. tuning) Performance

LLaMA3.1-8B-Instruct

InternLM3-8B-Instruct

Qwen-2.5-14B-Instruct

Mistral-Small-24B-Instruct

LLAMA-3.3-70B-Instruct

Spearman Correlation for Qwen2.5 Models (reasoning condition)

0.5B

1.5B

3B

7B

14B

32B

72B

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Flask

Vanilla Score

E-Score

LAGER(w, tuning)

0.5B

1.5B

3B

7B

14B

32B

72B

0.0

0.1

0.2

0.3

0.4

0.5

0.6

HelpSteer

0.5B

1.5B

3B

7B

14B

32B

72B

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Spearman Correlation Coefficient

preference tuning (Ye et al., 2024b; Hu et al., 2024b). InstructScore (Xu et al., 2023) combines human guidance with implicit knowledge from models like GPT-4, finetuned on the LLaMA model, to provide both scores and human-readable diagnostic reports. TIGERScore (Jiang et al., 2023b) is finetuned on the LLaMA-2 model to generate detailed error analysis, helping users understand each identified error and its associated penalty score. Prometheus (Kim et al., 2024a) and Prometheus 2 (Kim et al., 2024b) are finetuned for fine-grained evaluation, achieving new state-of-the-art performance on judgment benchmarks. In this work, we focus on the single-LLM point-wise evaluator without a reference response. Our proposed LAGER method improves judgment performance by estimation with aggregated layer-wise logits, which is different from the idea of prompt-based and finetuning-based methods.

C Addition Experimental Results

In this section, Figure 6 shows the comparison of the performance of different scoring methods across models on direct and reasoning conditions. Figure 7 shows the comparison of spearman correlation coefficients for Qwen2.5 models across FLASK, HelpSteer and BIGGen Bench Using Vanilla Score, E-Score, and LAGER (w. tuning) under the reasoning condition. In Table 5, we present the Pearson correlation coefficients for all models, including LAGER and other baselines, on the Flask, HelpSteer, and BiGGen Bench.

D Addition Applications of LAGER

D.1 Emotional Understanding

LLMs have demonstrated strong capabilities to comprehend and interpret complex emotions and their meanings in social contexts. Therefore, beyond merely assessing the quality of responses, LLM can also be used as an emotion judge. In the widely used emotion understanding benchmark EQ-Bench (Paech, 2024), EQ-Bench is a benchmark for language models designed to assess emotional intelligence. It uses a specific question format in which participants are required to read a conversation and then rate the intensity of one character's emotional

response. Each question is explanatory and aims to assess the ability to predict the intensity of four different emotions. The dataset originally contained 117 questions, and after processing, we obtain 430 instances, each corresponding to one emotion per conversation. The emotional intensity annotated by humans ranges from 1 to 9, with higher scores indicating stronger emotions. The LLM is asked to predict the intensity of the emotional states of characters in a dialogue with a score ranging from 1 to 9. In this part, we evaluate LAGER on EQ-Bench and report the consistency with human evaluations.

As shown in Table 6, LAGER significantly outperforms the widely used vanilla score across all LLM backbones and consistency metrics, with an average improvement of 8.42 points without tuning and 9 points with tuning. This demonstrates that LAGER is effective not only for evaluating the quality of responses but also in other LLM-as-a-Judge scenarios, such as emotional understanding. LAGER is tuned on the HelpSteer validation set, which has a different scoring domain and range compared to EQ-Bench (ranging from 1-5 versus 1-9). This further highlights the robustness of LAGER with tuning, as it successfully transfers from one domain and score range to other domains with different judging prompts and score ranges .

D.2 Enhancing LLM in Knowing When They Don't Know

Knowing when they don't know, also known as self-knowledge (Yin et al., 2023), is crucial for enhancing the reliability and trustworthiness of LLMs. In this section, we explore whether LAGER can improve an LLM's self-knowledge. We evaluate the SelfAware dataset (Yin et al., 2023) using LLaMA3.1-8B-Instruct and Mistral-7B-Instruct-v0.3, which consists of 2337 answerable and 1032 unanswerable questions from five diverse

categories. Specifically, given a subjective question in SelfAware, we ask an LLM to evaluate the answerability of this question on a scale of 1 to 5. A score of 1 indicates that the question is completely unanswerable based on the available knowledge, while a score of 5 indicates a highly accurate answer is possible. Following the original paper, we normalize the certainty score to a range of 0-1 and use a threshold of 0.75 as the boundary to classify LLM as either "know" ( ≥ 0.75) or "don't know" ( < 0.75).

Then, we can quantitatively calculate the F1 score to measure the model's level of self-knowledge.Table 7 compares LAGER with baseline methods, including the SimCSE baseline used in the original paper. The results show that LAGER (w. tuning) significantly enhances the LLM's self-knowledge, achieving a +19.7% and +1.9% increase in F1 score compared to SimCSE, and VScore on LLaMA-3.1-8B-Instruct, respectively, and +11.1%, and +4.8% increases in F1 score compared to SimCSE and VScore on Mistral7B-Instruct-v0.3, respectively. By providing a more

reliable measure of LLM's self-knowledge, LAGER enhances the LLM's ability to understand their limitations, shows performance comparable to the E-score.

E Experiment Details

E.1 Details About KDE for Score Distribution Visualization

We adopt kernel density estimation (KDE) to visualize the distribution of model-generated scores across evaluation samples. KDE provides a smooth, non-parametric estimate of the underlying score distribution and is preferable to histograms for highlighting fine-grained density differences.

Given a set of samples { x 1 , x 2 , . . . , x n } , the estimated density at point x is computed as:

where K ( · ) is the kernel function and h is the bandwidth. We use the Gaussian kernel:

All KDE plots are normalized to unit area for comparability. We employ kernel density estimation for visualization, with the bandwidth parameter selected using Silverman's rule or cross-validation when appropriate. KDE is used in Figure 4 to compare the score output distributions of LAGER and baseline models.

E.2 Details About the Benchmarks

FLASK We utilize the complete test prompt set from FLASK. (Ye et al., 2024a), a fine-grained evaluation dataset that includes various conventional NLP datasets and instruction datasets. This dataset contains data

on 2,001 entries, each consisting of a human feedback score and an evaluation score from GPT-4. We utilize 12 scoring rubrics: Conciseness, Metacognition, Insightfulness, Readability, Commonsense Understanding, Logical Robustness, Factuality, Comprehension, Completeness, Logical Efficiency, Logical Correctness, and Harmlessness.

HelpSteer HelpSteer (Wang et al., 2023) is an open-source Helpfulness Dataset. For each response in the dataset, it is evaluated based on five criteria: helpfulness, correctness, coherence, complexity, and verbosity. These evaluation scores are annotated by humans, ensuring that each instruction in the dataset is paired with responses of varying quality. We process these data, combining the prompt, response, and corresponding criteria into individual data samples. A total of 8.95k data points are generated, with the first 2k used for our evaluation.

BiGGen Bench BiGGen Bench (Kim et al., 2024b) is a comprehensive evaluation benchmark designed to assess the capabilities of large language models (LLMs) across a wide range of tasks. This benchmark focuses on free-form text generation and employs fine-grained, instance-specific evaluation criteria. BiGGen Bench evaluates nine distinct generation capabilities (e.g., instruction following, reasoning, tool usage, etc.) across 77 tasks, providing model outputs and scores for 103 different language models. We utilize the human evaluation test set.

E.3 Details About the Baselines

Due to the limitations of the GPTScore method, we do not report its results under the reasoning condition. Since TIGERScore-7B and Prometheus2-7B cannot directly output a single score, we do not report their results under the direct condition.

TIGERScore TIGERScore(Jiang et al., 2023b) is a trained metric designed for explainable, referencefree evaluation of text generation tasks. Unlike other automatic evaluation methods that only provide scores, TIGERScore uses natural language instructions to guide error analysis, identifying specific mistakes in the generated text. It is based on the LLaMA-2 model and trained on a carefully curated dataset, MetricInstruct, which includes 42K quadruples covering 6 text generation tasks and 23 datasets. TIGERScore is capable of generating detailed error analysis, including the error location, error type, error explanation, and revision suggestions, along with a penalty score for each error.

GPTScore Assuming the text to be evaluated is h = { h 1 , h 2 , · · · , h m } , The core of the GPTscore method lies in leveraging LLM to evaluate the quality of text by computing the average log probability 1 m ∑ m t =1 log p ( h t | h <t , T ( d, a, S ); θ ) under a specific context S and task instruction T ( d, a, S ) . The prompt template T ( · ) is composed of a task description and aspect definition.

Prometheus-2-7B The process of constructing the Prometheus-2 model (Kim et al., 2024b) involves the following steps: First, a fine-grained pairwise ranking dataset, PreferenceCollection, containing 1000 custom evaluation criteria, is created to support evaluation based on user-defined standards. Next, Mistral-7B is selected as the base model, and it is trained separately on the direct evaluation dataset FEEDBACK COLLECTION and the pairwise ranking dataset PreferenceCollection. Finally, by merging the weights of the models trained on these two formats, the final Prometheus-2 model is obtained.

E.4 Details of Evaluation Metrics

PEARSON Pearson correlation coefficient indicates the strength and direction of the relationship between two variables, ranging from -1 to 1. For the two sets of data x,y, we can calculate their Pearson correlation coefficients ρ XY as follows:

SPEARMAN Spearman rank correlation coefficient is a non-parametric measure that evaluates the statistical dependence between the ranks of two variables. It determines how well the relationship between these variables can be described using a monotonic function. We can calculate Spearman r s according to the following formula:

where d i represents the rank difference between the two variables for each observation (pairwise difference), and n denotes the number of fraction pairs.

Since the Spearman correlation coefficient effectively measures nonlinear monotonic relationships, is robust to outliers, and does not require data to follow a normal distribution, it is particularly suitable for handling ordinal data. Therefore, this work primarily presents results based on the Spearman correlation coefficient.

E.5 Details of layer-wise weight training settings

We randomly select 1,000 samples from the HelpSteer dataset as a held-out validation set, ensuring that it does not overlap with the test set, to tune the layer-wise weights of LAGER. The training is performed using the Adam optimizer with an initial learning rate of 0.01, and a batch size of 4. A random seed of 42 is set to ensure the reproducibility of the experiment. We also apply the ReduceLROnPlateau learning rate scheduler with a decay factor of 0.5, a patience value of 1, and a minimum learning rate specified by min_lr . Since some models in the Qwen2.5 family may not converge with just one epoch, we set the number of training epochs to 2 for all models in the Qwen2.5 series. For other backbone models, we set the number of training epochs to 1. All experiments are implemented using PyTorch.

F Details of SFT

For LAGER, E-Score and VScore, we establish seven dimensions (including answer accuracy, logical consistency, relevance, fluency and clarity, length appropriateness, diversity, and instruction difficulty ) and utilize the average score of each dimension, evaluated using the LLaMA3.1-8B-Instruct model, for data selection. The specific definitions and scoring standards for these seven dimensions can be found in Table G. SuperFiltering is an explicitly designed instruction data filtering method. We strictly follow the original paper's methodology, except for using LLaMA3.1-8B-Instruct as the backbone model for fair comparison. During the SFT training of the Llama-3-8B model, we use 4 L40 GPUs, and the total batch size during the training phase is 64. The specific training parameters and settings are detailed in Table 8.

G Prompt Template

In this section, we present all the prompt templates used in this paper, with the blue text indicating the parts where corresponding content should be inserted. For the direct evaluation scenario, Figure 9 is the prompt template for direct evaluation in a point-wise scenario that outputs only a single score. Figure 10 is the prompt template for reasoning evaluation in a point-wise scenario, including feedback(reasoning).

Prompt Template for Emotional Understanding

Yourtaskistopredictthelikelyemotionalresponseofacharacterinthisdialogue:

[dialogue]

[Enddialogue]

At the end of this dialogue,rate how strongly[human] is likely feeling[emotion_type]on a scale

You only need tooutput your rating,with no additional commentary:

### Task Description:

An instruction (might include an Input inside it), a response to evaluate and a score rubric representing a evaluation criteria are given.

- Write a detailed feedback that assess the quality of the response strictly based on the given score rubric, not evaluating in general.

- After writing a feedback, write a score that is an integer between 1 and 5. You should refer to the score rubric.

### The instruction to evaluate:

{instruction}

### Response to evaluate:

{output}

### Score Rubrics:

{Definition}

- Score 1: {criteria_for_1}

- Score 2: {criteria_for_2}

- Score 3: {criteria_for_3}

- Score 4: {criteria_for_4}

- Score 5: {criteria_for_5}

### Output format:

The output format should look as follows:

'Score: <give your score here>'.

Remember that you only need to provide the score. There is no need to generate explaination for it.

Prompt Template for Point-wise Direct Evaluation

### Task Description:

An instruction (might include an Input inside it), a response to evaluate and a score rubric representing a evaluation criteria are given.

- Write a detailed feedback that assess the quality of the response strictly based on the given score rubric, not evaluating in general.

- After writing a feedback, write a score that is an integer between 1 and 5. You should refer to the score rubric.

### The instruction to evaluate:

{instruction}

### Response to evaluate:

{output}

### Score Rubrics:

{Definition}

- Score 1: {criteria_for_1}

- Score 2: {criteria_for_2}

- Score 3: {criteria_for_3}

- Score 4: {criteria_for_4}

- Score 5: {criteria_for_5}

### Output format:

The output format should look as follows: \"

Feedback: <write your feedback here> Score: <give your score here>

\'

Remember that you should end with the the score. Please do not generate any other opening, closing, and explainations.'''

Prompt Template for Point-wise Reasoning Evaluation

You are an evaluator tasked with assigning a score from 1 to 9 based on the criteria below, to assess the quality of a specific aspect of the data entry.

### Evaluation Task:

Assess the given data entry based on the following aspect:

**{evaluation_aspect}**. For this task, your goal is to evaluate how well the **{evaluation_aspect}** is demonstrated in the provided **Instruction** and **Response**. Be sure to consider

the following:

- **{evaluation_aspect}** refers to a specific quality or characteristic in the data entry.

- Consider both the **Instruction** (the directive given) and the **Response** (the output provided in answer to the instruction).

- The evaluation should be based on how clearly, accurately, or relevantly the task is fulfilled according to the criteria listed below.

- Only assess the given **Instruction** and **Response**; do not consider any additional context or external information.

### Definition of

{evaluation_aspect}:

- **{evaluation_aspect}** is {Definition}

### Scoring Criteria:

- **Score 1**: {criteria_for_1}

- **Score 2**: {criteria_for_2}

- **Score 3**: {criteria_for_3}

- **Score 4**: {criteria_for_4}

- **Score 5**: {criteria_for_5}

- **Score 6**: {criteria_for_6}

- **Score 7**: {criteria_for_7}

- **Score 8**: {criteria_for_8}

- **Score 9**: {criteria_for_9}

Each score corresponds to a specific level of quality, from 1 (lowest) to 9 (highest). Use the provided criteria to select the appropriate score.

### Data Entry to Evaluate:

- **Instruction**: {instruction}

- **Response**: {output}

- Provide a **single score** from 1 to 9 in format of <score: <the score you give>> based on your evaluation of the **Instruction** and **Response**.

Prompt Template for Data Filtering

Your task is to evaluate whether a question can be answered and provide an answerable level, represented by a number between

1 and 5.

Here are the question and evaluation criteria:

<question>

{question}

</question>

<evaluation criteria>

Level 1:

No relevant knowledge is available; the question cannot be answered at all.

Level 2:

Limited or outdated knowledge; the answer is either too general or speculative, lacking precision.

Level 3:

Moderate knowledge is available; the answer can be constructed but lacks depth or clarity.

Level 4:

The answer is mostly accurate and comprehensive, but may have small details missing or require minor clarification.

Level 5:

Fully answerable with certainty; clear, precise, and complete response based on solid knowledge.

</evaluation criteria>

Provide answerable level based on the evaluation criteria. Your response should be in the format of

"Answerable Level: <An

integer from 1 to 5 showing how answerable the question is>".

Do not generate other content.

Prompt Template for SelfAware

Continued on next page

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces an approach that utilizes the internal hidden states, specifically, cross-layer logits, of a judge LLM to produce evaluation scores, moving beyond the conventional reliance on final-layer token probabilities. The method involves tuning L + 1 scalar weights on a held-out validation set, keeping the LLM backbone entirely frozen. The authors assert that these learned weights generalize across different benchmark datasets. The core experiments focus on comparing the model’s correlation with human judgments across three evaluation benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- The writing of the paper is clear and easy to follow.\n- The main goal of the paper itself is interesting and potentially useful in building a more effective LLM-as-a-judge system.\n\n### Weaknesses\n- The method lacks novelty. As the authors themselves acknowledge, the idea of using internal hidden states has been explored extensively since the introduction of BERT.\n- My key concern is whether the lightweight training of layer weights can truly generalize across diverse evaluation benchmarks without additional fine-tuning. Fig. 2 illustrates that the correlation with human scores fluctuates significantly across layers, suggesting potential dataset dependency. This argument would be stronger if tested on a broader and more diverse set of benchmarks. The current evaluation, limited to three datasets, does not convincingly establish generalizability.\n- Increasingly, modern evaluation benchmarks are label-scarce, more complex, or dynamically generated without human-annotated scores. It is unclear whether the proposed method is applicable or effective in such settings where scalar labels are unavailable or ill-defined.\n- The work focuses exclusively on evaluations that assign discrete integer scores within a predefined range. It does not explore more general evaluation scenarios, such as those involving natural language feedback or descriptive labels. It remains uncertain whether the proposed method would be robust and adaptable in these more open-ended evaluation settings.\n\nQUESTIONS:\n- As noted in the weaknesses, have you explored the method’s performance in more general or open-ended evaluation settings beyond predefined integer scores?\n- Do you have additional experimental results on recent benchmarks that are very different from the 1,000 samples of the HelpSteer dataset used for training?\n\nLIMITATIONS:\nTheir limitations should also acknowledge the narrow scope of benchmarks used, as well as the restricted evaluation scenario that assumes predefined, discrete integer score ranges.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

comments: SUMMARY:
This paper proposes LAGER, a framework for improving LLM-as-a-judge performance by aggregating score-token logits from all decoder layers of a LLM. Sepcifically, LAGER computes a weighted combination of layer-wise distributions and taking the expected score. Experiment results demonstrate the improvements over several LLM judge baseline methods.

STRENGTHS AND WEAKNESSES:
**strengths**  
1. The multi-layer aggregation approach is very simple and straightforward, using frozen backbones without modifying core LLM parameters.
2. Empirical results demonstrate improvements across several selected benchmarks and model scales.

**weaknesses**  
1. The proposed method relies on accessing the internal representations of the LLM, thus this limits applicability to open-source models; closed-source or API-only LLMs cannot benefit.  
2. The supervised weight-tuning step still requires human-annotated validation data. 
3. The experimental results are not thorough, missing important baselines, for example RewardBench and DPO rewarding. Also, LLM-as-a-Judge heavily rely on the prompt templates, so it would be better to test on different popular prompts which you can find in recent papers such as self-rewarding LLM.

QUESTIONS:
1. Will LAGER become better if scale more training data and params?
2. Please also see the weaknesses.

LIMITATIONS:
NA
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 1
DATE: 2025-07-01

comments: SUMMARY:
The paper proposes LAGER (Layer Aggregation for Enhanced Representation), a lightweight framework to improve the alignment of LLM-based evaluators ("LLM-as-a-judge") with human judgments. Instead of relying solely on the final output layer (which may overlook nuanced signals), LAGER aggregates internal representations from multiple intermediate layers of frozen LLMs to generate finer-grained, probabilistic evaluation scores. On standard alignment benchmarks, this approach is reported to enhance judgment quality by up to 7.5% over the best baseline, and also lead strong downstream applications.

STRENGTHS AND WEAKNESSES:
Strengths:

- The paper researches into LLM-as-a-judge, a popular topic within the NeurIPS community.

- The presented approach has a high efficiency in that it works under frozen backbone LLMs and requires minimal computational cost during training.

- 3 human-annotated datasets are chosen for evaluation to avoid preference leakage, making the results robust and reliable.

Weaknesses:
- In spite of the discussed in Section 3.4, the approach is mainly restricted to open-sourced models with accessible intermediate layers for idealistic performance, so its applicability is still limited.

- The evaluation is limited to point-wise benchmarks, while the pairwise/listwise evaluation are not involved.

QUESTIONS:
Apart from the weaknesses, I want to see a bit more insights in why reasoning leads to weaker results for LAGER (with supports from concrete examples if possible).

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nLAGER introduces a training-free (optionally tunable) way to boost LLM-as-a-judge quality by looking inside the model: it linearly combines the logits that every decoder layer assigns to each candidate score and outputs the expectation of that cross-layer distribution. Because middle-to-upper layers encode semantics that correlate more closely with human opinions, this fusion yields finer-grained, better-calibrated judgements than the usual “take the top token” heuristic. Tested on three alignment benchmarks, namely, Flask, HelpSteer and BIGGen, LAGER raises Spearman agreement with human scores by up to 7.5 % over the strongest existing baseline, while requiring no chain-of-thought prompting. It further enhances tasks such as instruction-data selection and emotional understanding, all while maintaining the backbone's frozen state for speed and portability.\n\nSTRENGTHS AND WEAKNESSES:\nPros:\n- LAGER (esp. the one with fine-tuning) significantly improves alignment with human preferences. Its innovative cross-layer scoring yields up to a 7.5 percentage-point Spearman correlation boost over GPTScore, VScore, and E-Score across three benchmarks.\n- The method keeps the backbone model frozen, training only a small set of fusion weights (or none in the zero-tuning variant). This means inference is as efficient as a single forward pass, and it avoids the expensive reasoning prompts used by many judges.\n\nCons:\n- The paper does not compare LAGER against several relevant evaluators that also target human alignment, such as G-Eval (reasoning-based) or HD-Eval (which also leverages multi-layer information). These comparisons are important to contextualise LAGER’s gains.\n- GPT-4o-mini achieved very strong results, raising questions about the impact of model scale. However, the current experiments focus only on cross-family comparisons. It would strengthen the work to test LAGER across different sizes within the same family (e.g., Qwen 2.5B, 7B, 14B).\n- While LAGER with tuning performs best overall, the tuning appears to succeed on some backbones (e.g., Qwen) but fails or regresses on others (e.g., Mistral, LLaMA). Yet the discussion (p. 6) focuses only on LAGER w/ tuning (without comparing to the ones w/ tuning), making the discussion a bit insufficient.\n- Compared to E-Score, LAGER without tuning performs worse (or on-par) on several backbones (e.g., InternLM, Qwen), suggesting that the observed gains largely stem from fine-tuning. This makes the comparisons not quite fair, comparing a tuned model to unturned baselines.\n\nReferences:\nYang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Singapore. Association for Computational Linguistics.\nYuxuan Liu, Tianchi Yang, Shaohan Huang, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, and Qi Zhang. 2024. HD-Eval: Aligning Large Language Model Evaluators Through Hierarchical Criteria Decomposition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7641–7660, Bangkok, Thailand. Association for Computational Linguistics.\n\nQUESTIONS:\nSee my comments above.\n\nLIMITATIONS:\nSee my comments above.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-21"}