Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry
Authors: Zhuochun Li
Yong Zhang
Ming Li
Yuelyu Ji
Yiming Zeng
Ning Cheng
Yun Zhu
Yanmeng Wang
Shaojun Wang
Jing Xiao
Daqing He
Venue: ICLR 2026 Poster (2026)
Keywords: Evaluation
Large Language Models Probing
Interpretability
Primary area: interpretability and explainable AI
TL;DR: We introduce INSPECTOR, a probing-based framework that uses internal embeddings of small LMs to approximate evaluation scores from stronger LLMs across multiple aspects.

Abstract: Large language models (LLMs) are widely used as reference-free evaluators via prompting, but this “LLM-as-a-Judge” paradigm is costly, opaque, and sensitive to prompt design. In this work, we investigate whether smaller models can serve as efficient evaluators by leveraging internal representations instead of surface generation. We uncover a consistent empirical pattern: small LMs, despite with weak generative ability, encode rich evaluative signals in their hidden states. This motivates us to propose the Semantic Capacity Asymmetry Hypothesis: evaluation requires significantly less semantic capacity than generation and can be grounded in intermediate representations, suggesting that evaluation does not necessarily need to rely on large-scale generative models but can instead leverage latent features from smaller ones. Our findings motivate a paradigm shift from LLM-as-a-Judge to Representation-as-a-Judge, a decoding-free evaluation strategy that probes internal model structure rather than relying on prompted output. We instantiate this paradigm through INSPECTOR, a probing-based framework that predicts aspect-level evaluation scores from small model representations. Experiments on reasoning benchmarks (GSM8K, MATH, GPQA) show that INSPECTOR substantially outperforms prompting-based small LMs and closely approximates full LLM judges, while offering a more efficient, reliable, and interpretable alternative for scalable evaluation. The code and data are available at: https://github.com/zhuochunli/Representation-as-a-judge

Published as a conference paper at ICLR 2026
RETHINKING LLM-AS-A-JUDGH REPRESENTATION- AS- A- PELA-JUDDE JUDGE WITH SMALL LANGUAGE MODELS VIA SEMANTIC CAPACITY ASYMMETRY
Zhuochun Li122, Yong Zhang Ming Li', Yuelyu Ji,, Yiming Zeng'. Ning Chengh,, Yun Zhu', Yanmeng Wang'. Shaojun Wang! Jing Xiao', Daqing He
Ping An Technology (Shenzhen) Co., Ltd. University of Pittsburgh University of Maryland, College Park *University of Connecticut
{2h1163, yuj49, dah44) (zhangyong203, chengningg21/epingaae com. en (mingl f edu {yiming zeng) @uconn edu
ABSTRACT
Large language models (LLMs) are widely used as reference-free evaluators via prompting, but this "LL.-as-a-Judde" paradigm is costly, opaque, and sensitive to prompt design. In this work, we investigate whether smaller models can serve as efficient evaluators by leveraging internal representations instead of surface gen- eration. We uncover consistent empirical pattern: small LMs, despite with weak generative ability, encode rich evaluative signals in their hidden states. This mo- tivates us to propose the Semantic Capacity Asymmetry Hypothesis: evaluation requires significantly less semantic capacity than generation and can be grounded in intermediate representations, suggesting that evaluation does not necessarily need to rely on large-scale generative models but can instead leverage latent fea- tures from smaller ones. Our findings motivate a paradigm shift from LLM-as- a decoding-free
a-Judge to pnvreseetattio--s-a--iiieee
evaluation strategy that probes internal model structure rather than relying on prompted output. We in- stantiate this paradigm through INSPECTOR, probing-based framework that predicts aspect-level evaluation scores from small model representations. Exper- iments on reasoning benchmarks (GSM8K, MATH, GPQA) show that INSPEC- TOR substantially outperforms prompting based small LMs and closely approxi- mates full LLM judges, while offering a more efficient, reliable, and interpretable alternative for scalable evaluation. The code and data are available at: https: //q.ttub. thub. wwaam///uocccuuuu // ion- as-a judge
INTRODUCTION
Large language models (LLMs) have demonstrated remarkable capabilities in generation, reasoning, and alignment tasks (Achiam et al., 2023) Touvron et al., 2023). A growing number of works leverage the paradigm of LLM-av-a-Judye wherein powerful LLMs are prompted to assess the quality of generated outputs without access to ground-trus truth references (Chang et al. 2024 Prasad al. 2023). This approach has achieved strong empirical results in reference-free evaluation across domains such as summarization and complex reasoning (He et al. 2023 Zhang et al. 2024).
However, this prompt-based evaluation paradigm has important limitations. First, requires autore- gressive decoding, making it computationally expensive even for single-point evaluations. Second, it relies on large proprietary models (e.g., GPT-4), whose internal mechanisms remain opaque and unverifiable. Lastly, its effectiveness depends heavily on prompt engineering, raising concerns about reproducibility, robustness, and scaling (Polo et al. 2024 Voronov et al. 2024 Mizrahi al. 2024)
Equal contribution.
Corresponding author.
Work was done during Zhunchun Li's internship Ping Technology.

---

Published as a conference paper at ICLR 2026
LLM-as--aJuudgee
B Acturato
Rassurca-iitennie
SLM-as-a-Judge
Inaocurato
Reesurce- Efficeet
Representation-as-a--uoo
Accurato Efficinnt
Proting
Figure 1: Illustration of Representation-ss-a-Jddgge
natural goal use smaller open-source LMs evaluators, they offer more lightweight and accessible alternative to large proprictary models. However, when prompted directly, their evalu- ation performance is poor and highly inconsistent compared large LLMs. Prior work (Li al. 2024a Waldis et al., 2024) has shown that small models, despite weaker generation, often possess large models. This suggests that their poor evaluation perfor- mance may stem from limitations in surface generation rather than fundamental lack of understand- ing. Building on this, we ask a more granular question: do small models encode evaluation-relevan signals in their internal representations, even when generation is poor?
semantic competence comparable
This question represents broader insight: the ability to evaluate may place lower demands on semantic capacity than the ability to generate. Even when generation fails, compact internal rep- resentations may already encode the features needed for judgment. We therefore formalize the Se- mantic Capacity Asymmetry Hypothesis: The semantic capacity required for accurate evaluation is significantly lower than that required for generation. Evaluation can be grounded in compressed internal representations of small language models, even when generation requires full-mode decod- ing.
Building on this hypothesis, advance an alternative perspective evaluation: Representation- as-a-Judge. Rather than relying on prompted text generation, evaluative signals can be extracted directly from the latent structure. This addresses key bottlenecks of prompt-based evaluation and opens the door to lightweight, interpretable, and scalable systems.
To explore this perspective, we introduce INSPECTOR INternal Signal Probing and EvaluaTion Of Representations), a probing-based framework for reference- fference-frre evaluation. Given (prompt, response) pair, INSPECTOR performs the following steps: (1) obtain aspect-level evaluation scores from a strong LLM judge across multiple aspects (e.g., logicality, fluency, consistency); (2) input the same evaluation prompt to a small LM and extract internal representations; (3) identify informative layers and train lightweight classifiers to approximate evaluation scores using latent embeddings.
Empirically, we validate this framework on reasoning benchmarks such as GSM8K, MATH, and GPQA, and find that internal representations from small models (c.g., 1.7B) achieve high predictive performance, substantially outperforming prompting-baid based baselines and in many cases approach- ing the fidelity of full-scale LLM judges. Furthermore, method can be used to filter noisy reasoning data, leading to measurable gains in downstream super- vised fine-tuning (Xu et al. 2024 Albalak et al., 2024 Li et al. 2024b).
show that classifiers trained with this
Our contributions are threefold:
We identify and analyze a consistent empirical phenomenon: small LMs, despite weak generation, encode evaluation-reled relevvan signals in their internal representations.
We formalize this insight as the Semantic Capacity Asymmetry Hypothesis, positing that evaluation requires less semantic capacity than generation and can be grounded in intermediate representations.
We advance a new perspective, RepresentatiionasseeJJdgg and instantiate through our probing- based framework, INSPECTOR. demonstrating high-fidelit evaluation and effi- cient data filtering for reasoning tasks. To the best our knowledge, this is the first work to investigate I.L. probing for evaluation tasks.
2

---

Published as a conference paper at ICLR 2026
Representation
Quality?
Q
Flunncy
Input
Labels
Fictuality
Probing Classifierd
Response
Response
E
Figure 2: Overview of our proposed INSPECTOR. We freeze the small LMs and prohe their rep- resentations, training only lightweight probing classifier to fit the proprietary LLM evaluations.
2
RELATED WORK
Our work builds on two research directions: (i) data evaluation methods, particularly the emerging LLM-as-a-Judge paradigm, and (ii) probing techniques for analyzing LLM representations. Below we summarize progress in each line and position our contribution.
Recent studies NLP have introduced reference-based evaluation metrics such BERTScore (Zhagg et al. 2019) and BARTScore (Yuan et al. 2021), which leverage pretrained language models to better align with human judgments. ROSCOE further develops unsupervised, reference-free metrics that outperform traditional n-gram methods like ROUGE (Lin 2004) and
LLM Evaluation
BLEU (Papineni et al.) 2002), as well as several model-based metrics. With the emergence LLMs, many works have begun to treat LLMs themselves evaluators (Zheng et al. 2023 Liu al. 2023
Liet al. 2024c), 20224c, identifying and filtering high-quality data samples. A variety of techniques enhance this LLM-as-a-Judde paradigm: Chain-of-Thought reasoning (Wei et al. 2022) to improve decision
quality, RECEVAL (Prasad et al. 2023) to assess reasoning chains via correctness and informative-
ness, and SOCREVAL (He al. 2023) to let LLMs generate their own answers before evaluation.
However, these methods often rely on prompt engineering and proprietary LLMs, limiting both their reliability and interpretability. Alternatively, we propose to probe and analyze critical internal representations of small LMs, training lightweight classifiers to approximate state-of-the-art evalua- tions from powerful LLMs. This yields a more efficient and explainable approach to data evaluation.
Probing LLM Representations There is growing interest in probing the internal representations of LLMs uncover interpretable features. Early work by (hi et al. (2016) introduced probing by training a logistic regression classifier on top of machine translation encoders to study the extent of syntactic information. Building this idea, later studies investigate more complex linguistic phenomena. For example, [Starace et al. (2023) examine how linguistic categories such as part-of- speech (POS) and dependency relations are jointly encoded across layers of LLMs, revealing shared and hierarchical structures in their representations. Beyond linguistic categories, recent research has explored whether LLMs capture higher-level abstractions of world knowledge and state. Zhang al. (2025) propose Sentinel, which probes decoder attention in small proxy LLMs to extract relevance signals for context compression, framing probing as a lightweight understanding task rather than linguistic diagnostic. More works further probe the internal representations of world states in Transformer models when processing game scripts and embodied sequences (Li et al. 2023 Jin & Rinard 2024 Di Palma et al., 2025).
In contrast to prior work, which mainly seeks to understand what knowledge LLMs encode, we leverage probing in new direction: extracting critical internal representations that are predictive of evaluation quality. To the best of our knowledge, this is the first work to bridge probing with the LLMeas-a-Judge paradigm, enabling evaluation that is both more efficient and more interpretable.
3 METHODOLOGY
The Semantic Capacity Asymmetry Hypothesis suggests that evaluation can be grounded in com- pact intermediate representations small LMs, without requiring full decoding. To operationalize and test this idea, we propose directly probes latent semantic structure rather than relying on surface-level outputs. We instantiate this paradigm through INSPECTOR. Illustrated in Figure2] it consists of three components: LLMs evaluation annotation, small LMs probing, and building probing classifiers, which we will discuss in Sec. 4.1, 4.2, and 4.3, respectively.
new evaluation paradigm that
3

---

Published as a conference paper at ICLR 2026
3.1 LLMS EVALUATION ANNOTATION
Following established rubric definitions from prior work, including ROSCOE Golovneva et al. (2022) and Socreval He al. (2023), we adopt five widely used evaluation aspects K. to assess the quality of reference-free rationales, with the corresponding few-shot prompts provided in Ap- pendix
Semantic Consistency: Do the solution steps and final answer remain faithful to the prob- lem facts (no invented events, omitted givens, or unstated assumptions)?
Logicality: Does each inference and arithmetic step follow valid rules and correctly apply operations?
Does the rationale include the essential steps and intermediate calcula- tions needed to verify the final answer?
Informativeness:
Is the text grammatical, clear, and easy to follow, with proper punctuation, sen- tence flow, notation, and presentation?
Fluency:
Factuality:
Are the claims, facts, evidence, references, and concrete assertions in the tionale factually correct and supported?
Let denote task instruction and the corresponding model medel-geneeated response. We first employ a medium-scale language model Mmed (10-50B) to generate responses for benchmark queries. Unlike state-of-the-arr LLMs Mtarge (50-100B+), Mieed is less powerful, which is desirable be- cause produces more diverse evaluation distributions, containing both good and bad quality sam- ples, across different aspects, thereby facilitating the subsequent probing process. This will construct the response dataset D = {II; ys N ]i1 where x; is a benchmark question and ys = Mmeea (x;) is the corresponding response. For each evaluation aspect i K., construct aspect-specific in- struction I, and integrate (x, y) into it, yielding I& (z. y) as the evaluation prompt. We then query state-of-the-ard LLM Mingge to obtain evaluation scores along each aspect. For given sample (xr, y), we prompt Marge with Ia (x, y) and obtain a scalar score E [1, 5]:
Maarge (Zi.
Collecting these pairs yields the probing dataset
Perub U (Ia((E))) 81,k) - kEK
To avoid bias toward over-represented quality levels, we construct a balanced probing dataset for each aspect: we first determine the minimum number of samples among the five score levels (1-5), and then randomly downsample the other levels to this size. This ensures that the multiclass probing tasks are balanced across labels and not dominated by frequent scores.
Specifically, original scores (1-5) are treated as multiclass classification tasks, while responses with scores higher than threshold 7 are labeled high quality and the rest low quality, forming simpler binary classification task. Since these aspects have been validated and the effectiveness of powerful LLMSs has been extensively studied in prior work, we treat the resulting scores gold labels for our subsequent LM probing experiments.
3.2 SMALL LMS PROBING
We use probing to test whether small LMs encode linearly recoverable evaluative cues in their hid- den states, with the probe's minimal capacity ensuring that any predictive signal reflects the model's own semantics. In our tasks, we observe that prompt-based evaluation inference from small LMs Msmall (0-10B) yields a large gap compared to the gold scores from Miorges primarily due to dif- ferences in model capacity. To address this, instead of relying solely the final decoded text, we analyze internal representations layer by layer to identify those most predictive of the gold evaluation scores. Specifically, convert each prompt Ig (I,, y:) Dros into collection of per-layer, pooled representation features, and fit these features with simple, cross cross-validane validated linear probes. More ex- planations could be found in Appendix B
4

---

Published as a conference paper at ICLR 2026
Extraction and pooling. For a sample i, let St denote the sequence length and ..., S; index tokens, we imput prompt (x,, y4) and run Mamall in evaluation mode, obtaining per-layer hidden states H and attention weights A! From (f) compute small but expressive set of pooled vectors: mean, last, min, max, and concat. These pooling variants capture complementary token- level and global signals across layers. For example:
8,
Σ
E
H
ER
E F,mean
1=1
H
[6, E RR,
4 max token i non-pad}
Past
Attention and statistical features. For each head of layer we compute an attention-entropy statistic (f) iip and compress them into the number of attention heads R per layer:
Σ
stda (eth)
max 2,
(3)
R
h
For each pooled vector r we also compute compact statistics, including its norm norm(r), variance var(r), and entropy Err).
Feature assembly. For each layer and pooling type p E last max, wtw.:aaxx.ommoooooo coneat), we con- struct a feature matrix by concatenating multiple components along the feature dimension:
((.p) | (CA) [norm, var, E(.)) [4 max 6ihh iml
(e)
( N
statistics
where PCA, denotes optional dimensionality-reduccion depend on the data (imputer, PCA, scaler) are applied inside a cross-validation pipeline to avoid information leakage.
components. All transforms that
Probing and layer ranking. We treat gold labels 85,8 E {1.....5) from Dpoo as both (i) a multiclass prediction target vit = si, and (ii) a binary target y,, = 1[ r] with threshold (high vs. low quality). For each ((,p) we fit logistic probe and evaluate generalization with stratified tross-validation. Based on the fitting results, we rank peotttuool-eeature configurations by chosen criterion (binary multiclass performance) for different downstream tasks.
The probing stage produces: (I) ranked list of layer-pool-fcature configurations with cross- validated predictive performance, (2) per-layer progression plots that visualize where evaluative signal accumulates inside M These results inform the subsequent section, where we build final probing classifiers using the selected features.
3.3 BUILDING PROBING CLASSIFIERS
The probing stage produces a ranked list of layer-pool-feature configurations with predictive per- formance. In this section we describe how we use that ranked list to assemble multi-layer feature sets, train final probing classifiers, and select an optimal evaluator.
From ranked configurations to candidate layer sets. Let denote the ranked list of layer-pool-feature tuples produced after the previous probing stage. We take the Top-K unique layers and start from the highest-ranked single layer, iteratively adding the next-ranked layer only if the addition improves performance.
Feature concatenation for multi-layer probes. For a chosen subset layers S {(1 ... {s) and a pooling method we construct multi-layer feature matrix by horizontally concatenating the pay-layer features defined in Eq. equation Concretely, for sample we form the concatenated feature vector
((5.))
(fa) (82) Tr.p ti.
(8ss)) ] E R!SI-4, t.p
(5)
where dp the dimensionality of the pooled vector for pooling p. attention summaries are cluded, we append the per-layer attention summaries to form the final feature vector. The assembled dataset for N examples (S.p)
(Spp)
(S.p)

---

Published as a conference paper at ICLR 2026
Classifier training and selection. For each candidate feature assembly (S.s), train family of simple, interpretable classifiers to predict both the multiclass target ymuli and the binary target ghin. We select the final probing classifier by maximizing task-specific performance criterion over the candidate set S and prohe families. Formally, letting denote the set of classifier hyperparame- terizations tested, choose
p .²,00"
argmaxx a.S.neet, (S, EXXPXC. (S.p.clf)
where {bin, multi} denotes the classification task type (binary vs. multiclass), and 6* are the learned classifier parameters for the selected configuration. In case of ties, we prefer the configura- tion with smaller a (more stable performance) and with fewer layers.
This search yields a compact, high-performing probing classifier that uses small subset layers identified by the probing stage, which is tuned for robust generalization via cross-validation and grid search. It can serve as an efficient surrogate evaluator approximating Marge judgments while remaining orders of magnitude cheaper at inference time.
4 EXPERIMENTS
4.1
EXPERIMENTAL SETUP
Datasets. To prove the effectiveness on reasoning evaluation tasks, we picked three popular bench- marks in Mathematics & Science Tasks: GSM8K (Cobbe al. [02TT. J MATH (Hendryeks al., 2021 and GPQA (Rein et al. 2024). Datasets statistics are shown in Appendix
Models. In our experiments, we select -ama-338B--Instruue (Dubey et al. 2024) as Mmedd for re- sponse generation, as it achieves reasonable performance on the chosen benchmarks while produc- ing diverse evaluation scores. For LLM-based evaluation annotation, we employ DeepSeck-V3 (Liu ett al., 2024) as MMarge owing to its strong reasoning ability and relatively low cost. To validate the effectiveness of our pipeline across different small LMs, we consider Qwen3-0.6B (Yang et al.) 2025). Qwen3-1.7B (Yang al. 2025), LJama-332-1BB I-IItruttt (Dubey et al. 2024). and Llama-3.1-
18-Instruct (Dubey et al. 2024 as M.malll-
Baselines. To the best of our knowledge, this the first study to investigate LLM probing for evaluation tasks, and therefore related work is limited. Accordingly, we compare our approach against three baselines: (1) direct prompt-based inference on small LMs, (2) fine-tuning small LMs (Qwen3-0.6B), and (3) RoBERTa (Liu et al., 2019) on probing datasets to fit the evaluation scores.
Implementation Details. We set the score threshold r = 4, labeling samples with scores 2 4 as high-quality and those with scores < 4 as low-quality. For dimensionality reduction, we apply PCA with d = 50 in the probing process. We use K = 5 for Top-K aggregation and experiment with variety of classifiers, including Logistic Regression, Random Forests, small Multilayer Percep- trons (MLPs), and linear Support Vector Machines (SVMs). Since all pooling, layer, and classifier variants operate on cached hidden representations, exploring these configurations incurs negligible computational overhead. All test results are reported on the zero-shot prompt and weighted average Fl score. Additional implementations are provided in Appendix D
4.2 PROBING CLASSIFIERS RESULTS
We present the main results of our optimal probing classifiers across Mathematics & Science bench- marks in Figure 3 Explicit statistics for the probing datasets and the numbers of main results can be found in Appendix E Notably, our strong results are obtained by only training on small probing datasets (typically fewer than 100 samples per score) due to the downsampling strategy (3.l]
Probing is much more effective than prompt-based inference. Figure shows substantial im- provements of our methods over baselines, with average Fl scores increasing by more than 20% on most tasks. This suggests that poor outputs from LLMs do not necessarily imply that the models lack the underlying knowledge to solve the task. Instead, crucial information may be

---

Published as a conference paper at ICLR 2026
Masitaiss Elasstticaton Results (scomes 1-32
Legiciity
Stawwry
Risorry tright spusting
NOCANTS Canlurecy
wativenns
Figure 3: Weighted average Fl score (%) across reasoning benchmarks with multiclass classification and binary classification tasks. Our probing method (colored bars with hatch marks) significantly outperforms prompting on the same models (colored bars without hatch marks) across all tasks.
already embedded in the internal representations, but remains hidden after the final decoding process. Probing allows uncover and leverage these latent understandings, whereas direct text generation can introduce noise and degrade human-interpretabe fectiveness of our approach is consistently observed across all evaluation aspects and for different sizes of Mamall across multiple model families, including results after fine-tuning. This provides strong indication that our method can be applied to more general scenarios.
performance. Importantly, the ef-
Larger LLMs do not necessarily provide stronger evaluation, either through inference or prob- ing. Although Llama-3. 1------BIInnuut achieves the best results several tasks, Qwen3-1.7B still outperforms it on certain benchmarks despite being much smaller. Within the same model family, both the Qwen3 and Llama3 series demonstrate that larger models do not consistently surpass their smaller counterparts across all aspects. For instance, on MATH, Qwen3-0.6B outperforms Qwen3- 1.7B in prompt-based inference for logicality (18.18% vs. 15.06%), and 1lama-3.2- 1---....-1BBnsstrree surpasses Alama-3.1-888 1-11---Bnstte Instruct in binary probing for fluency (96.32% vs. 92.65%). These results highlight that different models can exhibit distinct strengths across evaluation aspects, cautioning against a blind reliance scaling laws.
Probing classifiers for binary classification serve highly reliable data filters. While the per- formance of probing classifiers multiclass prediction remains modest (approximately 50-60%), ) this is expected given the difficulty of approximating Manrge that is hundreds of times larger with M.mmall In contrast, binary classification performance reaches 80-90%, making it sufficiently reli- able to function as a reference-free coarse filtering mechanism. This capability is particularly valu- able for common NI.P applications such curating high-quality data for supervised fine-tuning. where the filter can efficiently separate high and low quality samples during initial screening and thereby reduce the cost of further fnn-grained annotation.
5 ANALYSIS
5.1 ABLATION STUDY POOLING AND CLASSIFIER METHODS
For each probing result in Figure we don't report the combinations of various pooling methods (r(2) in Equation 2 and classifiers (clf in Equation 6). To better illustrate the effect of different pooling and classitier choices, we conduct ablation study the Informativeness aspect of the

---

Published as a conference paper at ICLR 2026
MATH dataset, using Qwen3-0.6B and Alama-3.2-1B IB-Inttucct binary classification, with results shown in Figure
The results demonstrate that mean pooling consistently outperforms other strategies, which is intu- itive since averaging preserves critical information while producing compact feature representations. Among classifiers, Logistic Regression achieves the best results. With limited data and potentially noisy labels from LLMs scoring, calibrated probabilities and regularization from Logistic Regres- sion can lead better overall Fl score. These patterns match findings from recent studies: K Tamneni et al. (2025) show that complex probes often do not strongly outperform simpler pooling + linear classifiers; Lee al. (2023) demonstrates that averaging over all token vectors can beat using only CLS or last token embeddings. Finally, these findings are consistent with the detailed results shown Table IT where top- por-perfogning configurations most often involve mean pooling combined with Logistic Regression classifiers.
Pooling
metheos
90.0
Owan-0.ck 0.68
Qwen-O BB
07.5
1imm--12-111
85.0
62:70
62.5
$3.19
80.00
76.11
75.0
YAST
72.5
70.0
lo.
Lagösk Magross
Liee 3/M
Kanlan Parll
HL'
Figure 4: Ablation study of different pooling and classifier methods on binary classification tasks. Left: logistic regression fixed. Right: mean pooling fixed.
5.2 DATA FILTERING AND SUPERVISED FINE-TUNIE TUNING (SFT)
To assess the effectiveness of our data filtering approach, we apply the trained probing classifiers of Qwen3-1.7B in knowledge distillation setup, with Llama-3-8B-Instruee (Dubey al. (024) as the teacher and Llama-2-7B-Chr (Tourron et al. 2023) as the student. For each benchmark, the classifiers assign binary scores (0 or 1) to every response across five aspects, which are then summed to yield total score between O and 5. Responses
ranked by this score from high to low to construct the training set, and the student is trained on progressively larger subsets in 10% increments. We directly compare DeepSeek-VV V3 filtering and (ii) random filtering. The result curves are summarized in Figure
probing-based based filtering against two baselines: (i) the gold
Comparison of data filtering methods for SFT 2-amm-2-7B--haaa on GPQA
0332
Probing Lllered
Sendom 2ilr
0335
Average F1 Anono
0.38
0223
30
20
30
70
100
Treining dela propuctio (Toy-k %)
Figure 5: Llama-2-7B-Cha. SFT performance under different data filtering methods (probing clas- sifier, DeepSeek- V3,, and random) with incremental training subsets.
Our findings can be summarized as follows: (1) filtering training data with our probing classifiers yields SFT performance comparable to using powerful LLM the filter, indicating that our ap- proach can approximate LLM-level data quality judgments: (2) both probing and DeepSeek-Vi consistently outperform random filtering, despite occasional mid-range fluctuations where random
8

---

Published as a conference paper at ICLR 2026
sampling may incidentally capture higher-quality data, reinforcing the importance of quality- aware data selection for downstream training; and (3) the observed up-down-up trend suggests that initial gains derive from training high-quality data, performance then declines lower-quality data is introduced, and subsequently recovers once the volume of training data becomes sufficiently large. This supports claims in prior studies (Sajith & Kathala 2024; [yer et al.) (022)): data quality plays the primary role in low-resource settings, whereas data quantity may become dominant fac- tor as training data scales.
SEMANTIC CAPACITY ASYMMETRY IN EVALUATIVE SIGNALS
6.1 EVALUATIVE SIGNALS IN INTERMEDIATE REPRESENTATIONS
To better understand why small models support accurate evaluation, analyze the internal representations used our probing classifiers ($4). Specifically, we examine how evaluative signals vary across layers and what types of features most effectively capture them. We present representa- tive results in Figure 6 with additional analysis provided in Appendix [
(1) Representations Encode Strong Evaluative Signals. Across reasoning datasets, observe that hidden representations show substantial correlation with evaluation scores from the strong LLM judge. These signals are especially strong in mid-to-upper layers, indicating that evaluative infor- mation is embedded throughout intermediate layers, rather than being restricted to the output stage.
(2) Evaluative Features Reside in Structured Feature Subspaces. Probing feature spaces derived from PCA-based subspaces often reveals evaluative signals more effectively than scalar or attention- derived features. These observations suggest that evaluative signals are present across structured feature subspaces, with PCA-based projections revealing them more clearly than simpler features.
Type Performance Layer
Feeture Type Derformmnne Layer
#irel
0.75
0.m0
Eeary Cwwsshoooo
0.70
8.65
0.0
Figure 6: Layer- wise probing accuracy for Factuality (left) and Semantic Consistency (right) using Qwen3-1.7B on the MATH dataset. PCA features perform best, peaking in upper layers.
6.2 THE SEMANTIC CAPACITY ASYMMETRY HYPOTHESIS
These patterns observed above support the hypothesis that accurate evaluation requires much less capacity than generation and can rely on intermediate representations. This reflects an intrinsic asymmetry between the tasks: generation involves discourse planning and long dependencies that demand substantial capacity and often require full decoding, while evaluation focuses on identifying inconsistencies or content errors, which are already accessible in intermediate model states.
Prior work has shown that internal representations often encode much richer and more reliable se- mantic information than what is reflected in surface outputs. Probing studies demonstrate that hid- den states capture fine-grained linguistic and semantic structure [Waldis et al. 2024 Rogers et al.) 2020). Complementing this, antent-knowledg analyses reveal that task-relevant information can main present in intermediate representations even when generated text is unreliable or intentionally misled (Kadavath et al. 2022 Burns et al. 2023 Mallen et al. 2023). Building these insights, show that intermediate representations not only encode general semantic structure but also contain evaluation-relevant signals that be directly leveraged for effective reference-free assessment.

---

Published as a conference paper at ICLR 2026
7 CONCLUSION
Despite suboptimal generation, small LMs retain strong evaluative signals in their internal repre- sentations. We validate the Semantic Capacity Asymmetry Hypothesis and introduce INSPECTOR, grobing-basedd pipeline that extracts high-fidelity judgments from these latents. Experiments on various reasoning benchmarks show the capability of our method, suggesting that Representation- as-a-Judge can serve as a scalable and interpretable solution for evaluation and data curation.
REPRODUCIBILITY STATEMENT
All datasets and models used in our experiments are publicly accessible. We also provide ad- ditional details in the Appendix, including dataset statistics, model parameters, and training hy- perparameters. We believe that the information provided is sufficient to reproduce meth- ods and results. Furthermore, we released all data and code in the public repository: https: 7/github. com/
judge
USE OF LARGE LANGUAGE MODELS (LLMS)
We used large language models (LLMs) only for minor text polishing, such as grammar and phras- ing. All ideas, experiments, analyses, and discussions were conducted solely by the authors. The LLM did not contribute to the design and interpretation of our research.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint wwXi:2303.087744 2023.
Alon Albalak, Yanai Elazar, Sang Michael Xie, Shayne Longpre, Nathan Lambert, Xinyi Wang, Niklas Muennighoff, Bairu Hou, Liangmmng Pan, Haewon Jeong, et al. A survey on data selection for language models. arXiv preprint 2.N.::2442..6822 2024.
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in lan- guage models without supervision. In The Eleventh International Conference on Learning Rep- resentations, 2023. URL htwperpeeeeeeeeeeeeeeee ew. net. oow.ooormnnn...uuuuooo..o
Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models. ACM transactions on intelligent systems and technology, 15(3):1-45, 2024.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reicchrr Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2 10.1401144.1 14168, 2021.
Dario Di Palma, Alessandro De Bellis, Giovanni Servedio, Vito Walter Anelli, Fedelucio Narducci, and Tommaso Di Noia. Llamas have feelings too: Unveiling sentiment and emotion representa- tions in Ilama models through probing. arXiv preprint urXiv:2505.1649, 2025.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad AI-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv e-prints, pp. arXiv-2407, 2024.
Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. Length-controlled al- pacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04477. 2024.
Olga Golovneva, Moya Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Fazel- Zarandi, and Asli Celikyilmaz. Roscoe: A suite of metrics for scoring step-by-ster reasoning. arXiv preprint arXiv:2212.07911 2022.
10

---

Published as a conference paper at ICLR 2026
Hangfeng He, Hongming Zhang, and Dan Roth. Socreval: Large language models with the socratic method for reference- free reasoning evaluation. arXiv preprint arXiv:2310.00074, 2023.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXhv:2103.03874 2021.
Vivek Iyer, Bhavitvya Malik, Pavel Stepachev, Pinzhen Chen, Barry Haddow, and Alexandra Birch. Quality or quantity? data scale and diversity in adapting large language models for low- resource translation. arXiv preprint arXiv:2408. 12780, 2024.
Charles Jin and Martin Rinard. Emergent tepresentations of program semantics in language models trained on programs. In Forty-first International Conference on Machine Learning, 2024.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Saafield-Doddie Nova DasSarma, Eli Johnson, et al. Language mod- cls (mostly) know what they know. arXiv preprint (xXiv2227.0052111 2022.
Subhash Kantamneni, Joshua Engels, Senthooran Rajamanoharan, Max Tegmark, and Neel Nanda.
Are sparse autoencoders useful? case study in sparse probing. arXiv preprint arXiv:2502.16681, 2025.
Kihoon Lee, Gyuho Choi, and Chang Choi. Use all tokens method to improve semantic relationship learning. Expert Systems with Applications, 233:120911, 2023.
Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Watten- berg. Emergent world representations: Exploring a sequence model trained on a synthetic task. ICLR, 2023.
Ming Li, Yong Zhang, Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang, Ning Cheng, and Tianyi Zhou. Superfiltering Weak-to-strong data filtering for fast instruction-tuning In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the As- sociation for Computational Linguisties (Volume I: Long Papers), pp. 14255-14277 August 2024a.
Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang. Tianyi Zhou, and Jing Xiao. From quantity to quality: Boosting LLM performance with self-guided data selection for instruction tuning. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.),
Proceedings of the 2024 Conference of the North American Chapter of the Association for Com- putational Linguisties: Human Language Technologies (Volume 1: Long Pupers), pp. 7595- 7628, Mexico City, Mexico, June 2024b. Association for Computational Linguisties. URL
https: wttpss///aaaanttoooo.oooogo 2024 waa.eaaaaalllloo 421
Zhuochun Li, Yuelyu Ji, Rui Meng, and Daqing He. Learning from committee: Reasoning distilla- tion from mixture of teachers with peer-review. arXiv preprint arXiv:2410.03663, 2024c.
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches oul, pp. 74-81, 2004.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412. 19437, 2024.
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: Nlg evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303. 16634, 2023.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: 1907. /, 1692, 2019.
Alex Mallen, Madeline Brumley, Julia Kharchenko, and Nora Belrose. Eliciting latent knowledge from quirky language models. arXiv preprint arXiv:2312.01037, 2023.

---

Published as a conference paper at ICLR 2026
Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, and Gabriel Stanovsky. State of what art? a call for multi-prompl LLM evaluation. Transactions of the Association for Com- putational Linguistics, 12:933-944 2024.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic
evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
Felipe Maia Polo, Ronald Xu, Lucas Weber, Mírian Silva, Onkar Bhardwaj, Leshem Choshen, Allysson Flavio Melo de Oliveira, Yuekai Sun, and Mikhail Yurochkin. Efficient multi-prompt evaluation of LLMs.. In The Thirtt-eighth Annual Conference Neural Information Processing net fortm??mrmmm/ wtt.oommmmmmmppeeeeer
Systems, 2024. URL
Archiki Prasad, Swarnadeep Saha, Xiang Zhou, and Mohit Bansal. Receval: Evaluating reasoning chains via correctness and informativeness. arXiv preprint arXiv:2304. 10703, 2023.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Di- rani, Julian Michael, and Samuel Bowman. Gpqa: A graduate-level google-proof q&a bench- mark. In First Conference Language Modeling, 2024.
Anna Rogers, Olga Kovaleva, and Anna Rumshisky. A primer in BERTology: What we know about how BERT works. Transactions of the Association for Computational Linguistics, 8:842-866, 2020.
Aryan Sajith and Krishna Chaitanya Rao Kathala. Is training data quality quantity more impactful to small language model performance? arXiv preprint 1.Xiv:2411.18221 2024.
Xing Shi, Inkit Padhi, and Kevin Knight. Does string- based neural mt learn source syntax? In Proceedings of the 2016 conference on empirical methods in natural language processing, pp. 1526 1534, 2016.
Giulio Starace, Konstantinos Papakostas, Rochelle Choenni, Apostolos Panagiotopoulos, Matteo Rosati, Alina Leidinger, and Ekaterina Shutova. Probing lims for joint encoding of linguistic categories. arXiv preprint arXiv:2310. 186966 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint 2tt.:::2222.399777 2023.
Anton Voronov, Lena Wolf, and Max Ryabinin. Mind your format: Towards consistent evaluation of in-context learning improvements. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL 2024, pp. 6287-6310, August 2024.
Andreas Waldis, Yotam Perlitz, Leshem Choshen, Yufang Hou, and Iryna Gurevych. Holmes E a benchmark to assess the linguistic competence of language models. Transactions of the Associa- tion for Compulational Linguistics, 12:1616-1644 2024.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi. Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-248311 2022.
Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. A survey on knowledge distillation of large language models. arXiv preprint arXiv:2402. 131 2024.
An Yang, Anfeng Li, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388 2025.
Weizhe Yuan, Graham Neubig, and Pengfei Liu. Bartscore: Evaluating generated text as text gener- ation. Advances in neural information processing systems, 34:27263-27277 2021.
12

---

Published as a conference paper at ICLR 2026
Qiyuan Zhang, Yufei Wang, Tiezheng Yu, Yuxin Jiang, Chuhan Wu, Liangyou Li, Yasheng Wang, Xin Jiang, Lifeng Shang, Ruiming Tang, et al. Reviseval: Improving an-udee via response- adapted references. arXiv preprint arXiv:2410.05193, 2024.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluat- ing text generation with bert. arXiv preprint arXiv:1904.09673 2019.
Yong Zhang. Yanwen Huang, Ning Cheng, Yang Guo, Yun Zhu, Yanmeng Wang, Shaojun Wang, and Jing Xiao. Sentinel: Attention probing of proxy models for Ilm context compression with an understanding perspective, 2025.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595 2023.
A LIMITATIONS
While the proposed method demonstrates competitive performance compared to other baselines, we acknowledge that there are still potential limitations:
Following prior work, adopt five evaluation aspects and construct corresponding prompt templates for our experiments. However, the absence standardized evaluation criteria, our chosen definitions may not represent the optimal formulation for all tasks. Furthermore, we observe that the difficulty of the selected evaluation aspects varies. For instance, fluency appears be the easiest, the majority of responses consistently receive high scores relative to other aspects. This suggests the need for further investigation into the design and selection of evaluation aspects.
Although we conducted experiments on various datasets, future work can explore more general reasoning fields, such as commonsense and code generation tasks.
We only use the DeepSeek- V3 rating model, due to its cost and availability. How- ever, this may cause evaluation bias and affect our downstream probing classifier training process. Exploring diverse rating LLMs from different organizations could be valuable direction for future research.
B SMALL LMs PROBING DETAILSS
Notation and extraction. mode, obtaining
For sample i, input prompt Ix(x;, ye) and run Mamall in evaluation
H R
A R RXS:S
for layers 1,.... L. Here S, is the token length, d the hidden dimension, and R the number of attention heads.
Pooling and representation. From H compute small but expressive set of pooled vectors.
These pooling variants capture complementary token-level and global signals across layers:
S1
It. ER.
i,mean
S.
[=1
r Inst
(6)
H
[1,
Rdd
f = max(t (t: token t non-pad),
(C)
min H :]
r (
max H (t. :],
F.min
iamax
[n.(0) r.min F
Laaxx (.mean 3d
cuncall
13

---

Published as a conference paper at ICLR 2026
Attention and statistical features. For each head h of layer & we compute incention-entrope statistic (with small constant for numerical stability):
(0)
1 S.
(2) s. log (A E).
We compress head-wise entropies into low-dimensional summaries per layer:
(2)
of" std stda (2))
maxe (8)
i.h
For each pooled vector we also compute compact statistics:
norm(r) ||r||, var(r) Var(r), E(r) Σ softmax(r).. log softmax(F)ms
Feature assembly. For each layer l and pooling type E ble candidate feature matrices:
we assem-
x
(CA.(T)))) (0) [norm, var, E(-]
(0) N of max 6 i==
(E)
where PCAa denotes an optional dimenniinality--eduction to components. All transforms that depend the data (imputer, PCA, scaler) applied inside cross-validation pipeline to avoid information leakage.
Probing and layer ranking. We treat gold labels 81 {I...., 5, from D peob as both (i) multiclass prediction target multi 84,k and (ii) binary target hin s1,b 2 r] with threshold (high vs. low quality). For each (f.r) we fit logistic probe (linear logistic regression; one-vs-rest for multiclass) and evaluate generalization with stratified validaaiii...n..
Each candidate probe yields performance tuple
(ahin)
(E.P)
bin malti
(E.p)
(f..p)
(f.p) o
multi
where alEP)) and a ((t.p) denote the mean accuracies of the binary and multiclass probes, respectively, (f.r) (f.p) multi their corresponding standard deviations across cros-validation folds. We rank configurations by chosen criterion (binary or multiclass accuracy mean) for different downstream tasks.
and bin o
C DATASETS STATISTICS
We download datasets GSM8K and GPQA from Huggingface, MATH from their official project website: httpp:: /ghthubbtbtttttggg www.cooooccieee http///nndcccc/at.......mm GSM8K dataset is split according to the official original split ratio. We use the official training set for Math and MATH-500 for the test due its high representation and low cost. Since there is official train/test split for GPQA, use gpqa_main main and extendedd as the training set, diamond as the test set. Table I shows the statistics of all datasets.
Dataset
Type
#Train
#Test
GSM8K
Mathematics
7473
1319
MATH GPQA
Mathematics
7500
500
Science
994
198
Table 1: Dataset statistics.
D IMPLEMENTATION DETAILS
Primary experiments are conducted on eight NVIDIA Quadro RTX 8000 and eight NVIDIA RTX A6000 GPUs.
14

---

Published as a conference paper at ICLR 2026
D.1 SMALL LMS PARAMETERS
The parameter settings for Mamili Qwen3-0.6B (Yang et al. 2025
Qwen3-1.7B Yang et al. 2025).
Llama-3.2-1 3.2-IBBIInstruee (Dubey et al. 2024), and Llama-3. In-nsrruett (Dubey al. 2024).
Parameter
Value
temperature
0.6
max new tokens
256
do sample output.hidden.aa states
True
True
True
torch dtype
float16
attn implementation
eager
Table 2:
Small LMs parameter settings.
D.2 MEDIUM LMS PARAMETERS
The parameter settings for Medi: Llama-3-80 13-8B-IInttree (Dubey et al. 2024).
Parameter
Value
temperature
max new tokens
512
do sample
True
torch dtype
float16
top-p
Table 3:
Medium LM parameter settings.
D.3 LARGE LMS PARAMETERS
DeepSeek-V3
LLi al. 2024) is required by the official API: https://api-oat
deepseek. com.
Parameter
Value
temperature max new tokens
2048
do sample
True
model response format
deepseek-chat
{'type': 'json object') }
Table 4: Large LM parameter settings.
D.4 MODEL TUNING HYPERPARAMETER
We list the training hyperparameters for Alamma--7B--C- (Touvron et al. 2023) in Section 6.4.
D.5 PROBING CLASSIFIER DETAILS
We train several probing classifiers internal features to fit gold scores. All classifiers are imple- mented in scikit-learn pipelines with StandardScaler for feature normalization. We perform hyper- parameter search via 5-fold cross-valls validdaaio using macro F1 the scoring metric. Table 6 below summarizes each classifier's 's fixed parameters and the hyperparameter ranges considered in the grid search.
15

---

Published as a conference paper at ICLR 2026
Hyperparameter
Value
epoch batch size
8
8
learning rate
le-4
warmup_steps
100
max seq length
1024
gradient accumulation steps
4
lora r
16
lora alpha
32
lora_dropout
0.1
modgets
["g_proj", "v_proj"]
Table 5: Student LM training hyperparameter settings.
Classifier
Fixed Parameters
Hyperparameter Grid
Logistic Regression
max.iter=2000, class.eeiggttt="aaaammmdd"" solver="lbfgs"
[0.001,0001.0...I]]
penalty: ['12'
Random Forest
class.weight--aaaaannedd random.states
n.estimators:
[100,300,500]:
max-depth: (None,10,00)
min.samples.leaf? [1,2.5]
Multi-layer Percep- tron
hidee._lyyr..iiees=335,,,
alpha: [le-,le-3,le-2]:
activatiio==rellu"
learning ni rate init:
alpha=1e-4,
[le-4.1e-3.le-2]:
learninggrate..nitstee...
hidden_laeer.sizess
max.iter-1000,
(200.100.(1011,220000000.0))))))
early.stoopping-Irue,
tandom.state=42
Linear SVM
kerne]="linearr"
C: [0.001,000010011111111100
class.ee.ght=>/aaanneed"
probability-Tree,
random state=42
Table 6: Probing classifiers including fixed parameters and grid search ranges for hyperparameter tuning.
E PROBING DATASETS STATISTICS
As explained Section 4.1, we first determine the minimum number of samples n among the five score levels (1-5), and then randomly downsample the other levels to this size. This ensures that the multiclass probing tasks are balanced across labels and not dominated by frequent scores. Thus, after downsampling, Prott contains total of samples, and split the dataset into training and test sets with an 80:20 ratio. In the following tables, we mark the minimum number n of each aspect in bold. These score distributions reflect the difficulty evaluation levels among various dataset questions and aspects.
D pross statistics for GSM8K:
D Drras statistics for MATH:
D pros statistics for GPQA:
16

---

Published as a conference paper at ICLR 2026
#score=1
#score=2
#score=3
#score=4
#score=5
Semantic Consistency
184
491
369
6435
Logicality
118
842
174
26
6353
Informativeness
46
81
111
49
7226
Fluency
17
23
70
86
7317
Factuality
377
300
479
85
6272
Table Number of M.dd- scores across five aspects of probing datasets on GSM8K.
#score=1
#score=2
#score=3
#score=4
#score=5
Semantic Consistency
441
467
1879
121
4575
Logicality
224
1288
1604
98
4269
Informativeness
180
748
2062
4418
Fluency
132
27
746
2857
3721
Factuality
609
945
1818
34
4077
Table 8: Number of LLM-judge scores across five aspects of probing datasets MATH.
#score=1
#score=2
#score=3
#score=4
#score=5
Semantic Consistency
257
529
60
125
Logicality
267
463
145
31
87
Informativeness
197
164
327
163
142
Fluency
135
26
68
210
554
Factuality
306
475
119
46
47
Table Number LLM-judge scores across five aspects of probing datasets on GPQA.
F DETAILED MAIN RESULTS
We show the specific numbers of results in Table TO
Methoe
Semondic Consislencs
CRQA
Logicality MATH
GPOA
GSMEK
Informotivenss
GIOA
Floency
MATH
GIOA
Factuality
GEOA
2on9
Proppe Infordee
1421
19.35
23.92
17.46
18
22.12 13.
14.62
15.00
14.53
16.65
20:47
10.10
14. 50
14.65
15.95
Tuning
16.38
24.76
16.00
15.64
17.68
29.03
14,63
15.01
25.14
18.18
21.91
19.54
29.35
15.23
15.15
Proling Clasoffer
40.97
27.33
40.45
$1.93
52.00
04.23
45.15
61.57
57.57
51.46
00.57
$4.60
43.50
55.15
50.67
Limma-322 12-1i-liriil Dancy cl 5024
Pronpe Inference
11.82
15.33
11.73
23.54
12.12
17.11
17.52
K.65
11.76
10.74
12:43
Qwea3-1.7
Probing
42.95
59.79
58.34
$4.37
Llama-J1 Iisssrrd
Prumpe Informoe
0.07
20:78
12.75
26
25.5
50.20
37.21
27.35
19.33
43
12.69
14.00
9.63
22.53
Prohing Clasefter
39.31
51.00
36.41
50.00)
59.63
15.40
63.95
61.7%
62.25
42.50
56.15
45.35
44.06
63.94
09.55
Binarr Tesstficattin
(high vs 1o quality)
R.BERT: cis 2019
43.57
45.40
43.90
45,
45.35
55
46.05
45.00
44.83
43.57
44.10
46,89
45.00
43.57
46.05
Owea 100 Siakk
Pronpe
56.08
Tusing
50:73
76.37
Inforence
1357
45.45
74.35
62.79
41.51
22.55
46.64
71.17
Priling Clascieer
$2.35
68.07
63.64
65.00
78.75
$0.85
72.05
86.50
76.93
82.35
96.32
$7.04
71.42
73.35
$0.52
Qwea3-1.1 1.18 Yank at 2023
Pninge Infonder
18.95
$1.88
$4.66
61.54
$1.94
63.74
$6.28
$0.26
41.54
40.95
90.17 17
69.85
98.23
40.00
Probing Clisiffer
76.13
29.21
68.38
72.78
$3.07
74.47
72,08
85.48
$8.96
$8.33
92.65
96.11
$1.36
73.35
$7.11
Llana-J.1 T. Insstro Dilee et 2004
Prrmpe
76/13
79.77
70.51
Table 10: Average Fl score (%) across various reasoning benchmarks with multiclass classification and binary classification tasks. The best performance among different classification tasks in each benchmark is marked in bold.
We also report the specific layers, pooling, and classifier methods that achieve the best probing performance, summarized in Table TO across different benchmarks in Table TT
17

---

Published as a conference paper at ICLR 2026
GSM8K
MATH
GPQA
Multiclass Classification (score=1-5)
Semantic Consistency
Qwen3-1.7B.
layers=[14,25]]
Qwen3-1.77.
layers=[15).
Qwen3-1.7B.
layers=[2].
pool="mean" LR
mool="meun", LS
pool="mean" LS
Logicality
Qwen3-0.6B,
[aycrs=[17].
1lama-3.1-8B--1 I-Stuunsttrctt
lay-
Qwen.3-1.7B.
[13).
pool="mean". "mmammmaa"""
eis=[17], pool="last" LR
poot= 'mean' LS
Informaativeness
Qwen3-1.7B, [16.
Llama-3.1-855 8-88-Innttret,,
lay-
Llama-3.1-8-- In-Isruutt
lay
pool=" LS
ers=[10], poop=poooaee LS
ers=(12), "ool="mean", LS
Fluency
Mlama-32--BBlnsstree
lay-
Qwen3-1.7B.
layers=[18]. [88].
pool="last", layers=[18). (88)
Qwen3-0.6B.
ess=[1], pool=ll LS
pool="meur"
Factuality
Qwen3-1.7B. layers= (14.88.
[lama-3J--B-Insrrucct lay-
Qwen3-L.7B. [1y),
pool="meant
LR
exs=[21], pool="mean". LR
pool="last", "ast""LR LR
Llama-3.2-II-IInsteee -aaary--Cassffication
(high vs low quality)
Semanntc Consistency
lay-
Qwen3-1.7BB
layers=[15],
Jamm-3.1-8B-Instunn lay-
ers=[3), pool="last", LS
pool="mean" LR
ers=[16], pool="last", LR
Logicality
Qwen3-1.7B.
[24)
Qwen3-1.7B.
layers=[188.
Llama-3.2-IBInnstu
lay-
pool="lust" LR Qwen3-1.7B.
pool="last", LR
ers=[3], pool="l"o" LR
Informativeness
[21e
lay-
Qwen3-1.7B.
[uyers=[17]].
pool="mean". 1.
er== pool="lust", L.S
pool="mean", L.R
Fluency
Qwen3-1.7B,
Llama-3/2-TB- 1imma-3/--BB-nssrueet
Qwen3-1.7B,
layers [14],
pool="last" L.S
er==[13], pool= "lost",,ss", LR
poot="
Factuality
Qwen3-1.7B,
layers=[16].
Jlama-3.J-BB-Instuue
lay-
Owen3-L.7B.
layers=[18).
pool="meen", ".aa",,. LR
ers= poola poollpooean
LR
pool="last", LS
Table 11:
Detailed layers, pooling and classifiers selection for best probing performances. "pool" denotes different pooling methods in Equation notes linear SVM.
"LR" denotes Logistic Regression and "LS" de-
G OUT-OF-DISTRIUTION (OOD) PROBING
To evaluate the generalization abilities of our probing methods on Out-of-Distribution (OOD) data, we conducted experiments using one mathematical reasoning dataset as the training set and another dataset as the test set. Table [ highlights the probing performance with Qwen3-1.70 as the M OOD scenarios. Our analysis reveals two complementary patterns. First, multiclass (1-5) prob- ing demonstrates limited transferability, with Fl scores dropping to approximately 10-25%, sug- gesting that fine-grained score prediction is datasct-specific. In particular, transferring from more challenging datasets (MATH) simpler ones (GSM8K) proves more difficult than the reverse. Sec- ond, binary probing demonstrates substantially greater robustness under distribution shift: out-of- distribution F1 scores range from ~35-62% comparable to in-distribution results in Table T0 These findings suggest that PCA-based linear probing reliably captures coarse, domain -ennral quality sig- nals, whereas fine-grained distinctions are too difficult to transfer. Consequently, for claims of OOD robustness, we emphasize binary evaluations and recommend controlling probe capacity when re- porting cross-dataset transfer.
Semantic Consistency
Logicality GSM8K MATH
Informativeness GSM8K MATH
Fluency
Factuality
Method
GSMSK
MATH
GSMSK MATH GSM8K MATH
Mustclass Classiication (score=1-3)
Qwen3-1.7B
Probing Classifier
10.31
14.26
10.19
23.50
16.47
11.65
13.35
11.60
16.63
27.57
ClauryCCassifeuttion (high quality)
Qwen3-L.7B
Prohing Classifier
35.76
27.20
60.39
45.00
23.59
36.70
Table 12: Average F1 score (%) of test results across mathematics reasoning benchmarks in Out- of-Distribution (OOD) scenarios. Speeificallly, we conducted experiments by training on GSM8K and testing on MATH, as well as training on MATH and testing on GSM8K.
H EXTENDED RESULTS ON OPEN-ENDED BENCHMARK
While our primary focus is reasoning evaluation, additionally evaluated our approach on the open-ended generation benchmark AlpacaEval 2.0 (Dubois et al. 2024), using the same experi- mental settings the three reasoning benchmarks in Table 10 The results of AlpacaEval 2.0 shown in Table [3] The results further supports the effectiveness and robustness our approach on broader range of domains and tasks, including reference-fres open-ended benchmarks.
18

---

Published as a conference paper at ICLR 2026
Method
Semantic Consistency Logicality
Informativeness Multiclass Classification (score=1-5)
Fluency
Factuality
RoBERTa (Liu al. 2019 2019
7.56
8.08
8.32
11.69
8.08
Qwen3-0.6B (Yang 2025
Prompt Inference
22.40
11.11
16,94
11.82
8.89
Tuning
14.29
11.11
9.93
10.61
12.70
Probing Classifier
36.84
42.59
46.71
61.21
23.33
Llama-3.2-IB-Isstr (Dubey et al.
2024
Prompt Inference
12.50
13.33
8.37
16.97
0.20
Probing Classifier
35.71
55.56
46.08
61.21
28.57
Qwen3-1.7B YYang at 2025
Prompt Inference
7.14
20.11
22.70
25.19
20.00
Probing Classifier
35.12
45.93
42,01
65.10
23.70
1amma-.11-8B-Isstiee (Dbey ct al.
2024
Prompt Inference
14.92
17.99
18.34
23.38
14.29
Probing Classitier
34.69
53.70
45.90
57.32
22.96
Binary-Clann ClauuryyCClssiiiattiin (high vs low quality)
RoBERTa (Liu et al. 2019
41.56
39,68
46.58
49.49
39.68
Qwen3-0.6B Yang at 2025
Prompt Inference
55.24
43.00
49.93
46.36
44. 44
Tuning
35.24
43.00
45.41
28.48
27.78
Probing Classifier
61.42
55.56
62.76
91.06
55.56
LJama-3.2- 1---s-.re-e- (Dubey al.
2024
Prompt Inference
49.20
34.19
59.26
64.24
39.68
Probing Classifier
67.14
70.42
67.44
81.82
65.80
Qwen3-1.7B (Yang et al. 2025
Prompt Inference
50.24
65.80
45.03
45.45
55.56
Probing Classifier
67.14
75.93
70.71
91.06
77.78
Llama-3.1-8B-Immn (ubey et al.
2024
Prompt Inference
39.77
48.15 77.78
58.34
62.42 82.12
27.35
Probing Classifier
70.16
69.96
64.07
Table 13: Average F1 score (%) on AlpacaEval 2.0 with multiclass classification and binary classi- fication tasks.
ADDITIONAL ANALYSIS OF EVALUATIVE SIGNALS
To complement the main findings in Figure include detailed layer-wise analyses for additional evaluation dimensions: Factuality, Informativenesss, Logicality, and Semantic Consistency. These results are based on Qwen3-1.7B evaluated on the MATH dataset. Each figure reports probing accuracy across layers using three feature types: PCA-projected embeddings, statistical summaries, and attention-derived vectors.
Feature Type Performance by Layer
atternior
Brooy Clant Portten AECUTEY
3.70
Figure 7: Layer-wise diagnostic results for Informativeness. Peak correlation appears near Layer 25. PCA accuracy increases steadily, while stats features decline upper layers.
19

---

Published as a conference paper at ICLR 2026
Resture Type Performance by Layer
0.76
530 ab0ndon
Braly Cloo/fca0ee AccuDcO
0.72
0.38
0.34
0.53
0.50
Figure 8: Liyer-wise diagnostic results for Logicality. Signals concentrate around Layers 17 and 27. PCA features show rising accuracy toward the upper layers. Type
0.00
0.75
0.70
0.03
pC3
0.00
Figure 9: Layer-wise diagnostic results for Fluency. Evaluative signals show local peaks at Layer 17.
J CASE ANALYSIS
This case compares three evaluators scoring the model-generated response: strong LLM judge (DeepSeek-V35) V3), a probing-basee judge that accesses intermediate representations of Qwen3- 1.7B, and a prompt-based judge that relies on the same model's generated evaluation text. Consistent with our main hypothesis, the probing-based boaseg scores closely match the strong judge, indicating that the semantic and factual signals required for accurate evaluation are already encoded in the small model's 's latent states.
In contrast, the prompt-based evaluation from the same small model diverges: although its numeric scores remain plausible, the generated justifications introduce spurious concerns, overinterpret mi- nor details, or provide unreliable reasoning. These deviations reflect the higher capacity demands of generating coherent evaluation explanation. The pattern illustrates core capacity asymme- try- try-evaluation depends on localized semantic cues accessible from intermediate representations. whereas producing well-formed meta-evaluative text requires long-range structuring and discourse planning that small models struggle to express reliably.
GSMSK EXAMPLE
Question:
Keaton climbed 30 foet ladder twenty times while working at the construction site. Recce, also working at the same site, climbed ladder feet shorter than Kcatoná ladder 15 times. Whaté the total length of the ladders that both workers climbed in inches?
20

---

Published as a conference paper at ICLR 2026
Gold Answer: 11880
Model Prediction: Let's break down step by step.
Kcaton climbed a 30 feet ladder 20 times. To find the total length of the ladder he climbed, we need to multiply the height of the ladder by the number of times he climbed it:
feet/ladder times 600 feet
Since there are 12 inches in foot, can convert the total length from feet to inches:
600 feet x 12 inches/foot 7200 inches
Now, letś calculate the total length of the ladder Reece climbed. Reece climbed ladder that was feet shorter than Kcatoná ladder, which means Reeces ladder was:
feet 4 feet feet
Reece climbed the ladder 15 times, the total length of the ladder he climbed is:
fect/ladder times 390 feet
Converting this to inches:
390 feet 12 inches/foot 4680 inches
Now, letš add the total length of the ladders climbed by both workers:
7200 inches (Keaton) 4680 inches (Reece) 11,880 inches
So, the total length of the ladders that both workers climbed is 11,880 inches.
DEEPSEEK- V3 OUTPUT
Semantic Consistency (score 5): Every step strictly follows the problem facts without inventing events, dropping givens, adding unstated assumptions. The response correctly processes Keatons 30-foot lad- der climbed 20 times, caleulates Reeces ladder 4 feet shorter (26 feet) climbed 15 times, converts both totals from feet inches using the standard conversion (12 inches per foot), and sums results accu- rately. The final answer is fully grounded in the given problem.
Logicality (score 5): All inferences and arithmetic are valid; each step follows logically from prior steps. The response correctly interprets the problem, performs accurate calculations for both workers (including the conversion from feet to inches), and the results appropriatcly.
Informativeness (score 5): Full, verifiable step-by-step derivation showing essential calculations:
Keatoná total feet (30 20 600), conversion to inches (600 x 12 = 7200), Reeces ladder height (30 4 26), Reeceś total feet (26 x 15 = 390), conversion to inches (390 x 12 4680), and final sum (7200 + re-check the answer from these clear, complete steps.
4680 11880). Anyone
Fluency (score 5): Clear, grammatical, well-punctuated, and casy to follow; uses readable notation and step-by-step presentation that flows logically from one calculation to the next.
Factuality (score 5) All factual claims are correct and well-supported: accurate arithmetic (30x20=600, 30-4=26, 26x1553390, 600x12=7200 390x12=4680 7200+4680=11885 1880), correct unit conversion (12 inches per foot), and proper application of the problems conditions. No errors, hallucinations, unsup- ported assertions.
OWEN3-1.7B PROBING EVALCATION
Predicted scores:
Semantic Consisteney:
Logicality: 5
Informativeness: 5
Fluency:
Factuality:
21

---

Published as a conference paper at ICLR 2026
QWEN3- 1.7B PROMPT-BASED EVALUATION OUTPUT
Semantic Consistency (score 3): Solution follows the problem facts. Minor consistency lapse: the rationale never states the tacit assumption that each 'climb' is a single ascent. Also, unit management is done per person instead of showing single fect-to-inches conversion. These are small omissions that do not change the numeric result, so mid-level credit is appropriate.
Logicality (score 3): Arithmetic steps are correct. Logical inefficiency exists: the chain converts cach person to inches separately rather than adding feet then converting once - harmless but unnecessary detour. Because all inferences are valid but the chain is slightly untidy, a middle score fits. Informativeness (score 3): Key calculations
present. However, the explanation omits explicit grouped intermediate (explicitly writing "total 990 ft' before conversion not shown in every version), and some steps are shown only implicitly. Enough detail to verify the answer, but not fully exhaustive hence partial credit.
Fluency (score 4): Language is clear, grammatical, and easy to follow. Sentences read naturally and numeric steps separated. Minor repetition and slightly verhose conversion phrasing prevent perfect score, but overall readability high.
Factuality (score 2): While the final numeric answer is correct, there are factual/imprecision concerns that lower confidence: (1) the rationale implicitly assumes 'climb' means one ascent and does not state this, which could be misread in other contexts; (2) converting per-person instead of converting total feet introduces an avoidable chance for unit- unit-handling mistakes variants of the problem. These represent multiple factual/imprecision issues significant enough to warrant a low factuality rating.
Figure: Complete evaluation outputs for the GSM8K case across three judges: DeepSeck-Va. V3, Qwen3-1.7B (probing), and Qwen3-1.7B (prompting).
K PROMPT TEMPLATES
We provide the one-shot evaluation prompt templates for five aspects defined in Section 4.1 across three selected benchmarks: GSM8K, MATH, and GPQA. We use these prompts to get LLM evalu- ation results. Some parts of the prompt design are inspired by SOCREVAL(He et al. 2023).
It's noticeable that when we use these same prompts to prob small LMss, we found the prompts too complex for small LMs to understand, which makes it difficult to acquire accurate probing results. Thus, for probing prompts, we remove the one-shot example and ask it to directly output the score, instead of JSON output. Specifically, for the probing evaluation prompt, we remove the "Example question". "Example generated response". and "Example representation". We also change the final output requirement to: "Now evaluate the Question and Generated response above based on the instruction. Return only the score."
22

---

Published as a conference paper at ICLR 2026
Evaluation Prompt of Semantic Consistency GSM8K and MATH
You are a judge that scores Semantic Consistancy of stop stes ratonale for reasonig problem, Defiition: Semantic Consistency the sol solitoo stops and fhat answer must stay tarttul to the problem facts (no inventing events, no dropping givens, no added unstated assumptions). step is inconsistent conradiccs problem, introduces facts not present in the problem. or ignores givers. Scoring: integer
Every step and the finel anewer strictly follow the problem lacts; no unstated assumptione sontradictions.
Steps mosily follow
small unstated ossurpption that doeen't change the outcome.
sightly weekers
Notiosable contradiatione added facts that silect the reasoning or outcome.
Steps contradict the prob em introduce major unstatted facts: enswer not grounded the problem.
Example question:
Notalla sold clos 48 of her friends April, and then sho sote half as many clips in May. How many lls did Natalaa sell attogether in April and May?
genersted response:
Apri. May she mary but donated theew friendi so May=s4-3=211 totala60.
Example representation:
"Justiication::
problem; longer strictly given facts. "]
Question: (irpul questloa)
Generated response: (npuut response)
Now wvaluate the Queetion Generated above bossed on instructionn and the format the representaiin. Relur required
JSCN formal: ("score: <int 1-50, "jutilication" "cleat expleining the resson for the scorw>")
Figure 10: Evaluation prompt of Semantic Consistency GSM8K and MATH.
Evaluation Prompt of Logicality on GSM8K and MATH
Instruction:
You judge thet Logically of step-try-tte refionale for reasoning problem. Defntton: Logicaliy whether each inferenoe and arithmetic Scoring: integer 1-5.
Salowne vald luke relly spolis operaionn. Penilize inveld deductiona misappled
logicelly small inference leep or minor justification gep, but overall logic holis.
step
steps.
Some sleps are quasiionable contain small mistalue, yel parts of reesoning remaii sound..
Multipia invalid inferencea arihhmatcc emars that maturially affect the solution.
llogical
reasoning
Example questione
Natalla
48 her friends April. and then she sold half as clips How clipe oid Natalla allogether in April and May?
Example generated response:
April - 4B. Hall many in May' interpreted as add half of April to April May 48 48/2 72, Total 48 + 120.
Example representation:
I'score": 1, "ustiiicatoon: "Masinterpetts half as many' and parforms an invalid rezsaning stop, producing an llogical total.")
Question: question)
Generated response: (irput responsej
Generated response above on
example only required
"juttfficat explaining
Figure 11: Evaluation prompt of Logicality on GSM8K and MATH.
23

---

Published as a conference paper at ICLR 2026
Evaluation Prompt of Logicality on GPQA
Instruction:
You are j.dge that scores Logicalty of stap-by- step miionall for ressoning problem. Defintton: Logicaity whether each inference and anthmeric step follows valdd muess and conretly applics openalions. Penalize invald deductions or misappled ressoningg Scoring inseger 1-5.
5 All inforences and arthhmetic are valid, caon stop fellws logi ooll from prlor stops.
One email Inference
justfication but overall holds.
Multiple infermncees arthmmetc amora materielyy reasoning
steps
Fundamertally llogioal nonsensical reasoning (itops do not connect.
Example questiore
targe gone exons, central oncs triple repeats.
proposoc therapy?
Optonn::
polyA
R-ooge
anriewmse
Example generated response:
Morpholnoo blocka exan, must involved, polyA used, R--lopp skipped, antieense binds randomiy.
Example representations
"ustification:
contradictory. eg, lantisense binds unsupported
Question: (irput quastion)
Generated response: (Irpul nisponse)
Now evaluzte the Question and Generated response above based on the instruction and the formet the example representation, Return only the required JSUN format (' ('score : -int 1-5x, "Justifcation": "ctext explaining the reason for the score=")
Figure 12: Evaluation prompt of Logicality on GPQA.
Evaluation Prompt of Informativeness GSM8K and MATH
Instruction:
You new judge that aczres Informativeness wos step-by stap-by-step netionale reasoning problem, Definiion: Informativeeesse - whether the relionale shows essential slaps and intermeciate calculations reeded verily inbugar 1-6.
answer (nol merely lerse Tinal number). Reward verliable, aleçrwiee derications. Scoring:
5 FUll, varifablle step-by stlp derivation; anyone can re-check the arower from the staps.
Most essential stops shown: ona two minor gapa out overall vertfable.
derivations
partially verifable.
intermediate calculations missing verfication hard. Only an answer inaalvant details; no unable dar vation.
Vary
Example questions
Notalis sold clps to 48 her friends in Apri,, and then she sold holf as many clips May. How many clps did Natalla sell altogether in Apri and May?
Example genersted response:
After thinking, the answer
Example
I'score": "usttlication:: "Only number giver; steps explanation verily the camputation on.']
Question: (mput question)
Generated responso: (input responsa)
evaluate Questien Generated
based instruction the representation, Return required
JSON format: ('score": "ustilicaion": " explaining for scare>")
Figure 13: Evaluation prompt Informativeness on GSM8K and MATH.
24

---

Published as a conference paper at ICLR 2026
Evaluation Prompt of Informativeness GPQA
Instruction:
You are a jadas that scores Informalivenees of stop by step rational for roasoning problem. Defrition: Informaiiennss whether the rationale shows the essential steps and interediatee ca calcuraiions neeced verty the fiai answer (not merely a terse final numbert. Reward verilablle stspwise derivallone. Scoring: integer 1-5.
5 - Full verifable step by allep deriattion; anyone can re-d check the arowwer from the slepe.
4 Most essential shopo shown; ona or two minor gapa but overal verflable.
derivations omritte, partially caloulations missing cation
Very terse:
Ony an enswer or evant deta is; no usable derivation.
Example question:
lage gene exone,
triple repeato,
propoeec therapy?
DolyA
R-lcops
antisense
ariat
Example generated
loops
Example representation:
I'scare:: "Justitcation": the final
Question: ('nput questin))
New evaluate the Question and Generated response above based on the instruction and the format the example representation, Return only the required (8ON formet: ("sccee! 1-55, "Janifiation" exclwining meson the spones
Figure 14: Evaluation prompt of Informativeness on GPQA.
Evaluation Prompt of Fluency GSM8K and MATH
You are jucige that scores Fluency freacabilty and clariyy of step by-s rationatee for reesoning problem. Defnition: Fluency the text grammatical,
cleer, and esey follow. Judge sunctuation, sentence flow. reedable noterion and prepentation. Pluency does NOT eveluete comeccreess Scoring: integer
Clear, grammatical, well punctuated, and easy to tollow; notation readable.
rasing punctustion phrasing, is punctuation, Hard to fallow; many grammatical issues or poor notation.
Mostly clear
comprehension.
Unreadiable incoherent ang.
Example question:
Natalaa sold clipa to har friendis in April, and then sha soid half as many clps May. How many clipa did Natalia sel altogathar in April May?
Example generated response:
sold spril then half may
Example representations
("score" "Juatifoation: "Ungrommatical meaning
phrasing and punatustion poor.")
Question: (Input queetion)
Generated response: [Input response)
based instruction JSON format: ("score "Jaatifcation" "ctext explaining for
evaluato Question
reprosentation. Retum
Figure 15: Evaluation prompt Fluency on GSM8K and MATH.
25

---

Published as a conference paper at ICLR 2026
Evaluation Prompt of Fluency on GPQA
Instruction:
judge Fluency feadabilit
step-by--teg
problem. Fluency grammatical, Flanny comeetnees. Seoring: inleger
easy Judge punchuation,
gremmettical, punciuated, easy follow; notation
Mostl clear with smal phrasing punctuation issues.
Understandable but awkwaar phrasing punsiuation, notation that slowe comprehension.
Hard to follows grammeticaal
incoherent language.
Example questiore
large gene has dozena exons, of which the central onea code folded bripke helical repeats. Which structure below is not invalved the proposed therapy?
Option::
poljA
anrisense
lariet
Example generated response: morpholino. involve mloope
Example representation:
("acom": 2, "uatfication" 'Short, ungrammatioal presoiig and poor punctaation make hand to mart, withuggh meaning is mecovenblle")
(irpu question)
Generated response: (nput response)
representattin, only
JSON format:
explaning sconex")
Figure 16: Evaluation prompt of Fluency on GPQA.
Evaluation Prompt of Factuality on GSM8K and MATH
Instruction:
You are a judge that scores Factuallty of stop by stop rationale for reasoning problem, Definitorn Fastusiity whether the claims, stated facts, dence, references, and concrete assertion the rationale are foctually correct and supoorted Penalize incomect facts, unsuppored assertions, hal fallcinations, wrong attarions, mieeop led doman knowledge. Scoring: integer 1-5.
All factual claims referenced facts correct wel-supported the rationale common knowledge, No hallucinations. clain)
Some facturl emons or crmppported claims uxist: wesely-suppored
impresision (ypo,
change arswer may selengeable with connetions or additiinnal evidenoe.
Multiple tach mistakse or serius unsupporied assertins that matarially affect confidence the condusioon.
Major fachual emrrss clear Pallucinaions, or fundanentally
domain knowledge that render the answer incorret.
Example question:
Natalla soid clips 48 of hor friends Apri, and then she sold half as many clips May. How many clips dia Natalla sell aloggether in April and May?
Example generated responaec
She sold 48/2 - 20 clips in May. Natalia sold 48-20 68 clips ahogether. #esa 68
representation::
("score": "ustificaion" "Arithmetic is incormec (48/2 1u0960 20), cleims numerrcc rees wrong "
Question: [inuut question)
Ganarated response: (input resporsa)
Now Question Genorated
instruction format: 1-5x, "justfication"" "vtxxt explainig the
representation.
Figure 17: Evaluation prompt of Factuality on GSM8K and MATH.
26

---

Published as a conference paper at ICLR 2026
Evaluation Prompt of Factuality GPQA
Instruction:
You are jurige that scones Fachuritty of step-hy-step miloonw for meeoring problem. Delnitio: Fachuwilyy whether the cleims, stated facts, evidence references, end concrete sesertions in the rationale are factually cornect and supporied Penslizs incoreet facis, unsupported asseriions, hellucinstions, wrong otalkors, or misapoliac domain knowledge Scoring: integer 1-5.
All factual claims and referonced faots are corract and wel supported the rationale common knnwwedge. No hallucinaiios.
Minoo foctual imprectsion
weakdy
conclusion.
Some
answer may salvageable recteiiions Multipe factual mistakes serous unsupported assertions materially affect confidence the conclusion.
unsupported
Major factual emons, cliar hillacintions, fundsmentally wrong domain knowledge that sander the arowwer incornett.
Example
lirge ginw dovens excore, central folded tripie repretts
involved the proposed tharapy?
polyA tall
R-ooos
Example generated resporse:
The polyA tail not used; antisense and R-loops both inalevant: lariet forms inolovant loops.
('score":: "Jstitcation": claims are factually impracise (o.g.. inalevant loops supporteed by quasstion), answer may soill
Question: (Input question)
response: (Input resporae)
Now avaluste the Question and Ganerated response above based on the instruction and the format of the example repessenaation. Return only the reqqlieed JSON format: ('score": 1-6x, "ustiiictioon: -doxt exalaining the reason for the score>")
Figure 18: Evaluation prompt of Factuality on GPQA.
27


Reviews:
[{"review_id": "g2wuJVOvSu", "reviewer": "Reviewer_GC61", "summary": "This paper observes that small LMs, although with their limited generative ability, can still provide reliable evaluations within their hidden representations. The authors then propose the Semantic Capacity Asymmetry Hypothesis, which posits that evaluation requires significantly less semantic capacity than generation and can rely on intermediate representations. The authors propose Representation-as-a-Judge, and implement this idea through INSPECTOR, a probing-based framework that predicts aspect-level evaluation scores directly from small model representations. Experimental results demonstrate that INSPECTOR substantially outperforms prompting-based small LMs and closely approximates large model judges.", "questions": "1. Can the authors provide a formal definition or metric for semantic capacity asymmetry beyond the conceptual description?\n\n2. The work uses judgments from DeepSeek-V3 as gold labels for training and evaluation, but does not incorporate human annotations as ground truth. Have the authors considered comparing (a) human judgments, (b) large LLM-as-a-Judge outputs, and (c) the proposed Representation-as-a-Judge ouputs?\n\n3. Some minor questions: \n\n   (1) In Eq (6), the symbol $\\mathcal{C}$ is missing, and the meaning of $\\mathcal{F}$ is unclear.\n\n   (2) In the ablation study shown in Figure 4, is the classifier fixed as Logistic Regression in the left plot (where pooling methods vary), and is mean pooling fixed in the right plot (where classifier methods vary)?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Novel idea for LLM-as-a-Judge.**\nThe concept of semantic capacity asymmetry is creative and appealing. It provides a fresh lens for understanding evaluation-relevant signals in internal representations, suggesting that weak evaluation performance in small LMs may arise from surface-level generation limitations rather than an inherent lack of semantic understanding.\n\n**Effective probing design.**\nThe paper designs probing mechanisms to capture evaluation-related signals from model representations. These probes are empirically shown to be effective and help our understanding of how evaluation-relevant signals work in hidden layers.\n\n**Practical significance.**\nBuilding on this idea, the work demonstrates that smaller open-source models can serve as reliable evaluators, substantially reducing cost while maintaining evaluative fidelity, which is practically meaningful.", "weaknesses": "**1. Missing intuition behind the probing design.**\nThe paper introduces probing experiments to support the idea of semantic capacity asymmetry, but the intuition for the probing setup is underexplained. It is unclear what specific representational property the probes aim to capture or why this design shows semantic capacity.\n\n**2.Limited dataset coverage.**\nThe evaluation focuses on three reasoning benchmarks (GSM8K, MATH, and GPQA), all within the mathematics and science domain. Other tasks types, such as open-ended generation, summarization, dialogue and code generation are not considered here. Incorporating a broader range of domains and task formats would strengthen the generality of the proposed idea.\n\n**3. Clarity and presentation.** \nSeveral sections are dense and could be improved with additional figures or concrete examples. For instance, a figure to illustrate how linear probes are constructed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2kCGY6uDt0", "reviewer": "Reviewer_CaW1", "summary": "This paper proposes INSPECTOR, a probing-based framework that evaluates model outputs using internal representations instead of text generation. It introduces the Semantic Capacity Asymmetry Hypothesis, suggesting that evaluation requires less semantic capacity than generation. By leveraging small open-source LMs, INSPECTOR enables lightweight, interpretable, and scalable evaluation, achieving performance comparable to large proprietary models.", "questions": "1. Is there a specific reason why the paper focuses only on decoder-only models instead of considering encoder-only architectures?\n2. I believe the Semantic Capacity Asymmetry Hypothesis aligns with a well-established understanding in LLM research. It would be helpful to cite prior work that supports this perspective.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper reframes model evaluation as Representation-as-a-Judge, shifting from prompt-based evaluation to representation-based probing.\n2. By leveraging internal representations from small open-source LMs instead of large proprietary models, INSPECTOR offers a lightweight and interpretable alternative that significantly reduces computational cost.\n3. The framework achieves high predictive accuracy on reasoning benchmarks (e.g., GSM8K, MATH, GPQA), demonstrating that small models (1.7B) can approximate large-scale evaluators.\n4. The proposed method can enhance LLM evaluation pipelines, reduce dependence on expensive closed models, and support data filtering for downstream tasks.", "weaknesses": "1. Experiments are primarily focused on reasoning benchmarks; it is unclear whether the method generalizes to other domains such as dialogue, summarization, or open-ended generation.\n2. The performance quality of the judge (small LM) could be dependent on how the evaluation criteria are established.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PlqaOIMY6X", "reviewer": "Reviewer_DiLJ", "summary": "This paper proposes a new evaluation paradigm called Representation-as-a-Judge, which shifts from relying on large language model (LLM) generation to using internal representations from small LMs (SLMs) for evaluation. The authors introduce the Semantic Capacity Asymmetry Hypothesis, arguing that evaluation requires less semantic capacity than generation. They instantiate this idea in INSPECTOR, a probing-based framework that trains lightweight classifiers on SLM hidden states to predict quality scores (e.g., logicality, factuality) derived from a strong LLM judge. Experiments on GSM8K, MATH, and GPQA show that SLM probes outperform prompting baselines and closely approximate full LLM evaluations, especially in binary classification tasks. The approach also proves useful for filtering training data to improve downstream supervised fine-tuning.", "questions": "1. Have the authors evaluated or considered how the probe’s judgments (or the large LLM’s scores it learns from) align with human evaluations of the responses?\n\n2. The experiments focus on mathematical reasoning problems. How well do the authors expect the Representation-as-a-Judge approach to transfer to other domains or tasks?\n\n3. Did the authors experiment with regression instead of classification to predict aspect scores on a continuous scale?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The Representation-as-a-Judge framework is technically sound, which could be an alternative to the prevalent “LLM-as-a-Judge” approach.\n\n2. The approach enables efficient evaluation using smaller, open-source models instead of proprietary LLMs.\n\n3. The empirical results are strong, showing significant gains over some prompting and fine-tuning baselines.", "weaknesses": "1. A limitation is that the method still requires a powerful LLM in the loop to obtain initial evaluation scores (for training data). This paper assumes the LLM’s scores are gold-standard. It would strengthen the work to either validate against human ratings or discuss the implications of this dependency.\n\n2. The probing classifiers achieve relatively low accuracy on fine-grained multiclass (1–5) predictions. This might limit the method’s use if one requires precise scoring, and also indicates some inherent issue that the approach doesn’t fully match the large model in terms of detailed gradations of quality. \n\n3. The INSPECTOR pipeline introduces additional complexity and tuning that a direct LLM judge does not require. One must decide how to pool representations, which layers to select, what classifier to use, etc. This paper doesn’t deeply explore the sensitivity to these choices.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ENCrONlfE7", "reviewer": "Reviewer_DGMU", "summary": "This paper challenges the conventional LLM-as-a-Judge paradigm by showing that small language models (SLMs), despite weak generative ability, encode rich evaluative signals in their hidden representations. It proposes the Semantic Capacity Asymmetry Hypothesis, suggesting that evaluation requires less semantic capacity than generation. The authors introduce INSPECTOR, a probing-based framework that predicts aspect-level evaluation scores using internal representations of SLMs. Experiments on reasoning datasets (GSM8K, MATH, GPQA) demonstrate that INSPECTOR approaches the performance of large LLM judges while being more efficient and interpretable. This work advances a Representation-as-a-Judge paradigm that enables cost-effective, scalable, and transparent evaluation of model outputs.", "questions": "1. Could multi-task or cross-aspect probing improve performance compared to aspect-specific classifiers?\n2. How stable are the layer rankings across different datasets — do the same layers consistently encode evaluative information?\n3. Can this method be extended to explain why a response was scored poorly (e.g., highlighting reasoning errors)?\n4. Could the representation-based judge itself be used as a reward model in reinforcement learning from human feedback (RLHF)?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces the Representation-as-a-Judge framework which is a new way of performing evaluation without decoding, shifting the paradigm from generation-based judgment to representation-based probing.\n2. The paper demonstrates consistent and large gains over prompt-based baselines across multiple reasoning datasets, showing that evaluative signals exist in intermediate representations.\n3. The proposed approach significantly reduces inference cost, avoids dependence on proprietary LLMs (like GPT-4), and provides interpretable layer-level insights.\n4. Uses binary probing classifiers to filter low-quality data effectively, improving downstream supervised fine-tuning.", "weaknesses": "1. Despite promoting efficiency, the pipeline still depends on large LLMs (like DeepSeek-V3) to provide \"gold\" evaluation scores. Gathering these scores is itself expensive.\n2. Experiments focus only on mathematical reasoning (GSM8K, MATH, GPQA); it’s unclear whether the approach generalizes to open-domain, creative, dialogue tasks or safety evaluation.\n3. While simple probes (logistic regression) are interpretable, they may underfit complex evaluative signals, limiting the upper bound of achievable fidelity. \n4. The paper lacks qualitative analysis and error analysis. Observing some samples where these models perform much better than baselines, or looking at error buckets would be nice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)