Bayesian Post Training Enhancement of Regression Models with Calibrated Rankings
Authors: Kevin Tirta Wijaya
Bing Xu Hu
Hans-peter Seidel
Wojciech Matusik
Vahid Babaei
Venue: ICLR 2026 Poster (2026)
Keywords: post training
regression enhancement
calibrated rankings
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Accurate regression models are essential for scientific discovery, yet high-quality numeric labels are scarce and expensive.
In contrast, rankings (especially pairwise) are easier to obtain from domain experts or artificial intelligence judges. We introduce RankRefine++, a novel plug-and-play method that improves a base regressor's prediction for a query by leveraging pairwise rankings between the query and reference items with known labels. RankRefine++ performs a Bayesian update that combines a Gaussian likelihood from the regressor and the Bradley-Terry likelihood from the ranker. This yields a strictly log-concave posterior with a unique maximum likelihood solution and fast Newton updates. We show that prior state-of-the-art is a special case of our framework, and we identify a fundamental failure mode: Bradley-Terry likelihoods suffer from scale mismatch and curvature dominance when the number of reference items is large, which can degrade performance. From this analysis, we derive a calibration method to adjust the information originating from the expert rankings. RankRefine++ shows a stunning 97.65\% median improvement across 12 datasets over previous state-of-the-art method using a realistically-accurate ranker, and runs efficiently on a consumer-grade CPU.

Published as a conference paper at ICLR 2026
BAYESIAN POST TRAINING ENHANCEMENT OF REGRESSION MODELS WITH CALIBRATED RANKINGS
Kevin Tirta Wijaya', Bing Hu², Hans-Peter Seidel', Wojciech Matusik Vahid Babaci! 1:15
1 Max Planck Institute for Informatics, University of Waterloo, 3 Massaacussetts Institute of Technology: FFraunhofer SCAI, University of Bonn kevintirta gma i 1. com, b25hu @wt.uuuwwwrrrrr.. www.ateerooooo.oom ca, vahid.babae i isccai. fraunhoer.de
ABSTRACT
Accurate regression models are essential for scientific discovery, yet high-quality numeric labels scarce and expensive. In contrast, rankings (especially pair- wise) are easier to obtain from domain experts or artificial intelligence judges. We introduce RANKREFINE++, a novel plug-and-play method that improves a base regressor's prediction for a query by leveraging pairwise rankings between the query and reference items with known labels. RANKREFINE++ performs a Bayesian update that combines a Gaussian likelihood from the regressor and the Bradley-Tery Terry likelihood from the ranker. This yields a strictly log-concave pos- terior with unique maximum likelihood solution and fast Newton updates. We show that prior state-of-the- the-art is a special case of our framework, and we iden- tify a fundamental failure mode: Bradley-Terry likelihoods suffer from scale mis- match and curvature dominance when the number of reference items is large, which degrade performance. From this analysis, method to adjust the information originating from the expert rankings. RANKRE- +INE+ shows a stunning 97.65% median improvement across 12 datasets over previous state-of-the-art method using a realistically-accum ranker, and runs ef- ficiently on a consumer-gradd grade CPU.
derive calibration
INTRODUCTION
Regression underpins many scientific and engineering pipelines. In materials science and drug dis- covery, for example, peccture-to-proerte predictors (i.e., through regression) can guide inverse molecular design loops by replacing expensive experiments and simulations with fast surrogates (Sanchez-Lengeling & Aspuru- (spuru-Guzik] 2018 Liang et al.) 2021 Allen & Tkatchenko 2022). Yet, many domains, collecting accurate absolute labels (e.g., molecule x is toxic at I nM) is slow and expensive (Wijaya et al. 2024), leading to data scarcity and limiting the accuracy of learning-based regressors.
An alternative to absolute labels is pairwise ranking (e.g.. is molecule x more toxic than molecule y?). Pairwise rankings can be collected from human experts or from computational models, in- cluding general-purpose large language models (LLMs), at relatively low cost and with competitive accuracy. For humans, pairwise ranking reduces cognitive load (Routh et al. 2023) and mitigate scale-interpretation bias (Hoeijmakers et al.] 2024) compared to directly predicting the absolute labels. Similarly. LLMs exhibit strong pairwise ranking ability in technical domains (Guo et al.,
2023 Sun et al. (2025.. This raises central question: can we enhance a pretrained regressor using only a handful of absolute labels plus inexpensive pairwise rankings without retraining the regressor?
We propose RANKREFINE++, a post-training enhancement method that combines a regressor's pre- diction with calibrated expert rankings via Bayesian inference. Given a pretrained regressor and an external pairwise ranker that compares a query item to small set of labeled references, RANKRE- +INE+ forms posterior over the unknown scalar by combining (i) Gaussian likelihood centered at the regressor's ' prediction, and (ii) a Bradley-Terr likelihood for pairwise rankings. We show that the posterior is strictly log-concave, yielding unique solution and enabling fast optimization.

---

Published as a conference paper at ICLR 2026
An additional key insight is that the default Bradley-Teri Terry model can be mismatched the behavior of pairwise rankers, leading to biased estimates and increased errors. RANKREFINE++ addresses this with (i) learned temperature that calibrates the sigmoid slope of the Bradley-Tere model, and (ii) an accuracy-awand aware soft gate that regulates the influence of the ranker. Specifically, contributions are:
1. We introduce REFINEE which combines regressor likelihood with a calibrated ranker likelihood to enhance predictions without retraining. We prove that the state-of tate-of-thhe- art, RankRefine (Wijaya et al., 2025), is a special case under the Gaussian assumption.
2. We show how an uncalibrated Bradley-Terry model can bias the estimates when ranker likelihood dominates, and we provide a temperature calibration with accuracy-aware soft gating mechanism to mitigate the issue.
3. Across 12 cross-domain datasets (including real-world molecular datasets), RANKRE- +INE++ significantly improves over existing post-training enhancement methods (Wijaya et al. (2025); Yan et al. (2024), and 3 other baselines) and remains effective when using imperfect LLM rankers. Specifically, RENKREFINE++ achieves 19.33% median MAE duction relying on 30 reference samples and a ranker with 65% accuracy, which translates stunning 97.65% relative improvement compared to Rank 9.78% median MAE reduction.
Together, these results position RANKREFINE++ a practical and principled way to leverage readily-available pairwise information to enhance scalar regressor data-scarce domains. Source code is available at httpss//giiubb.oo///ttt//rergg
2 BACKGROUND
Pairwise Comparison Models. In pairwise (binary) comparisons, the probability of item z; preferred to æj is often modeled as P(z; > xz) F(y: Mj). where y are latent scores and F is a cumulative distribution function (Cattelan 2012). Classic pairwise comparison models include mhustonn--osteelle with Gaussian link function (Thurstone) 1927 Mosteller [151] and Bradley- Terry with a logistic link function (Bradley & Terry 1952). These models provide a one-dimensional likelihood for the unknown scalar label that be estimated using MAP MLE estimation.
-us------judge. General-purpose LLMs have demonstrated strong performance on relative com- parisons across various domains (Qin et al. 2024 [W et al. 2024 Guo al. (023), with evidence
of better performance in pairwise comparison compared to score prediction (Zheng et al. 2023).
With widely available web APIs (OpenAl 2025 Anthropic 2025 Google 2025), collecting com-
parisons at scale is increasingly casy, making LLMs a convenient external ranker when numeric absolute labels are scarce.
Regression Refinement using Pairwise Rankings. Two representative approaches in this area are the Projection method Yann al. (024) and RankRefine (Wijaya al., 2025). Projection constrains the regressor's prediction to feasible interval implied by non-contradictory pairwise outcomes. Meanwhile, RankRefine fuses the regressor's 's output with a rank-only estimate via inverse variance weighting. Our proposed method introduces different paradigm: it reframes fusion as a Bayesian inference, where the regressor and ranker contribute likelihoods that jointly determine the posterior.
Relation to Learning from Human Feedback. Recent works on fine-tuning with human feedback CCristiano et al. 2017 Ziegler et al. 2019 Ouyang et al. 2022) also use pairwise rankings modeled via Brerlyy-Terry, but with different goal. They (i) learn a global reward model and (ii) fine-tune pretrained model accordingly. contrast, we apply a per-query, post training prediction correction by combining the regressor's likelihood with calibrated ranker likelihood. This produces a one- dimensional posterior without modifying the regressor's parameters.
METHOD
In this section, describe the base formulation of RANKREFINE++ (Section 3.1), along with the analyses of its behavior (Section 3.2) and proposed modifications for improving the performance (Section 3.3). We provide the detailed proofs of the analysis in the Appendix.
2

---

Published as a conference paper at ICLR 2026
3.1 BAYESIAN INFERENCE ON EXPERT RANKINGS TO IMPROVE REGRESSION MODELS
Let f be a regressor trained on = ((t,) with scalar labels Yj E R and predic- 0j. Let R be an expert pairwise ranker that returns binary comparison: 0 otherwise. We are given reference set {(r y:)) with known Yi. which can be, but not always is, the regressor's training set C. For query with unknown E I, we collect expert rankings against all references: where r; E (0,1)
tion f(xz)
R(T,, Tb) = 1 if it predicts > ybb and I((
= {r: R(xo, z. ))){
Assuming conditional independence of regressor and ranker given the true yo, the Bayes' rule yields
90 G)
x
p yo))
G yo)
p(yo)
(1)
posterior
rgg. likelibood
likelibood
prior
with Gaussian regressor likelihood p(108 yo) N i Mo, oite) and rank likelihood
p(r; = you 81) = syyo vi), s(z) = 1/(1+
(2)
h
p[ v0) = II s1y0 - y1) (1- syo - 9:) 1-n
(3)
which is the product of all pairwise comparisons in G under the Bradley- Terry model (Bradley & Terry 1952).
With the two likelihoods, the base formulation of RANKREFINE++ enhances regressor predic- tion with expert rankings using maximum a posteriori (MAP) estimation (Murphy 2022 which maximizes y with the objective function
Cly)
1 20
-2)
rs log s(y y:) + (1 r²) log(1 s(y wil]] + logr(y)-
we assume a flat (uninformative) prior, i.e., p(y) 1, RANKREFNE++ reduces to a maxi- mum likelihood estimation (MLE) (Murphy (202). The objective function of the RANKREFINE++ (Equation 4 is strictly log-concave so we can obtain the maximum efficiently.
Lemma 3.1. (Strict log-concavity.) If o > 0 and logp(yo) is concave (including the flat prior), then log ((V
77 G) is strictly concave in Yo- O
Corollary 3.2. (Existence and uniqueness.) The maximum posteriori (and maximum likelihood under a flat prior) estimate exists and is unique.
Proposition 3.3. (RankRefine (Wijaya et al. 2025) is special case of RANKREFINE++ under Gaussian assumption.) Let (eT(w) denote the rank-only log-likelihood and /e arg max, (vr(y) By assuming Gaussianity the ranker likelihood (similar to Wijaya et al. 2025 ) the second-order expansion of LBr(U) around 8 yields
p(G y) == N(v: 50 oi)
o =
Σ s s 31)(1 s(U v.))
(5)
Combining this with the Gaussian regressor likelihood and a flat prior gives the inverse-variance weighted (IVW) (Cochran & Carroll, 1953) estimator, i.e., RankRefine (Wijaya et al. 2025). for
which the estimate %6 is,
2h ///02 + /0/00 3
(6)
1/0 + 1/0
Proof. The gradient and curvature of Lvr(y) are
k
C'Br(y) = Σ (ri s(y - v:)),
C" (v) = Σ
(sy 1.)(1 s(y - ((v-)))))
(7)
1=1
i=1
3

---

Published as a conference paper at ICLR 2026
The second-order Taylor expansion of LBT around y 1/6 is
(y) (BT( + L' (2)) (y Vo 1 BT (1b) .V (
(8)
The first term is constant and the second term is zero, therefore,
k
(V)(U) &
s y1 s( yy)) 1
2
Similar to RankRefine, we make a strong assumption of Gaussianity for the BT likelihood,
p[G vo) & exp(CBr(u)) = exp (-
s s 9:)(1 =(100' by -00) = =N (Nura pn,)
(10)
Pro yo
2
( 9:)(1 s y:))
The joint likelihood of the conditionally independent regressor and ranker is now product of two Gaussians. The MLE, i.c., MAP estimate with a flat prior, then solves a weighted least-squares problem whose maximizer is the inverse-variance weighted average. D
3.2 ANALYSIS OF POTENTIAL. PERFORMANCE DEGRADATION OF BASE RANKREFINE++
Under the base formulation, RANKREFINE++ can degrade when the reference size & and ranker accuracy exceed cer- tain thresholds. The effect is butaset-dependen and arises from mismatch between the Bradley-Tery model and the behavior of real-world rankers.
Eaury) Step Function
Ejury) - Sigmoid Function
11
Do 0.61 Fravible Range (dhm Massa
Separating the rank likelihood from the objective in Equa- tion 4 and maximizing it yields the rank-only MLE 18- If the ranker were perfectly accurate and the likelihood per- fectly specified, then i0 E (ym, 3m+1)) where you < yo Um++1 are the closest references and m = E, ri is the num- ber of references ranked below Yo- In practice, the sigmoid in the Bradley- Terry model induces a soft- vs. hard-count mismatch: the rank-only target solves soft-count equation and need not lie in (ym, 1m+
Eada) Value
1.
VC' step
50
sigmold
References (w])s are uniformly spaced
6
(0
noataatinn 0.047
0.12
0.31
0.4
0.50
0.60
0.69
0.79
0.89
Lemma 3.4 (Rank-only MLE may target pseudo ground truth.) Let (,y s(y - w) and m Σ rs. The Bradley-Terry log-likeling ligelioel (Br(y) is strictly con-
Figure 1: Soft- vs. hard-count mis- match from Lemma 3.4. We gener- ate a synthetic reference set {y:) and ground truth yo- From Equation II] the rank-only MLE 06 is the inter- section between Σ (.,1)) and m, and (ym 3m++1 forms the feasible range. We can see that Σ (v()) with step function correctly intersects the feasi- ble range. However, using sigmoid function results in biased intersec- tion outside the feasible range.
cave and its unique maximizer 7/ satisfies
(br( -
Σ (4())) m. (11)
Because is hard count while u, sigmoid soft counts, the solution Σi= (176') m is a pseudo target do- which can lie outside (Um,8m++) (m++1) even under a perfectly accurate ranker, biasing 1/6 See Figure for illustration. O
Bias from the rank-only target can increase the overall error the rank likelihood dominates the regressor likelihood.
Rank likelihood dominance is governed by its curvature (Fisher information grows with k.
Ly et al. 2017)), which
Lemma 3.5. (Rank curvature (Fisher information) scales with k.) The Fisher information of the rank log-likelihood is L (v) Σk (1(y) uz())), so L grows linearly with k. Moreover, 0 < Trank (v)
4

---

Published as a conference paper at ICLR 2026
Lemma 3.6. (Info-weighted Newton step and dominance.) As l grows, the optimization process using Newton's 's method (Murphy 2022) to solve Equation 4 is dominated by the rank likelihood term. That is, denoting ()) and sor (3) the total gradient and Fisher information of both likeli- hoods at Frey as the Fisher information of the regressor likelihood, and j"a (y) as the target of the ranker likelihood term at y, Newton step is
Itot (y) Ire 96 + Lrank (V) (7) (y)
(12)
Tre + lank (1)
which is information-weightee average using Fisher information as the weights.
Since Leg = 1/ is constant, Lemma 3.5 implies the rank likelihood term eventually dominates as k grows. Apart from Newton, for first-order step y 6 y 7haa (g), the same decomposition yields < + 7 reg (i% y) Trank (y) (17 v)), showing similar information-weightte pull. The dominance of the rank likelihood then follows from Lemma 3.5.
Dominance alone is harmless, but combined with Lemma 3.4, biased rank likelihood target ju can steer the refinement away from yo- Note that dominance occurs only when both & and the ranker accuracy are sufficiently large. Moreover, the accuracy threshold decreases with k.
Lemma 3.7. (Accuracy-k threshold for rank dominance.) Define
p
1 (yo > vi).
ply)
(yy).
(13)
Let a E [0.5, 1] denote the ranker accuracy. Then, at any iterate the rank likelihood dominates the regressor likelihood whenever
1
(14)
2 2k
p(y)
The right hand side of the inequality is the threshold accuracy Ont(y), and it decreases as 1/k for fixed y, of and 1p" 11p-~00p -py)] 0. Therefore, the required ranker accuracy such that the rank term dominates the Newton step shrinks as grows.
Corollary 3.8. (Rank likelihood can degrade base RANKREFINE++.) Under the base formu- lation of RANKREFINE++ the rank-only target can be biased (Lemma 3,4). This may lead to performance degradation when the ranker likelihood term dominates (Lemma 3.6). In general, ranker likelihood dominance grows with the reference set size k and the ranker accuracy a. When k large, the required for the ranker likelihood dominate lower (Lemma 3.7).
3.3 RANKREFINE+4 WITH CALIBRATED EXPERT RANKINGS
The degradation stems from the Bradley-Tere Terryy modeling mismatch (Lemma 3.4): when many pair- wise gaps (ya yv) fall on the sigmoid's transition region rather than its saturated tails, the rank-only target becomes biased. This is unit-scale issue; datasets with small average ly. yo! have higher probability of placing many pairs on the transition slope.
We address this by introducing temperature to adjust the slope, i.e., replace u s(y 91) with (y; T) s(y - 7.)/T). This (i) aligns the logistic slope with label units so that the tempered match the observed count m Σ, ri, removing the soft- hard-count mismatch, and (ii) sets the rank curvature to
score Σ, (58;;
Frank (y: T)
Σ (. (y))))).
(15)
thereby controlling rank dominance at large k. We estimate for a dataset via a one-parameter logistic fit on the reference set, thus requiring extra labeled data,
Pr(r = 1 Va: y6; ()) = s(W(y, - y%)). (Va, yb) E D,
(16)
and set 7 feal 1/6. With this calibration of expert rankings, the Newton update is an curvature,
information -eooittioo-weightedd average with cemperature-controle
y Lrog 2/ Lrank (3:: T) (3:
(17)
1rog + mran (y, T)
5

---

Published as a conference paper at ICLR 2026
Input
Output
.T0
Expert Ranker
Bradley-Terry with Calibration (7 & Goting)
Query Item
Refined Prediction Mn
Zo
Reference Data (D)
Pretroined Regressor
Posterior
.re Do
Regressor Likelihood
Base Prediction
Vo
ere
Figure 2: Illustration of RANKREFINE++ For a query To, R DANKKEFINE++ the pairwise rankings from an expert ranker and converts them into ranker likelihood via the calibrated Bradley- Temry model. The ranker likelihood is then fused with the regressor likelihood, and optionally with a prior, to define a posterior. The refined prediction 30" maximizes the posterior distribution.
S (1s (un()1:))
where (") r) = y +
Setting 1 increases the rank likelihood curvature (via the - factor) and shifts the update toward the rank target. When the ranker is accurate, this permits rank dominance without the bias highlighted by Lemma 3.4, improving performance over the naive RANKREFINE++
On the other hand, when the ranker is less accurate, overly large curvature would let noisy rank signals dominate and harm performance. We therefore apply an accuracy-aware soft gate;
r(a) = 1 + (fca 1)))()))
721,
(18)
w(a) = max(0, 2a E [0.1].
(19)
Thus T(a) R 1 when & 0.5, T (()) + Feal as - 1, and intermediate produces smooth inter- polation. The final RANKREFINE++ with calibrated expert rankings is summarized in Algorithm ! and illustrated in Figure 2
Algorithm RANKREENNEE Algorithm
Inputs: To. T6 02 G ((x,, & D Hi R; (optional: p(v))
Collect Comparisons: r; (R x44)
Estimate Temperature: fit s(a(y. 9))) on labeled pairs from DD; set feal 1/a. Estimate the ranker accuracy a; set T +- + (Feal (mxx (0,20 2a 1))7
MAP / MLE Optimization: Maximize Equation wwth s(-) replaced by s((-)7). With Newton steps described on Equation 17 iterate to convergence.
Output: ø and approximate uncertainty of
(Freg + Lran (Wi:T)) 7)
EXPERIMENTS
To demonstrate the benefits of RANKREFINE++ in data-scarce domains, evaluate on 9 molecular property prediction datasets from the TDC ADMET regression task (Huang et al. (02T): Caco-2 Wang et al. 2016)) Clearance Microsome and Clearance Hepatocyte (Di et al. 2012)) log Half-Life Obach et al. 2008). FreeSolv (Mobley & Guthrie 2014). Lipophilicity (Wu et al. 2018), PPBR, Solubility (Sorkun al. 2019). and VDss (Lombardo Jing 2016). For each experiment, we sam- ple N = 100 molecules from the original training split and -= 100 molecules from the original test split. We repeat this train/test resampling for 10 random seeds, similar to a Monte Carlo cross- validation. The reference set MC is sampled from the training set with & E {3, 10, 20, 30, 50, 100). Two types of base regressors are used: random forest (RF) (Ho 1995) and multilayer perceptron (MLP) (Rumelhart et al. 1986), both trained the N training labels. We also three tabular gression datasets: crop yield prediction from sensor data (Soundankar (025), -erffon-pe prediction (Cortez 2014), internationaleeducatioo cost estimation (Shamim, 2025

---

Published as a conference paper at ICLR 2026
__ann
Ciaarancs_Happaloeyye Hapalocyta_4
Basoranne_Mirrooomm__oo Microsoma
1.00
1.00
1.00
Rerttette (0)
MAE Radi
0.75
0.73
MNE Rato
0.70
0.60
0.50
0.50
0.25
0.25
0.26
0.5
0.6
0.7
0.8
0.9
1.0
0.5
0.6
0.7
0.5
0.9
1.0
0.5
0.8
0.7
0.8
0.9
1.0
Ranker Accuracy
Ranker Accuracy
Renker Accurasy
Hell_Life_Obaer Obach
HydrationFrreeEneegyy Sroeeolv
LipophiiiityAAstrazeeei
1.00
1.00
1.00
MAE Farte (0)
MAE Ruse (8)
0.75
0.75
0.76
MAE
0.60
0.50
0.50
0.25
0.25
0.26
0.5
0.6
0.7
0.8
0.9
1.0
0.5
0.6
0.7
0.8
0.9
1.0
0.5
0.8
0.7
0.8
0.9
1.0
Raniket Ancuraty
Ranker Acturacy
Reriker Accuraty
PZ
Sulubiill_AASollB
VDes_Lomberdo
1.00
1.00
1.00
MAE Festo (8)
0.75
0.75
MIE Rotio
0.76
MAE
0.50
0.50
0.50
0.25
0.25
0.26
0.5
0.6
0.7
0.8
0.9
1.0
3.5
0.6
0.7
0.8
0.9
1.0
0.5
0.8
0.7
0.8
0.9
1.0
Ranker Ancurany
Renker Acturacy
Renkee Ancuraty
Regnessor-cnly
Projection
Bracley-Terr
Thurstone
RaniRetine
RaniSteting+
Figure 3: MAE ratio 3 (lower is better) as function of oracle ranker accuracy a nine TDC ADMET datasets 30). We report the mean and standard deviation across random splits. RANKREFNN+++ (MLE-GatedTTemp variant) outperforms projection and RankRefine across ranker accuracies, with especially strong gains at moderate, real-world accuracy. Results for other values with similar trends are available in Appendix.
We evaluate four RANKREFINE++ variants: MAP with a Gaussian prior, MLE, MLE with temper- ature scaling (MLB-Temmp), and MIEE with temperature scaling and soft gating TML.E-GatedTTemp The full RANKREFNNE++ i.e., GatedTteedemp the default variant we use in the experiments. We compare method to two post-training regression enhancement baselines: (i) Projection- based approach (Yan et al. 2024) and (ii) RankRefine (Wijaya et al.) 2025). Additionally, we adapt two pairwise ranking models to create rank-only regression enhancement baselines: (i) Bradley- Terry (rank-only MLE using the Bradley-Tere model), (ii) Thurstone (rank-only MLE using the Thurstone-Mosteller model (Thurstone) 1927 Mosteller 195T)). Following |Wijaya al. (2025), we report 3 = MAEpos/ MMABnser which is the ratio between the post-enhancement Mean Absolute Error (MAF) and the MAE of the regressor-only predictions (lower is better).
We use two types rankers: (i) oracle ranker to study the effect of ranker accuracy, and (ii) LLM ranker to demonstrate real-world use cases. For the oracle ranker, we r; [ y:) and then flip the outcomes with probability 1 - to simulate a ranker with accuracy a E [0.5, 1]. For LLM rankers, we prompt aublicly-available models compare a list of pairs of molecular text representations (SMILES (Weininger 1988) and measure the accuracy
small validation set.
Additional discussions and experiments on LLM usage, temperature calibration, multi multi-target regres- sion, and reference set size are available in Appendix.
4.1 MAIN RESULTS WITH ORACLE RANKERS
Across all nine ADMET datasets at k 30, RANKREFINE++ (MLE-GatedTemm variant) consis- tently outperforms previous otate-tf-the-art, RankRefine, and other baselines (Figure B) Gains recent custom-built enhancement methods, RankRefine and Projection, are largest in the mid-

---

Published as a conference paper at ICLR 2026
k=3
k=20
1.00
1.00
1.00
MEE Rato (5)
0.75
MAD Resto
0.75
0.76
MA4
0.00
0.7
0.8
Accuracy
Accuracy
Ranker Accuracy
k=30
k=50
k= 100
1.00
MARE Patto 68)
1.00
1.00
WE Rate
0.50
0.50
0.25
0.25
0.25
0.00
0.7
0.6
0.00
0.6
1,0
0.00
Ranker
Ranker
Ranker
Rugresaor-orly
MLE
MLE-Tump
MLE. NLE-CatedTeerp Ranalattrer
Figure 4: Effects of reference set size and temperature scaling / soft gating on the Clearance Hepatocyte dataset. The shown methods are the four variants of RANKREF R with ours being MLteGatedTemp. We show the MAE ratio 3 (lower is better) as function of oracle ranker accuracy for k {3, 10, 20, 30, 50, 100). Without temperature scaling, MAP and MLE variants degrade at larger due to rank curvature dominance. Temperature (MLE-Temp) fixes the scale mismatch, and soft-gating
further improves robustness when ranker accuracy is low.
accuracy regime (65% 90%). At lower ranker accuracies (<65%), RANKREFINE++ matches or exceeds RankRefine and clearly outperforms Projection. high ranker accuracies, RankRe- fine performance plateaus, while RANKREFINE++ continues to improve, converging to Projection's performance at perfect (yet unrcalistic) ranker accuracy.
4.2 ABLATION: PRIOR, TEMPERATURE, AND ACCURACY-AWARE SOFT GATING
Figure shows that MAP/MLE variants RANKREFINE++ can degrade as A grows, confirming that momk-dmminncce in the Newton step and the soft-hard count mismatch the rank term bias the enhancement (Corollary 3.8). The rank-only Bradley-Tere and Thurstone models in Figure also exhibit similar degradation, validating that the source of degradation is the ranker likelihood. Interestingly, MAP degrades less than MLE, indicating that prior can act as a regularizer. As pro- posed in Lemma 3.7, the ranker accuracy threshold which degradation appears shifts lower increases.
Our full method (MLE-Gate Gate--Gatddempp variant) solves the performance degradation issue. Temperature scaling aligns the sigmoid slope to the label scale and controls the rank curvature, mitigating degra- dation high ranker accuracy, but can worsen performance low ranker accuracy (Figure MLE- Temp). Adding accuracy aware soft-gating removes this trade-off (Figure 4 MLE-Gated Temp), enabling full RANKREFFNE+ to deliver strong performance across the complete ranker accuracy range. Note that Figure 4 also confirms that our enhancement method works with a reference set as small as = 3, which is highly practical in the real-world setting.
4.3 CROSS- CROSS-DOMAIN DOMAIN GENERALITY AND REGRESSOR DIVERSITY
To test cross-domain and cross-model generalization, we add three non-molecular tabular datasets and run both Random Forest (RF) and Multilayer Perceptron (MLP) regression models. Figure shows that RANKREFINE++ yields < across the ranker accuracy range, improving both RF and ML.P models, and proving its generalization capability other domains and regression models.
8

---

Published as a conference paper at ICLR 2026
international. education
smart
student performance
MAE Ruiio 351
0.75
2.76
0.75
0.50
0.25
0.25
0.02
0.5
0.6
0.0 Ranker Accuracy
0.7
1.0
0.00
0.6
0.7
0.8
0.9
1.5
0.00
Rannior Accuracy
Ranker Recuracy
Regneasor-only
Projection
RankRafine RandomForest
RankRufire MLP
Rar Renn
Farkitulinee MLP
Figure 5: MAE ratio 3 on three non-molecular tabular datasets with Random Forest (RF) and Multi- layer Perceptron (MLP) as base regressor models. We show that the refinements are generalizable to other domains and regressor models. Baseline (non-refined) MAEs are: International Education Cost: 0.0926 (MLP), 0.0609 (RF); Smart Farming Yield: 0.1448 (MLP), 0.1361 (RF); Student Exam Performance: 0.0727 (MLP), 0.0819 (RF).
Dataset Name
Half Life
FreeSolv 62.27 + 2.00
PPBR 63.29 + 2.65
Solubility 62.87 = 3.27
VDss 65.95 = 1.96
ChatGPTS PRA (%)
62.20 + 1.89
RANKREHINE++ (8)
0.952 0.057
0.997 0.021
0.928 0.045
0.985 0.015 0.985 0.020
0.853 0.194
RankRefine (8)
0.977 0.024
1.002 0.022
0.968 0.016
0.952 0.055
Projection (8)
1.056 0.171
1.119 + 0.100
1.007 0.026
0.992 0.039
0.949 0.082
Dataset Name
Half Life
FrecSolv
PPBR
Solubility
VDss 60.85 + 2.12
Claude4 PRA (%)
52.36 + 2.22
72.47 + 1.40
51.73 + 3.25
60.41 = 2.45
RANKREFINE++ (8)
0.955 0.058
0.977 0.011
0.965 0.042
0.990 0.010
0.850 0.191
RankRefine (B)
0.981 0.010
1.070 0.052
0.979 0.019
0.988 0.016
0.951 0,056
Projection (8)
1.124 0.279
1.153 0.126
1.031 0.039
0.995 0.014
0.959 0.038
Table I: Results using LLMs (ChatGPTS, Claude4) as external rankers, measured in over ten train/test splits with N 50, k = 20. PRA stands for pairwise ranking accuracy. Best result for each dataset-ranker is bolded. RaNkREhNE++ generally outperforms other refinement methods.
4.4 LLM AS IMPERFECT YET PRACTICAL RANKER
We replace the oracle ranker with off-the-sheit LLMs obtain noisy but scalable expert rankings. With ChatGPTS Thinking and Claude Sonnet 4 as rankers at N 50 and k = 20, RFN generally achieves the best B across five ADMET datasets. Detailed results are shown in Table
4.5 RUNTIME ANALYSIS ON VARYING REFERENCE SET SIZE
Excluding the ranking predictions, RANKREFINE++ runs in less than ms on an Intel i7-13700 CPU when the reference set size l: < 1000, covering realistic scenarios in data-scarce domains. The average running time to predict expert rankings using SOuGGPTS-Thinking for 1000 queries and 50 references (50,000 pairs in total) in parallel is 152.4 54.7 seconds. Therefore, RANKRE- FINE+ can be run efficiently on consumer-grade computes, and LLM inference is not significant bottleneck.
4.6 RANKREFINE IS A SPECIAL CASE OF RANKREFINE++.
We showed that RankRefine (ijaya et al. 2025 is special case of RANKREFNE++ in Propo- sition 3.3. We verify this empirically by comparing enhanced predictions from RankRefine and RANKREFINE++ under the Gaussianity assumptions. Across five ADMET tasks, the mean absolute difference A, is at the level machine epsilon (Table 2 confirming that Rankrefine is a special case of
with Gaussianity assumptions.

---

Published as a conference paper at ICLR 2026
Dataset Name Ay 10-
Half Life 0.028 + 0.028
FreeSolv 0.030 + 5.71
PPBR 8.51 1.42 0.029 = 1.33
Solubility
VDss 3.24 + 2.96
Table 2: Mean absolute difference between the refined predictions of RankRefine and RANKRE- FINE++ with Gaussianity assumptions. The values are around the range of machine epsilon for floating point precision, confirming that RankRefine special case of RANKREFINE++.
5 LIMITATIONS
While RANKREFINE++ significantly outperforms prior arts, there are several aspects that can be explored further in future works.
Noise model for the oracle ranker. We inject pairwise flips uniformly reach the desired oracle ranker accuracy. Real rankers (human or LLM) might exhibit systematic biases, which could negatively affect the Bayesian inference at lower ranker accuracies.
Dependence on regressor uncertainty. RANKREFINE++ requires a base regressor that can quantify uncertainty, which many off-the-shee regressors lack. A straightforward so- lution to use ensembling or Monte Carlo dropout.
Reference-set selection. Regression improvements may depend on the composition of the reference set (coverage, label diversity, resolution). Future works explore more sophisticated, sequential sampling strategy to select the references.
CONCLUSION
We introduced RANKREFINE++, framing post training regression enhancement a problem of Bayesian inference rather than heuristic fusion. This perspective not only generalizes prior state- of of-the-art, but also reveals subtle failure modes, explains when and why performance degrades, and offers principled solutions through temperature calibration and gating. Empirically, RANKRE- +INE++ delivers consistent gains across diverse datasets, operates effectively with both oracle and noisy LLM rankers, and runs under ms per query. We believe RANKREFINE+++ can serve as a practical bridge between label-scarce applications and the growing availability of pairwise signals from LLMs.
REFERENCES
Alice EA Allen and Alexandre Tkatchenko. Machine learning of material properties: Predictive and interpretable multilinear models. Science advances, 8(18)ceab77185 2022.
Anthropic. Anthropic api guide. 2025. Accessed: 2025-09-16.
Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324-345, 1952.
Manuela Cattelan. Models for paired comparison data: review with emphasis on dependent data. Statistical Science, pp. 412-433, 2012.
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances neural information processing sys- tems, 30, 2017.
William G Cochran and Sarah Porter Carroll. A sampling investigation the efficiency weighting inversely the estimated variance. Biometrics, (44::44-459, 1953.
Paulo Cortez. Student Performance. UCI Machine Learning Repository, 2014.
Li Di, Christopher Keefer, Dennis Scott, Timothy Strelevitz, George Chang, Yi-An Bi, Yurong Lai, Jonathon Duckworth, Katherine Fenner, Matthew D Troutman, al. Mechanistic insights from comparing intrinsic clearance values between human liver microsomes and hepatocytes to guide drug design. European journal of medicinal chemistry, 57:441-444 2012.
10

---

Published as a conference paper at ICLR 2026
Google. Gemini developer api. httpp::/ai. http://tt///.....oo....///////aa
2025. Ac-
cessed: 2025-09-16.
Taicheng Guo, Bozhao Nan, Zhenwen Liang, Zhichun Guo, Nitesh Chawla, Olaf Wiest, Xiangliang Zhang, et al. What can large language models do in chemistry? comprehensive benchmark eight tasks. Advances in Neural Information Processing Systems, 18559662-59688 2023.
Tin Kam Ho. Random decision forests. In Proceedings of 3rd international conference on document analysis and recognition, volume pp. 278 278-282. IEEE, 1995.
Eva JI Hoeijmakers, Bibi Martens, Babs MF Hendriks, Casper Mihl, Razvan L Miclea, Walter H Backes, Joachim E Wildberger, Frank Zijta, Hester A Gietema, Patricia J Nelemans, et al. How subjective image quality assessment becomes surprisingly reliable: pairwise comparisons instead of likert scale. European Radiology, 34(7):44944455) 2024.
Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor Co- ley, Cao Xiao, Jimeng Sun, and Marinka Zitnik. Therapeuties data commons: Machine learning datasets and tasks for drug discovery and development. Advances in neural information process- ing systems, 2021.
Qiaohao Liang, Aldair Gongora, Zekun Ren, Armi Tihhonen, Zhe Liu, Shijing Sun, James R De- neault, Daniil Bash, Flore Mekki-Berrada, Saif A Khan, et al. Benchmarking the performance of bayesian optimization across multiple experimental materials science domains. npj Computa- tional Materials, 7(1):188, 2021.
Franco Lombardo and Yankang Jing. In silico prediction of volume of distribution in humans. extensive data set and the exploration of linear and nonlinear methods coupled with molecular interaction fields descriptors. Journal of chemical information and modeling. 56(10):2042-2065 2052, 2016.
Alexander Ly, Maarten Marsman, Josine Verhagen, Raoul PPP Grasman, and Eric-Jan Wagemmak- ers. A tutorial on fisher information. Journal of Mathematical Psychology, 80:40-55, 2017.
David L Mobley and Peter Guthrie. Freesolv: database of experimental and calculated hydration free energies, with input files. Journal of computer-aided molecular design, 28:711-720 2014.
Frederick Mosteller. Remarks on the method of paired comparisons: Iii. of significance for paired comparisons when equal standard deviations and equal correlations are assumed. Psy- chometrika, 16(2):0772118, 1951.
Kevin P Murphy. Probabilistic machine learning: an introduction. MIT press, 2022.
Scott Obach, Franco Lombardo, and Nigel J Waters. Trend analysis of database of intravenous pharmacokinetic parameters in humans for 670 drug compounds. Drug Metabolism and Disposi- tion, 36(7))1385-11405 2008
OpenAl. Openai api reference.
https::/platfoo openai.............. www.doo.//ooooooooooooooo......c apt-reference//eeeeee int woouctouuioooooo 2025. Accessed: 2025-09--6.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Yan, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, et al. Large language models are effective text rankers with pairwise ranking prompting. In Findings of the Association for Computational Linguistics: NAACL 2024, pp. 15104-118,,, 2024.
Jennifer Routh, Sharmini Julita Paramasivam, Peter Cockcroft, Sarah Wood, John Remnant, Corélie Westermann, Alison Reid, Patricia Pawson, Sheena Warman, Vishna Devi Nadarajah, et al. Rating and ranking preparedness characteristics important for veterinary workplace clin- ical training: a novel application of pairwise comparisons and the elo algorithm. Frontiers in Medicine, 10:1 128058, 2023.
11

---

Published as a conference paper at ICLR 2026
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back- propagating errors. nature, 323(6088):5333 1986.
Benjamin Sanchez-Lengeling and Alán Aspura-Guzik. Inverse molecular design using machine
learning: Generative models for matter engineering. Science, 361(6400)3300365,, 2018.
Adil Shamim. Cost of international education, 2025. URL https::/ httpp:::/w....aaegeeeee www.ww...w.............w........
com/
Batasets/adi ootar//nnnnoooooo-o-etoeteooooon
Murat Cihan Sorkun, Abhishek Khetan, and Süleyman Er. Aqsoldb, a curated reference set of aqueous solubility and 2d descriptors for a diverse set compounds. Scientific data, 6(1):143, 2019.
Atharva
Soundankar.
Smart farming sensor data for yield prediction, 2025.
URL
wttp://........atatte//aaaaaaaaa smart - farming sensor- data -ort--------eell--eeddcti
Michael Sun, Gang Liu, Weize Yuan, Wojeiech Matusik, and Jie Chen. Foundation molecular grammar: Multi-modal foundation models induce interpretable molecular graph languages. In International Conference on Machine Learning. PMI.R, 2025.
LL Thurstone. A law of comparative judgment. Psychological Review, 34(4):273-28) 1927.
Ning-Ning Wang, Jie Dong, Yin-Hua Deng, Min-Feng Zhu, Ming Wen, Zhi-Jiang Yao, Ai-Ping Lu, Jian-Bing Wang, and Dong-Sheng Cao. Adme properties evaluation drug discovery: prediction of caco-2 cell permeability using a combination of nsga-ii and boosting. Journal of chemical information and modeling, 56(4):763-75) 773, 2016.
David Weininger. Smiles, chemical language and information system. introduction to method- ology and encoding rules. Journal of chemical information and computer sciences, 28(1):31-36, 1988.
Kevin Tirta Wijaya, Minghao Guo, Michael Sun, Hans-Peter Seidel, Wojeiech Matusik, and Vahid Babaei. Two-stage pretraining for molecular property prediction in the wild. arXiv preprint arXiv:2411.03337, 2024.
Kevin Tirta Wijaya, Michael Sun, Minghao Guo, Hans-Peter Seidel, Wojeiech Matusik, and Vahid Babaei. Post hoc regression refinement via pairwise rankings. arXiv preprint arXiv:2508. 16495, 2025.
Tong Wu, Guandao Yang, Zhibing Li, Kai Zhang, Ziwei Liu, Lconidas Guibas, Dahua Lin, and Gordon Wetzstein. Gpt-4y (ision) human-aligned evaluator for text-to-3d generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 22227- 22238, 2024.
Zhengin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. Moleculenet: benchmark for molecular machine learn- ing. Chemical science, 9(2):513 13050,0 2018.
Le Yan, Zhen Qin, Honglei Zhuang, Rolf Jagerman, Xuanhui Wang, Michael Bendersky, and Harrie Oosterhuis. Consolidating ranking and relevance predictions of large language models through post-processing, In Proceedings of the 2024 Conference Empirical Methods in Natural Lan- guage Processing, pp. 410 423, 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li. Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595-46623, 2023.
Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint 1 2019.
12

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.I DETAILED PROOFS
Lemma 3.1. Proof. The regressor log-likelihood contributes 25 (10 yo)² with curvature
-1/0 < 0 Each BT term adds logs(yo y:) or log =s30 vi), whose second deriva- tive is -s(1 -8) 8) S O. Sum concave terms and concave prior concave; the Gaussian term's strictly negative curvature makes the sum strictly concave.
Lemma 3.4. Proof. (y) (Proposition 3.3.) strictly decreasing, therefore, (BT is strictly concave. Setting (3) = 0 gives the equation. Since F(y) = >; (y) is continuous and strictly increasing with (0) = 0, F(cc) = k, there a unique solution at the observed count m.
Let 1/1 ... S Y/ be the ordered reference set and suppose the true yo (ym Mm+1).
if most (y y1) in the sigmoid saturated regime, F behaves like hard count, so F(yo)= s m. F(jo)= = m, and yo = to R yo up to the resolution (ym+1 ymm)
if many (y - ys) lie in the transition region, F(yo) # m, F(jo) = m, and y0 3o 76 yo- This shifts does not induce rank bias as long as yo (ym Vm+1).
rank bias arises if the shift exits the interval, i.e., Do yo! 2 min ym, y 1 - yo ) which is more likely when the number of (y y.) that lie in the transition region is high.
Lemma 3.6. Proof. Let Tee 1/ be the Fisher information of the regressor term and write the total gradient and Fisher information at
(y) grog (1) Irank (y)
(20)
(y 16
Σ (r. (z)),
02
Ior(y) = Log + Lrank (3).
(21)
Define the local rank target for single Newton step
(r) y Grank (y)
Trank (1)
(22)
+ Ek (r; vz()))
Irank(
By substituting Grey (1) = Ireg (0 - y) and Grank (y) Irank (2)
(17 - y), a single Newton
step is an information-weighted average as written in Equation 12
Lemma 3.7. Proof. Using [[[;]] = (1 a) + (2a- 1)1(1o > yı) to model flip errors in a noisy binary ranker gives
E. Trank (v) Σ E(re] Σ ui()) k ((1 a) (2a 1)p" p(v)).
(23)
pf 1/22 p(v) pr < 1/2 < p(y), then
F Elgnank (1)
2 k(2a 1)(p" - -p(y)
(24)
Substituting Grank (1) with -1/0 16 the expected Newton step is rank-dominated whenever
y
(2a 1)ap1 p(v)) >
(25)
ko
13

---

Published as a conference paper at ICLR 2026
A.2 USE OF LARGE LANGUAGE MODELS
We large language models (LLMs) (i) polish writing (restructuring sentences and proofread grammars and typos), (ii) search for related works, (iii) predict pairwise rankings of molecule pairs. For writing and search, ChatGPT5. For pairwise ranking predictions, ChatGPT5 and Claude Sonnet 4 (Copilot version). The prompt that we used for pairwise ranking prediction is as follow,
You are expert molecular reasoning model tasked with predicting pairwise rankings of molecules based on a described molecular property of interest (e.g., solubility, polarity, etc.).
You will given json files containing the test molecules to be compared with the reference molecules. The property of interest is described within the json files with the key "description".
Your Task Follow These Steps:
1. Read the dataset's description to understand the molecular property being ranked (e.g., "higher solubility", "lower toxicity"). Be careful with the mesurement unit. For example, higher number in IC50 could mean lower toxicity.
2. For each molecule pair (test molecule vs. reference molecule): Use your internal knowledge to infer which molecule ranks higher for the property.
You may use structural patterns, substrings, atom types, SMARTS-like features, coken-level patterns, or other insights you may have.
You are encouraged to develop your own heuristics or scoring logic using Python.
3. Assign a "pairwise_rank" to each pair:
1 a test molecule ranks higher than reference, meaning test_ test_property > deference_propert.
0 - test molecule ranks lower equal to reference Caution! Only care for the value of the property. For example, if texicity is measured in IC50, and test_molecule IC50 value
IC50 value, you should output 1. 4. Save your results as new JSON file with similiar structure.
is greater than reference_molecus
You Are Allowed To:
1. Write your own Python logic.
2. Use basic Python and string-based pattern recognition
3. Think step-by-step to develop useful ranking heuristics
4. Use helper functions from ChemInformatics libraries, as long
as you do not use them to directly predict the porperty of interest values.
* You Are NOT Allowed To:
1. Use the internet to search for the property values
2. Use cheminformatica libraries like RDKit to directly predict
the property of interest values, e.g., solubility of molecule for solubility datasets.
3. Access files not specified in this prompt
Bach entry in your output JSON should look like this: "test_molecule"" "smiles"; "...", "reference molecules"; "reference_noieecnles:::
{ "id": "..." "smiles": "..."
"pairwise_ranks" 1,
14

---

Published as a conference paper at ICLR 2026
that is, put the pairwise_rand predictions inside the reference reference_moleculee molecule. Your output json file should be named cetreedicciinn..o htt.ppprrrrd..ooss_ssstt.i.iiii. jsson"
Tips for Better Performance:
1. Think aloud: before you begin ranking, describe what is the property of interest and why one molecule might rank higher based on your knowledge
2. Use token or substring patterns (e.g., "more OH groups" or "more aromatic rings")
3. Define scoring rules: e.g., "count ('0r) count ('N')"
to estimate polarity
A.3 EFFECTS OF NUMBER OF SAMPLES FOR TEMPERATURE CALIBRATION
In Section 3.3, the temperature is set ical, the cal- ibrated value estimated from the labeled reference set. natural question is the robustness of this procedure in data-scarce regimes, where only few labeled refer- ences are available. Sigure fcal as a function of the calibration set size Keal. Across the 9 TDC ADMET datasets, Fcal converges at around keal Rs 10, indicating that optimal calibration can be obtained with only 10 la-
PPBR, AZ
Clarancc Hepatocyeeeeeeeeeeeo AZ
Clearance Microsome
Caco2_ Mroo_Wangggg
0.8
VDss. Loss_rooadddo
Half, Life_Obach Obach
Fiteed Temparature (t)
HydrationFreeEneryyyeeee FreeSolv
Uppopiiiicity AstraZenieaa Lppphiiiciy_AttrzZeeeea
0.6
Solubility Aq
beled samples. Furthermore, except for PPBR_AZ, the estimates obtained with Keal 10 are already close to their converged values. This suggests that fcal remains reliable when very limited calibration data are avail- able. Consistent with this observation, in the experiment where fix the reference set size at 50 but vary Keat E [3, 50] (Figure 7. the resulting performance curves are nearly identical. Together, these results demonstrate that the number of references used for temperature cal- ibration has negligible effect on overall regression en- hancement performance.
0.0
10
20
Number Samples for Temperature Calibration (k_
A.4 REGRESSION ENHANCEMENT FOR MULTIPLE TARGETS
Figure 6: Effects of number of samples used to calibrate the temperature on the value of feal-
The discussion and experiments in Sections 3 and 4 fo- cus on scalar regression tasks. In many practical settings, however, the goal is predict multiple properties simul- taneously, i.e., to produce vector-valued outputs. Extend- ing RANKREFINE++ to this setting is straightforward: one can assume independence across output dimensions and apply RANKREFINE++ separately to each component of a target vector y Rd yielding d independent regression enhancement processes.
In practice, the components of y are often correlated. For example, in molecular chemistry, aqueous solubility and membrane permeability are often related, as are clearance and half-life. In such cases, dependencies among outputs can be captured by employing a low-rank approximation of the covariance structure, which enables a computationally efficient implementation. Exploring this extension is an interesting direction for future work.
A.5 MORE RESULTS ON TDC ADMET
We show results for more & values in Figure g Figure 16 In general, RANKREFINE++ can improve the regression error (i.e., / < with a reference set size as small as k = 3.
15

---

Published as a conference paper at ICLR 2026
PPBR, A7
Clearance Hepatocyto A7
Clearance Micrssame A7
Lanc
MAE Ratio 18
0.
MAE Ratio (B)
MAR Ratio
La0-13
0.0
37
10
13
0.8
15
3.0
03
0.9
67
10
Ranker Accuracy
Ranker Accuracy
Ranker Accuracy
Caco2 Wang
VD Lombardo
Half Life. Obach
MAE Ratio (0)
MAC Rotio (8)
Loa 418
Laa-ds
02
02
6.
0.5
0.8
37
0.9
1.3
0.5
05
07
3.
6.
1.0
35
05
07
33
3,5
3.0
kor Acouracy
Accuracy
Rankerr Accuracy
HydrrtioonFreennenyy
Lipophilicity
Solubility
1.0
1.2
MAE Ratio (8)
0.C-
MAE Ratto (P)
0.5
MAF
L
Beyssioren
Baylstur
Seysturaty
Accuracy
Nogreasiin aeneetcc eer wnaaas siic be-00
Ranker Accuracy
Figure 7: Effects of number of samples the regression enhancement. We measure 3 (lower is better) as a function of number of samples used to calibrate the temperature Keal- The reference set size l is set 50.
k+3
k=10
k=20
1.00
1.50
1.30
MMB 201o
MAB Date
0.50
0.50
0.50
0.00
0.6
0.7
1.0
0.00
0.5
077
0.8
0.20
0.6
077
1.0
Ranker Annumayy
Ranker AcsiiNtS
Barker Arounayy
k-30
k=50
Ioe k=100
1.00
1.50
MAR Satt
0.75
0.76
MAT Sate
0.75
0.50
0.28
0.00
0.7
0.30
0.4
0.30
0.7
Barker
Banker
Tarrer
Zngreeecc-cctly
Projection
Bradley-Tary Jamr
Thurdions
Rantflaine
Rank/leltma
Figure& 8: More results with varying reference set size (k) for the Caco2. Wang dataset.
16

---

Published as a conference paper at ICLR 2026
0.76
0.76
MAE Sato
0.76
MA5
0.25
0.25
0.25
Ranker
Rarker
k-50
k-100
1.00
1.00
1.00
0.75
0.76
0.30
MAE
0.25
0.23
0.00
0.0
0.7
0.0
1.0
0.6
0.7
0.8
0.0
1.0
0.6
0.7
0.9
1.0
Acourass
Acouledy
Accurads
Higresace-orty
Projection
Bradley- Terry
Thursions
Hartflatin
Bankitatinar
Figure 9: More results with varying reference set size (k) for the Clearance dataset
K-3
k-10
k-
1.00
1.00
1.20
0.75
MAT Dato
0.50
0.50
0.50
0.00
0.6
0.7
0.50
0.5
0.8
0.30
0.6
Rarker Acourasy
Ranker Acouradyy
Rarker Acouracy
k-30
km50
km100
MAT Deto (8)
MA E3000
0.75
0.75
0.75
0.25
0.25
0.25
0.00
0.30
Rarker
Ranker
Rarker
Engreoocevarly
Projection
Beadiay-Taryy
Thundone
RankRwine
RarkBerines
Figure 10: More results with varying reference set size (k) for the Clearance Microsome AZ dataset
K-3
k- 10
k=20
1.00
(F)
1.00
1.50
MAB 26100
MAB Dalto
0.50
0.25
0.25
0.5
0.5
0.7
0.8
3.9
1.0
0.5
0.8
0.7
3.8
0.9
1.0
0.5
0.5
0.7
0.8
0.9
1.0
Rarker Acourasy
Acouraty
Parker Accuracy
k330
k=50
Ice 100
1.00
1.50
1.30
0.50
0.50
3.00
0.00
0.5
0.7
0.0
1.0
0.00
0.5
0.5
0.7
0.8
1.0
0.20
0.5
0.5
0.7
0.9
1.0
Rarieer Acouraty
Panket Accuracy
Parrker Accuriity
Pagreeoce-arly
Projection
Bradley-Terry
Figure 11: More results with varying reference set size (k) for the Lafe Obach dataset
17

---

Published as a conference paper at ICLR 2026
3=3
10
k-20
1.00
1.00
1.50
MM Dallo (8)
0.50
0.50
0.5
0.00
0.5
0.6
0,7
0.0
1.0
0.00
0.5
0.0
0.7
0.8
0.9
1.0
0.20
0.5
0.6
0.7
0.8
0.9
1.0
Rarker Accuracy
Parket Accuraty
Parker Agouracyy
k-30
k-50
k=100
1.00
1.00
MAE Satto (1)
MAE S200
0.75
0.76
0.76
0.50
0.50
0.25
0.00
0.30
0.2
0.30
Rarlier Acourasy
Ranker Acourecy
Rarker Accurasy
Ingrexscnectly
Projection
Bradiey-
Thurdione
Rantilatnn
Bankitelinat
Figure 12: More results with varying reference set size (k) for the HydrationFFreeEnngyyFFeee dataset
1.00
(8)
M45 Pallo
0.75
0.75
Callo
MAS
0.25
0.25
0.20
0.5
0.7
0.5
0.8
0.5
0.7
1.0
Acoureyy
k-30
k-00
k-100
1.00
1.00
1.20
MAERHHO
0.75
MAERHHO
0.75
0.00
0.50
0.25
0.00
0.5
0.5
0.7
0.8
3.5
1.0
0.5
0.8
0.7
D.8
0.9
1.0
0.00
0.5
0.5
0..7
0.8
0.9
1.0
Btarker Accuresy
Acouracy
Harrker Accuraec
Projection
Bradley-Tarry
Thurstone
RentRelne
RankGefiness
Figure 13: More results with varying reference set size (k) for the Lipophilictty dataset
K=S
k=10
k-20
1.00
1.50
MAE 3200 (8
0.75
0.76
MAE Satoo
0.76
0.50
0.50
0.25
0.00
0.6
0.50
0.8
0.30
1.0
Rarlicr Acourasy
Ranker Accurety
Rarker Accuracy
MAE 3316
0.76
0.75
0.76
0.25
0.25
0.25
0.8
Rarker
Projection
Bradiey- Terry
Thursions
Hantiteina
Figure 14: More results with varying reference set size (k) for the PPBR AZ dataset
18

---

Published as a conference paper at ICLR 2026
k=3
k=10
k=20
1.00
1.50
MAE Dara 39
MME Rato
Ralt
0.75
0.50
0.50
0.50
0.00
0.6
0.7
1.0
0.50
0.5
0.7
0.8
0.30
0.6
0.7
1.0
Rarker Acouraty
Ranker Acouraty
Rarker Accuraty
k:30
k=60
MA MMEEbateo 883
0.75
0.75
MMT Date
0.75
0.25
0.25
0.29
Ranker
Ranker
Rarker
Rogres2or-orly
Projection
Brodiy- Tony
Thurstone
RomiRolne
RankRoficce*
Figure 15: More results with varying reference set size (k) for the ColubilityAqqoolll dataset
k=3
k=10
k=
1.00
1.00
1.20
14M Rako
0.75
MAE Date
0.50
0.50
0.00
0.5
0.7
1.0
0.00
0.5
0.8
0.00
0.5
0.7
1.0
Rarker Accuracy
Tartker Accurayy
Farreer Accuracy
km60
M4 Ratto 83
MAC Datle 85
0.75
0.75
0.77
0.60
0.50
0.25
0.26
0.25
Btarker
Tarnker
Tbarker
Reg Pngresscr-orly
Projection
Bradley-Tore Temy
Thurstone
RantRofne
RankReffnee
Figure 16: More results with varying reference set size (k) for the VDss Lombardo dataset
19

---

Published as a conference paper at ICLR 2026
A.6 THE CORE BAYESIAN FORMULATION OF RANKREFINE++ WITHOUT CALIBRATION OUTPERFORMS PRIOR WORKS
Calibration and soft-gating offer principled way to maximize the performance of RANKREFINE++. Nevertheless, the core Bayesian formulation of RANKREFINE++ already brings substantial im- provement over the baselines by itself. As shown in Figure plain RANKREFINE++ without calibration or soft-gating outperforms both RankRefine and Projection across the pairwise ranking accuracy.
Caco7_ W_agg
Clearance_Hepatooe Hew.seoetttttttt_oooo
Clearance Microsame
1.00
1.00
1.00
81
M6E Sato (8)
0.75
0.75
0.75
0.25
0.25
0.00
1.
0.00
0.00
0.7
Ranker
Ranker
Rarker
Half, Life. Obach
hlddrationFredEnergy FreeSolv
Lipophilicky Astrazeeca
1.00
1.00
1.00
MAK Radio 083
MAE Rato (5)
0.75
3.50
0.25
0.25
0.25
0.00
1.0
0.00
0.5
0.00
0.7
0.8
0.5
1.0
Barkeu Accuracy
Ramber Accur
Rar Accuracy
PPBR AZ
Solubility, AqseIDB
VDss_ Lombardo
1.00
1.00
1.00
MAl Ra68
0.50
0.50
MAEE
0.50
0.25
0.25
0.25
0.00
0.5
0.6
0.7
0.8
0.9
1.0
0.00
0.5
0.6
0.7
0.8
0.9
1.0
0.00
0.5
0.5
0.7
0.8
0.9
1.0
Ranker
Ranker Accuracy
Ranker Accuracy
Begresier-cnly
Projection
RandSwfine
MLE
Figure 17: Plain RANKREFINE++ (MLE variant) RankRefine and Projection. Reference set size k= 20. Non-calibrated RANKREFINE++ consistently achieves lower error compared to RankRefine and Projection baselines across datasets, often by large margin
A.7 SENSITIVITY ANALYSIS FOR UNDER/OVERESTIMATTIO REGRESSOR UNCERTAINTY
Figure 18 shows the performance of RANKREFINE++ when the pointwise uncertainty of the regres- sor set the pointwise absolute or squared error, scaled by s. Overestimation (e.g., s 10.0) has minimal impact on the performance, while underestimation brings substantial degradation due to excessive confidence the imperfect regressor.
A.8 SENSITITVITY ANALYSIS FOR NOISY ESTIMATION OF THE RANKER ACCURACY
The soft-gating mechanism relies on the estimation of the ranker accuracy. Figure 19 shows the effect of noisy estimation of the ranker accuracy. When injecting the ground-truth ranker accuracy (a 1.0) with Gaussian noise, there is substantial performance degradation up to o² 0.50. Furthermore, for o² > 0.50, the performance starts to slightly degrade only at relatively high ranker accuracy, demonstrating the robustness of RANKREFNNE++ most real-world scenarios.
20

---

Published as a conference paper at ICLR 2026
Wang
Hepatocyto,
Clearance,
1.00
1.00
3.00
MAD Ratto (3)
IMAL Ballo
MME Detto
0.50
0.50
0.50
0.25
0.25
0.25
0.6
0.7
0.7
0.8
3.0
Accurary
Accuracy
Accurary
Halt Life. Obach
HydrottonFrreeEergy FreeSolv
Lipophiliity AstraZeneca
Batto
MA0 Etaloo
MAY Balto
0.50
0.50
0.50
6.25
0.25
0.25
0.7
0.6
ady
PPBR,
Solubility AQSoIDS
Lombarda
2.00
1.00
3.00
Ratto
IMAL Batlo
0.75
MAT Balto
0.50
0.50
0.50
0.25
0.25
0.25
0.7
0.8
3.0
Accuracy
Accuracy
Accurary
MAE,
Figure 18: Sensitivity analysis under overestimation of regressor uncertainty. Overestimating uncertainty (c.g., = 10.0) has minimal impact on performance, while underestimating it (c.g., 0.1) degrades performance due excessive confidence in the regressor.
21

---

Published as a conference paper at ICLR 2026
Caco? Weng
Clearance Heparacyte AZ
Clearance Merroome, 1V
MAE Ratlo (9)
MAC Pirto (11
Noise
prw0.00
gl-0.05
efe0.05
pf-0.10
gf-0.10
ef=0.10
u750.50
g1=0.50
gf-0.0
Ranker Acturacy
Ranker 6F
Ranker
19
HydrationFreeEneny
Upopnilicty
MAE Ratio
MAT Rutio
No nossc
No noise
Nois
6f=0.01
g-0.01
6'+0.01
0'~0.05
g=00.55
6=x0.05
6550.50
9-0.00
e0-050
01=1.00
df-000
1.000
Kanker Accuraty
Banker Acouracy
Ranker Accuracy
PMB2
Solubiity Aqpiolo
Viss, Lombbrdo
MAEE Ratio 151
MAC Darto 101
151
pf=0.01
g1m0.01
0t-0.01
plm0.05
gf-0.05
pte0.05
0010.10
0--0.10
0--0.10
02
82
60
69
80
39
Accuracy
Accuracy
Accuracy
Figure 19: Sensitivity analysis when the estimation of the ranker accuracy is noisy, resulting in non- optimal values. RANKREFINE++ is robust even when using suboptimal tau value caused by softgating bias
22


Reviews:
[{"review_id": "smNMWwIrIb", "reviewer": "Reviewer_rqoX", "summary": "This paper proposes BAYES-ECR (Bayesian Enhancement with Calibrated Rankings), a novel post-training Bayesian refinement framework for regression models using pairwise ranking information. The method integrates the Gaussian likelihood from a pretrained regressor with a Bradley-Terry ranking likelihood, forming a strictly log-concave posterior that enables efficient MAP optimization.\nThe authors identify a key failure mode in prior ranking-based refinements (e.g., RankRefine): when the number of reference items increases, the Bradley-Terry likelihood can dominate due to curvature mismatch, causing bias. BAYES-ECR corrects this through temperature calibration and accuracy-aware soft gating, controlling the relative contribution of ranking information.\nEmpirical results across 12 datasets (including molecular ADMET and tabular tasks) show that BAYES-ECR significantly improves predictive accuracy and robustness", "questions": "- How sensitive is BAYES-ECR to the regressor variance σ²_re? Does under- or overestimated uncertainty systematically bias the refinement?\n\n- The analysis assumes symmetric flip noise. What happens if the ranker has systematic bias (e.g., consistently underestimates large values)? Could the Bayesian posterior incorporate a bias term instead of gating?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The paper is with fair conceptual clarity and novelty. It cleanly reformulates post-training regression enhancement as a Bayesian inference problem rather than heuristic fusion, providing a unified probabilistic interpretation that subsumes RankRefine as a special case.\n\n- The theoretical grounding is solid. The analysis of rank-likelihood dominance and curvature scaling (Lemmas 3.4–3.7) is rigorous and insightful, pinpointing why Bradley–Terry models can fail at large k and how temperature calibration mitigates this.\n\n- The overall evaluation is comprehensive. The paper evaluates multiple variants (MAP, MLE, MLE-Temp, MLE-GatedTemp) across molecular and non-molecular datasets, using both oracle and LLM rankers. Ablations, runtime, and robustness studies are thorough.", "weaknesses": "I'm not the expert of reco.sys, but I tried my best to understand the paper. Here are some general concerns based on my understanding:\n\n1. Despite the clean Bayesian framing, much of the improvement stems from calibration heuristics rather than a fundamentally new probabilistic model. It seems that the method extends, rather than transforms, RankRefine’s core idea.\n\n2. The proofs dominate the narrative. Figures are dense and sometimes too mathematical (Fig. 1–3). The paper could benefit from a small worked example demonstrating Bayesian updates visually. While technically sound, the exposition can feel overloaded: long derivations, dense notation reuse, and sometimes unclear transitions (e.g., from Lemma 3.7 to Corollary 3.8). Some results could move to Appendix without loss. Clearer clarification on the background will be appreciated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "spJOXv6Jdt", "reviewer": "Reviewer_zBQs", "summary": "In this work, authors propose BAYES-ECR, a novel plug-and-play post-training method designed to improve the predictions of a base regression model. The key idea is to leverage pairwise rankings between a query item and a set of reference items (with known labels), often easier and cheaper to obtain than high-quality numeric labels. BAYES-ECR is based on a Bayesian update using Gaussian likelihood (from base regressor) and another likelihood from external pairwise ranker. Authors find that the resulting posterior is log-concave thereby ensuring a unique maximum likelihood solution which can be recovered using efficient optimization procedures like Newton updates.", "questions": "Some additions questions to the authors:\n- How sensitive is BAYES-ECR's performance to inaccuracies or miscalibration of the base regressor's uncertainty estimate?\n- How much is the computational burden of the Newton update steps in real-world applications? Can the authors provide some clear numbers to understand the overhead for using the proposed method?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I find the following strengths in this paper:\n- I find combining regressor predictions with pairwise ranking information is a novel way as the form of Bayesian update, a cool idea and which I believe is this work’s significant conceptual strength, moving beyond other heuristic approaches.\n- The core problem, i.e. scarcity of high-quality numeric labels versus the availability of easier-to-obtain pairwise rankings, is highly relevant in many scientific and engineering domains (not just LLMs), for e.g. such as materials science and drug discovery, therefore this method has a wide applicability.\n- I find the proposed temperature calibration and accuracy-aware soft gating mechanisms to be well-motivated by the theoretical analysis.\n- Based on the empirical findings, the proposed method runs very efficiently and from what I find, inference is unlikely to be a bottleneck for this method.\n- The study by including off-the-shelf LLMs as imperfect rankers highlights the method's applicability to real-world scenarios.", "weaknesses": "Few weaknesses that I would like the authors to address:\n- The core Bayesian update relies on the assumption of conditional independence between the regressor and ranker - while I think this assumption may make some sense in general, but I also think thai may not always perfectly hold in practice, especially if the ranker's \"expert\" knowledge is implicitly derived from data similar to the regressor's training data. Is this something the authors have considered or have any view on? How will the method change if this is the case?\n- The paper mentions that regression improvements may depend on the composition of the reference set and suggests future work on sophisticated sampling strategies. A brief discussion of potential biases / optimal strategies for reference set selection, even if speculative, could enrich the overall study in this paper.\n- One of the commonly discussed limitations of the Bradley-Terry model is that it inherently assumes transitive relationship among the pairwise comparisons, i.e. if A > B and B > C, then A > C. While this is usually fine in many cases, some real-world \"expert\" rankings might be locally inconsistent or reflect non-transitive preferences in certain contexts, which could challenge the model. Can the authors comment on such a setup? Or what impact it can have on such applications? Finally, any thoughts on how the current framework can be extended to handle it?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2QzMQYnsAJ", "reviewer": "Reviewer_JneQ", "summary": "This paper addresses the challenge of improving regression models in data-scarce domains, where high-quality numeric labels are expensive, but pairwise rankings are relatively cheap to obtain. The authors propose Bayesian Enhancement with Calibrated Ranking (BAYES-ECR), a novel and efficient \"plug-and-play\" method to enhance a pre-trained regressor's predictions post-training, without requiring any retraining. The core idea is to perform a per-query Bayesian update. For a given query, BAYES-ECR combines two sources of information: (1) A Gaussian likelihood from the pre-trained regressor, centered on its prediction $\\hat{y}_{0}^{re}$ with variance $\\sigma_{re}^{2}$. (2) A ranker likelihood derived from pairwise comparisons between the query and a small set of $k$ reference items with known labels. This likelihood is modeled using the Bradley-Terry model.", "questions": "1. Could the authors please clarify precisely how the regressor variance $\\sigma_{re}^2$ was estimated for the Random Forest and MLP experiments? Given its critical role as $1/I_{reg}$ in the information-weighted update step, this detail is essential for reproducibility and for understanding the method's behavior. Did you, for example, estimate $\\sigma_{re}^2$ as the Mean Squared Error of the base regressor on a hold-out set?\n\n2. According to Section 3.3, labeled pairs from the reference set $\\mathbb{D}$ are fitted with $s(\\hat{\\omega}(y_a - y_b))$ to estimate $\\hat{\\tau}_{cal}$. Is the true rankings (i.e., $r_i = 1$ if $y_a > y_b$) obtained from the ground-truth labels in $\\mathbb{D}$ used in this fitting process? Or does it make use of the external ranker's noisy rankings $r_i = R(x_a, x_b)$?\n\n3.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The main contribution of the work is its analysis of where the Bradley–Terry model breaks down. Although using BT models for ranking is fairly common, the authors offer a fresh insight: the “soft-count” sigmoid mechanism interacts with the number of reference points, $k$, in a way that creates a kind of curvature dominance—an effect that amplifies bias. This observation is both new and important. This insight moves the field from heuristic fusion (like inverse-variance weighting) to a more principled understanding.", "weaknesses": "1. The framework builds on combining a regressor likelihood, $\\mathcal{N}(\\hat{y}{0}^{re}; y_0, \\sigma{re}^2)$, with a ranker likelihood. The Newton step works as an information-weighted average, where the regressor’s contribution is scaled by $I_{reg} = 1/\\sigma_{re}^2$. This makes $\\sigma_{re}^2$ a key parameter. The paper notes that many regressors don’t naturally provide this value and suggests using ensembling or dropout to approximate it—but it never clearly explains how $\\sigma_{re}^2$ was actually estimated for the Random Forest and MLP models in the experiments. Was it treated as a single hyperparameter tuned on a validation set?\n\n2. The accuracy of the ranker, $a$, is necessary to operate the accuracy-aware soft gate, $\\tau(a)$. According to the authors, a \"small validation set\" is used to gauge this accuracy. However, rather than focusing on how sensitive the results are to the accuracy estimate itself, the robustness checks in the appendix examine how well $\\hat{\\tau}_{cal}$ is calibrated (see Figures 5 and 6). The estimate of $a$ may not be accurate if that validation set is very small, which could render $\\tau$ less than ideal. The robustness claims would be much stronger if a brief sensitivity analysis of how $\\tau$ varies with various estimates of $a$ were included.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iF21FkLqzd", "reviewer": "Reviewer_aXK6", "summary": "This paper presents a plug-and-play Bayesian enhancement to regression models. Specifically, pairwise ranking information complements scarce absolute labels as in RankRefine, but with Bayesian aggregation.", "questions": "Any additional analysis to answer questions in the weakness section would help rebuttal.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Performance gain: Similar ideas as RankRefine but shows performance\n* Efficiency: Plug-and-play avoids costly model retraining. \n* Data Efficiency: Complementary pairwise signal helps data-scarce fields such as science.", "weaknesses": "Some questions left unanswered.\n* Motivation behind pairwise: Would listwise ranking be a stronger signals?\n* Generalization: While we understand science is a data-scarece domain, evaluating only on this scenario makes us wonder whether it would generalize to general regression problems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)