Eliciting Numerical Predictive Distributions of LLMs Without Auto-Regression
Authors: Julianna Piskorz
Kasia Kobalczyk
Mihaela van der Schaar
Venue: ICLR 2026 Poster (2026)
Keywords: mechanistic interpretability
uncertainty estimation
LLMs
time series
probing
Primary area: foundation or frontier models, including LLMs
TL;DR: We demonstrate that LLMs' hidden states contain information about their own numerical predictive distribution, that can be elicited without the need of auto-regressive decoding.

Abstract: Large Language Models (LLMs) have recently been successfully applied to regression tasks---such as time series forecasting and tabular prediction---by leveraging their in-context learning abilities. However, their autoregressive decoding process may be ill-suited to continuous-valued outputs, where obtaining predictive distributions over numerical targets requires repeated sampling, leading to high computational cost and inference time. In this work, we investigate whether distributional properties of LLM predictions can be recovered _without_ explicit autoregressive generation. To this end, we study a set of regression probes trained to predict statistical functionals (e.g., mean, median, quantiles) of the LLM’s numerical output distribution directly from its internal representations. Our results suggest that LLM embeddings carry informative signals about summary statistics of their predictive distributions, including the numerical uncertainty. This investigation opens up new questions about how LLMs internally encode uncertainty in numerical tasks, and about the feasibility of lightweight alternatives to sampling-based approaches for uncertainty-aware numerical predictions.

Published as a conference paper at ICLR 2026
ELICITING NUMERICAL PREDICTIVE DISTRIBUTIONS OF LLMS WITHOUT AUTOREGRESSION
Julianna Piskorz', Katarzyna Kobalczyk; Mihaela der Schaar Department of Applied Mathematics and Theoretical Physics
University of Cambridge
(jp2048, knk25) @cam. ac.uk
ABSTRACT
Large Language Models (LLMs) have recently been successfully applied to regres- sion tasks fasks-such time series forecasting and tabular prediction by leveraging their in-context learning abilities. However, their autoregressive decoding process may be ill-suited to continuous-valuus outputs, where obtaining predictive distribu- tions over numerical targets requires repeated sampling, leading to high computa- tional cost and inference time. In this work, we investigate whether distributional properties of LLM predictions can be recovered without explicit autoregressive generation. To this end, we study set of regression probes trained to predict sta- tistical functionals (e.g., mean, median, quantiles) of the LLM's numerical output distribution directly from its internal representations. Our results suggest that LLM embeddings carry informative signals about summary statistics of their predictive distributions, including the numerical uncertainty. This investigation opens up new questions about how LLMs internally encode uncertainty in numerical tasks, and about the feasibility of lightweight alternatives to sampling based approaches for uncertainty aware numerical predictions. predicttonn.. Code to reproduce experiments can be found at httptttss///m /ecttss:/githiie thub. com kasi i koba cz guess_ 1 11m. git
INTRODUCTION
With the increasing capabilities of LLMs, a growing body of work has explored their use for structured data prediction most notably for tabular data regression (e.g. Requeima al., 2024 Hegselmann et al. jet al. 2024] Xue & Salim| 2023). These studies demonstrate that L.MMs can act as competitive regressors, even without task-specific fine-tuning. This advantage is especially pronounced in low- data regimes, where LLMs can leverage their pre-training, prior knowledge, and capacity to condition on auxiliary textual context, matching outperforming specialised models. We provide more detailed overview related works in Appendix
2023 Shyshhey et al.) 2025 Vacareanu et al. 2024) and time series forecasting (c.g. Gruver
However, issuing numerical predictions with LLMs requires sequential autoregressive generation: real-valued numbers typically span multiple tokens requiring multiple forward passes of the input through the LLM to generate single prediction. This makes inference costly and time-consuming, especially when many samples are needed-for example, to quantify the uncertainty (Gruver et al. 2024
Requeima al. 2024) improve prediction accuracy (Requeima al.) 2024).
This raises a natural question: there a way of eliciting the LLM's predictive distribution, and its underlying uncertainty, without performing costly autoregressive number generation and repeated sampling? This question is non-trivial: generating a real number requires determining its order of magnitude, including decisions about decimal placement and number typically made only after several tokens have already been generated. By exploring whether we can gain insight into such decisions without performing autoregressive token generation, our work contributes to exploring the broader question of how LLMs 'plan' their outputs before generation begins (Lindsey et al. 2025).
ceciiaationchoicee that are
Authorrs contrihuted equally

---

Published as a conference paper at ICLR 2026
Empirical Distribution
Magnitude cleesiffer
predicted probabilties of magnitude orders
Time series
Samples y=l
Forden
Pintaox
x
[T1 22
(x')
hel-
Hiddon States
Probing Model
Value regressor
conditiona
(a) The goal: can recover the LLM's numerical pre- dictive distribution from internal representation of the input?
(b) Schematic diagram of the magnitude-factoriseed probing model used section
Figure 1: Illustration of this paper's goals and methodology.
In this work, focusing the problem of time series forecasting specifically, we explore to what extent the LLM's internal representation of the input sequence (requiring just a single pass through the LLM) can be used to reconstruct its predictive distribution of the next real number. Concretely, we explore the following three questions:
Do LLMs encode the next number they intend to generate? (Section 2 We begin by examining whether LLM's ss internal representations of the input series encode sufficient information to recover point predictions prodictions-spccifiaalll the greedy output, mean, and median the predictive distribution. To test this, we develop magnituude-fcctorisee regression probe that separates prediction into two components: a coarse magnitude classification and a scale-invariant value regression, such that our model can effectively learn predict numbers of varying orders of magnitude. Trained on LLM embeddings from synthetic time series data, our probe accurately predicts numerical targets across data with varying orders of magnitude.
We then ask whether uncertainty information is also captured in LLM's hidden states. Using a similar magnitude- factorised quantile regression model, we train probes to predict the quantiles of the LLM's output distribution, approximated via sampling. The resulting models accurately recover the interquartile range and produce well-calibrated confidence intervals.
Can we elicit the uncertainty of the LLM's predictive distribution? (Section 3
What is the practical value of our findings? (Sections 4 Jand The ability to recover numerical predictions directly from LLM embeddings holds the potential to bypass autoregressive sampling- offering savings in inference time and computational costs, which we demonstrate empirically. However, for such probes to be practically useful, they must generalise beyond the specific conditions under which they were trained. We therefore evaluate whether single probing model can be deployed across varied settings without retraining. First, we test generalisation to unseen time series lengths. Second, we assess generalisability of our previous results real-world data. We investigate whether probes trained on real-world data generalise across different sub-domaing and whether probes trained on synthetic data generalise to real-world data. We demonstrate that, while some drop in calibration datasets, our probes demonstrate encouraging generalisation abilities,
occurs on showing the universality of L.LM's 's representations of numerical quantities.
Our findings provide new insights into the numerical capabilities of LLMs: much of the "reasoning" underlying numerical predictions appears to be encoded in the model's internal representations the input, prior to token-level decoding. This contests whether autoregressive sampling is necessary to extract realev outputs from LLMs, and opens the door to developing more efficient, single- pasee approaches. By showing that both point estimates and uncertainty can be reliably extracted from hidden states, our work suggests a lightweight, general-purpose strategy for deploying L.MMs in regression tasks particullll in settings where computational efficiency and uncertainty estimation essential. We hope these results motivate further study of how LLMs internally represent numerical quantities, and how this information can be surfaced for practical downstream use.
2 Do LLMS ENCODE THE NEXT NUMBER THEY INTEND TO GENERATE?
LLMs are trained for next-token--reediiine predictiond. Thus, as a single number typically spans multiple tokens, obtaining a complete numerical prediction from the LLM requires repeated auto-regressive sampling. This can be computationally expensive in number-heave tasks, particularly when one would like to obtain repeated samples for the purpose of uncertainty estimation. To mitigate this overhead, ask: to what extent is the full number the LLM intends to predict-beyone just its leading
2

---

Published as a conference paper at ICLR 2026
encoded in the LLM's internal representation, prior to any token-by-token generation? such information can be reliably extracted, one could sidestep autoregressive generation altogether. However, this possibility is not trivial: critical aspects of number generation, such as the placement of the decimal point or number termination, which determine the order of magnitude of the number, often occur late in the decoding process, particularly for large magnitudes.
2.1 METHOD
be a sequence of numbers (e.g., an equally-spaced time series). Given x, language model induces predictive distribution PLLM(- x) over the next value +1+ In this section, we investigate whether the internal representations of the LLM encode sufficient information to predict this distribution's 's key statisties. Specifically, we aim to train independent probing models to recover: (a) the LLM's greedy prediction, (b) the mean, and (c) the median of
Objective. Let x [x
PLLM. The empirically estimated scalar statistics of the LLM's predictive distribution are our targets for prediction based on the LLM's ' hidden representation of the input series x.
LLM Representation. Following Gruver et al. (2024), we serialise an input series x to text as "T1, T2, ................................ Exp=" We do not apply any scaling to the time series before serialising the inputs. This is important, as LLMs often contain contextual prior knowledge and scaling of the original time series may prohibit the LLM from using this prior knowledge effectively. From pre-selected set of N transformer layers denoted by H. we extract the final token's hidden state from each layer, obtaining a set of embeddings { [h1-1] E Rdaaai : E H]. We concatenate these embedding vectors to form single input for the probe:
== concat (be[-1])eH E R.du
(1)
where duppu dmmoeet X [7/). The choice of the hidden layers H. is a hyperparameter of our model. Throughout the main body of this paper we use the Llama-2-78 model, for which dmodel 4096. and we set H as the last 8 layers. This choice of model is motivated by the fact that the tokeniser of 1lama-2-7B encodes each digit as a separate token, thus allowing to ensure that numbers with larger orders of magnitude consist of more tokens (making it more difficult to obtain LLM's predictions without autoregressive decoding). Results with other LLMs can be found in Appendix C alongside an ablation study on the choice of layers.
Datasets. We use synthetically generated datasets to evaluate probing performance. Each sequence x is sampled from a set of functions exhibiting varied dynamics, including sinusoidal patterns, Gaussians, beat functions, and random noise (see Appendix B33 for details). The generated time series also vary in the length, n, and the level of noise, to ensure diversity of the input embed- dings and target distributions. We generate variants of the dataset by scaling the value range from [-1, to [-10, 10), [-1000, 1000), and [0100, 10000). We also combine the datasets of differ- ent scales to obtain larger dataset of approximately 66k unique sequences, balanced across the different orders of magnitude. In this section, our training datasets take the following structure: N
Ji=i' where N is the total number of examples in a dataset, greedy the LLM's greedy prediction given x,, and y y meddian the empirical mean and median, respectively, estimated based on Noa samples from the LLM'' predictive distribution, {uf Nag ~ PLIM(-(x;) In our experiments we set No. 100.
{ {(x;, e. Freedy
Probing Model. To the best of our knowledge, existing approaches to neural network probing limited to binary or categorical probes (see Appendix (A). A significant challenge in training regression probes for LLM numerical predictions is the wide spread of target magnitudes. Standard regression losses such as the MSE, transformation techniques like log-scaling, fail to provide stable gradients, prioritising optimisation of the largest values only. To address this, we introduce a magnitude-acctorieed factorised regression model that consists of two components (both initialised as MLPs. see Appendix B.4 for hyperparameter details):
fooder Rede - IR M, classifier predicting the order of magnitude of the target number.
. fval : 11 + 12 M: a regressor predicting the target value scaled by the predicted order.
The magnitude classification component predicts the order of magnitude of target value y", where E (greedy, mean, median). Formally, we define the order of magnitude of a scalar y as: m(y)
3

---

Published as a conference paper at ICLR 2026
[ogio((ol)] For given input and its corresponding representation e, (e) outputs vector of logits of a set of pre-defined magnitude classes my E {Mmin..... Mmax) In our experiments, we let mmin and max be the minimum and maximum orders of magnitude in the training data. The predicted vector of magnitude class probabilities (after softmax) is denoted as (fatmax(Saalee =
p(x) Pmmmin P )
The regression component predicts scaled version of the target value, conditioned on all possible magnitude classes (to allow the model to adjust its prediction based on the predicted magnitude). For each magnitude class my {mmmin.... Mmmax }. the corresponding scale is equal regression head takes as imput the concatenation of the feature representation and the scale factor sk outputting Tæ = ( skl) for each k. This produces regression outputs = [Tmmmn ..... m'maa for all magnitude classes. The conditional prediction for each magnitude order mg then computed as: V/- Fk 10me.
10 The
Loss Function and Training. Our model supports a two-phase training procedure:
Phase 1: Train only the classification head while freezing the regression head, using the classifi- cation loss standard cross-entropy loss for magnitude prediction:
1
Lurder
CrossFatropy..oss(((xs))). m(v.)),
No
where Ng is the batch size.
Phase 2: Train only the regression head while freezing the classification head, using the regression
loss, i.e. the mean squared error between the predicted scaled value and the scaled target value:
No
C
Time
where
n. = arg max Pm.. (xx).
N
10m(
Empirically, we find that the 2-stage training procedure performs better than joint training of the order and value heads. To further provide justification for our approach, in Appendix C.1 we compare the performance of our magnitude-factorisse probe against a vanilla MLP probe, showing significant performance gains.
Expected Prediction. During evaluation of our model, we compute the expected prediction by marginalising over the top-K magnitude classes: Ex [D] = > EmmppKK - Pm; Wss where top-K refers to the K magnitude classes with highest predicted probabilities (we set K 3).
2.2 RESULTS
Order of magnitude. We first investigate to what extent our probing model can correctly recover the order of magnitude of the number the LLM intends to generate. We train three separate models, for each of the mean, median and greedy targets. In this experiment, we the combined dataset consisting of time series with varying scales. The bar chart on the right hand side of Figure visualises that the classifier part of our magnituee-factorised model achieves above 90% accuracy in predicting the exponent of the target values. Further, as visualised with the seatter plots on Figure 2 we find strong correlation between our probes' final predictions and the target statistics.
Precision in generated digits. To further assess whether the LLM's internal representations encode fine-grained information beyond the order of magni- tude, we focus on the dataset with time series values in the interval [-1, 1). We report the mean squared error (MSE) of the predictions obtained with our prob-
Table 1: MSE obtained when predicting the statistics of the LLM's predictive distribution.
Ve target
0 (ours)
X;
Fi.n
mean
0.006
0.256
0.035
0.085
median
0.006
0.260
0.041
0.087
ing model in Table [ For context, we compare the performance of our probe against three simple base-
greedy 0.015
0.273
0.065
0.109
lines, using as the prediction: a) the average value of the entire training dataset (X), b) the average of
the series x; (x,), or c) the last value of the series x; (2),.). We also provide scatter plots for this dataset in Appendix C, The results demonstrate that using the last token embeddings of the LLM the input, our probe accurately recover the L.LM's predictions,
accuracy level which
4

---

Published as a conference paper at ICLR 2026
Target moon
Tuget median
Target: grewdy
Accuney by reposent scrows farget
Poddee S (Iv va
Peeiidien Aczuncy
90
9.8
Parnen 16 0.00
Parrsn JD 6.33
Prance 6.00
True (Ritg scan)
Volun (HE,, ROle)
Figure 2: Predicted vs. true values of mean, median and greedy prediction, presented on log10 scale, The probing model accurately recovers the number that the LLM intends to predict, indicating that the internal representations encode the order of magnitude of prediction.
significantly surpasses simple baselines naively constructed from the input. Interestingly, among the three targets considered (mean, median, greedy), the model performs worst when predicting the greedy output. We hypothesise that this is because the greedy prediction is not an explicit function of the model's s predictive distribution, but rather by-product of the autoregressive decoding process, making it harder to recover precisely from internal states.
② These results show that the internal representations of pre-trained LLM encode detailed information about its intended numerical output- even before any tokens are generated. Our probing model accurately recovers not only the order of magnitude, but also fine-grained point estimates of the mean, median, and the greedy output. This demonstrates that much of the numerical reasoning performed by the LLM is already present in its hidden states, and may not require the autoregressive decoding process.
CAN WE ELICIT THE UNCERTAINTY OF THE LLM's PREDICTIVE DISTRIBUTION?
In the previous section, we demonstrated that point estimates- as the greedy prediction, mean, and of an LLM's predictive distribution PLLM(- x) can be recovered from its internal representations without the need for performing autoregressive sampling. Encouraged by these findings, we now investigate whether we can go beyond point estimates recover the uncertainty of plim by approximating its distributional shape. Specifically, we attempt recover multiple quantiles of PLIM, enabling a coarse-grained reconstruction of distribution function and providing an easy way of estimating the confidence intervals for the LLM's predictions.
3.1 METHOD
Quantile Regression. Since the distribution PLLM may be multi-modal and non-Gaussian, we rule parametric approximations. Instead, we adopt quantile regression, which enables direct estimation of distributional shape without strong assumptions about its form. Let Q = [r'..... 7 be list of target quantile levels. For each r" E [o, 1], we denote the predicted quantile value as g" We train the quantile predictor using the pinball loss (Koenker & Hallock, 200T), computed with respect to
LLM samples vf PL x:). For single quantile level r, predicted quantile value ÿ and single
LLM sample v/. this loss function defined
PinballLoss(7, v max (T(r) (1) -v,))
Probing Model. As in section we use magnitude-factorise model to address the challenge of scale variance numerical outputs. The quantile model takes an equivalent form to one in section 2 except
introduce S classification and regression heads, one for each quantile level, denoted by förter and f respectively. As previously, each classification head outputs a vector of magnitude class probabilities pd [Pminn---- ... P-nax] .,...ax conditional scaled values r value is computed as q² = r 10T0.
The regression heads output vectors of i r. For an order my the predicted conditional quantile
Datasets. We use the same datasets in section but with target values being the raw LLM samples (v. N. Jj=1, instead of the aggregate statisties.
5

---

Published as a conference paper at ICLR 2026
Training. Unlike in the previous section, joint training approach (2-phase training did not yield any sigmificant improvements in performance). We construct the total loss as the sum of the cross-entropy losses for magnitude prediction and pinball losses for quantile regression:
s (oondrr + B Leal)
(5)
Na
1
C under
LoossEntropyyoss (p' m(()))
(6)
Nb
No No.
C
1
Σ>
PinballLoss
(1.1 (0m
No.
In the above, of denotes the empirical quantile value derived from the LLM samples {u% Jy=1 In our experiments, we use
S quantile levels: [0.025, 0.05, 0.25, 0.5, 0.75, 0.95, 0.975]. This choice of allows us to casily estimate: the median, the interquartile range (IQR), as well as the 90% and 95% confidence intervals.
3.2 RESULTS
IQR Prediction. To investigate whether the LLM's internal representations encode information about the spread of its predictive distribution, we estimate the interquartile range (IQR) using the predicted 25th and 75th percentiles. As the IQR is sensitive to scale, we normalise it by the predicted median, and similarly normalise the empirical IQR from LLM samples using the sample median. If the probe captures uncertainty faithfully, we should observe monotonic relationship between the predicted and empirical (normalised) IQRs. Scatter plots Figure show a strong correlation between predicted and sample-based IQRs, with points aligning well that our probing model is able to infer distributional spread from the LLM's hidden states.
ar. This demonstrates
Dotaest scale: 1.0
Donaset scale: 10.0
Dotasct scal: 1000.0
Dotazet scale: 10000.0
Prodicea
Proroe Specmoon 91
Fooroo DAS COCCEES
P Fooeoo
Fioeoo Susurii A 058
5 SIOOEEN
Sample IQR
Figure 3: Predicted cample-based IQR (both median-normalissn The model accurately tracks the variability the LLM's output distribution.
Confidence Interval Coverage. Next, we evaluate whether the predicted quantiles yield calibrated con- fidence intervals. Given a desired confidence level and its associated interval C(ax) predicted by the probe, we compute the empirical coverage by check- ing what fraction of LLM samples fall within the predicted interval. We expect that:
Table Coverage of the predicted confidence intervals. Values denote empirical coverage (%) + standard error.
50%
90%
95%
1.0
52.0 0.4
90.9 0.3
95.5 + 0,2
10.0
0.2 90.7 0..
0.5
96.1 9..
1000.0
95.7
= Fyyppt-1o) (x) [y E C(a)))
10000.0
48.
90.5
95.4 4
Nar
H C(a)), where yd (-).((((x).
Table 2 reports the empirical coverage for 50%, 90%, and 95% intervals across datasets of varying scale. In all cases, empirical coverage closely matches the target level, indicating that the quantile probe is well-calibrated.

---

Published as a conference paper at ICLR 2026
? Our findings provide strong evidence that the uncertainty of the LLM's predictive distri- bution is encoded in its internal activations and can be effectively elicited using quantile regression probe. The probe is capable of predicting meaningful spread measures, producing well-calibrated confidence intervals that match the empirical coverage. These results suggest that L.LMs internalise rich distributional information during generation, which can be accessed and approximated efficiently via probing. This opens up opportunities for downstream applications that rely on uncertainty quantification- as safe decision -ociiionnmmakngg... model-based control, and probabilistic reasoning-whin avoiding the overhead of autoregressive sampling.
4 EFFICIENCY AND ACCURACY
In this section, we further investigate the practical applicability of probing models from the perspective of computational and inference time efficiency as well as accuracy with respect to the ground truth time series value. Throughout this section, we focus on the scalar probing models from section 2
4.1 ERRORS WITH RESPECT TO THE GROUND-TRUTH
Table 3: MSE of predicted values vs. the
In this experiment, we compare the error of probes' predictions with respect to the ground-truth next value Fi,++1 of the input time series x; (..,1,-...::4 Fii.. We present the comparison between the error achieved by using the probe trained to predict the mean, me- dian and greedy statistics of the LLM distribution (Gioohe [(0 2i,n+1) )2) vs. by using these statis- tics directly (ELLM = E [(y} 1) For a better sense of scale, we additionally present the errors using the simple baselines introduced in section 2.2 as well as the error of the mean prediction of GP model fit to the input time series x;. The MSE values of the predictions with respect to the ground-truth next value the time series are presented in Table 3 Results were obtained for the probing models from section 2.2 and the dataset with scale 1.0. Our probe attains errors comparable to those obtained by sampling directly from the LLM. This places its performance in context: the probe captures enough information from hidden states to match the LLM's own accuracy on the one-step-ahead prediction task.
ground truth Ti,++1
predicted value
probe (j:)
LLM (y))
greedy
0.0652
0.0668
mean
0.0562
0.0555
median
0.0561
0.0553
x
x,
Fi.
GP
0.3454 0.0878
0.1226
0.0717
4.2
SAMPIEE EFFICIENCY AND COMPUTATIONAL. COSTS
Sample Efficiency of Mean Estimation
We further compare the error of the probe in predicting Zin+1 Vs. using the mean prediction of the LLM using N empirical samples. Figure 4 illustrates this on a dataset with scale 1.0. The horizontal lime shows the error attained mean
0.25
MSE w.r.t. Test Volue
LLM Sampling (mean)
0.20
Probe
0.15
by our probe, "probe points with error bars show the error attained when using
[(t.mean Iinn+1) ) and the blue
0.10
0.05
N LLM samples, mean (N) E
(LLM(N)
(w/
mean, 1i,,41)
0.00
where y/ mean,, is the mean N L.LM samples: (vf J (-)x). Our probe outperforms empirical sampling for all N up to 20-25 samples, demonstrating that probe of this kind can serve as computationally efficient sur- rogate for making numerical predictions.
0
5
10 15
20
25
Number of LLM Samples
Figure 4: The probe achieves compara- ble error to using 20-25 LLM samples on the one step ahead prediction task.
For a detailed discussion regarding the computational costs comparisons of LLM sampling and inference with our probe, see Appendix C.3
? Key Insights. The probes trained to predict the LLM's output from its hidden state achieve high enough precision to be used as competitive predictors on the one-step-ahead time series prediction task. This demonstrates that they be used lightweight way of obtaining LLM predictions, delivering comparable performance at substantially lower inference cost.
7

---

Published as a conference paper at ICLR 2026
0..00
Moddl teps
0.80
9.50
roreniceed
00%
50%
0.50
Contert Length (Urssun for the Restricted Model)
Figure 5: Generalisation to unseen context lengths. A probe trained on a restricted context length
range (Restricted) exhibits greater deviation in empirical coverage outside its training range.
GENERALISATION
Finally, we investigate the generalisation capabilities of our approach along several axes including context length generalisation, applicability to real-world data, and cross-dataset generalisation. As the process of training probe can be costly, such generalisation capabilities are important for real-world applications, we would like to use a pre-trained probe on new datasets with different distributional properties. Throughout this section, we focus the quantile probing models from section 3
5.1 GENERALISATION UNSEEN CONTEXT LENGTHS
First, we ask whether a probe trained on a fixed range of input sequence lengths generalises to longer or shorter contexts. We train and compare against each other two models:
Base: Trained input sequences with lengths in range [3, 40).
Restricted: Trained only input sequences with lengths in range 110,2
At test time, evaluate both models on contexts shorter than 10 and longer than 20. We assess generalisation by measuring the empirical coverage predicted confidence intervals, as defined in section
3.2
Figure 5 shows results the dataset with the scale factor 1.0.
We observe that while both models achieve reasonable calibration, the Restricted model exhibits slightly greater deviations from the nominal coverage, particularly for context lengths further from the training distribution. These results suggest that the probe generalises to novel context lengths, but training wider context ranges should be beneficial for more robust generalisation.
5.2 APPLICABILITY
WORLD DATA
Table 4: Coverage of the CI intervals on previ- ously unseen testing inputs.
Thus far, our analysis has focused on synthetic data. In this section, we evaluate whether our probing model can be trained successfully on real-world datasets, and how well predictions can generalise across different types of input series.
a 50%
90%
95%
Model
Real (all)
48.8 + 0.1
88.5 0.1
94.3 + 0.1
Real (5 fold)
43.4 0.2
82.1 92.1
89.4 0.2 77.3 0.4
Synth
0.3
67.7 0.4
To assess this, we construct a dataset using time series from the Darts (Herzen et al. 2022) and Monash (Godahewa et al.
2021) collections. Following
the same format as in our synthetic experiments, we generate LLM embeddings and samples for approximately 45,000 distinct sequences across 31 sub-datasets (c.g.. US Births, Air Passengers). Furthermore, we also investigate an even stronger form generalisation: from a model trained on synthetic data only to testing real-world data. For this purpose, we train the following models:
Real (all): Trained on a random 80% of all sequences across all sub-datasets. The remaining 20% held out for testing.
Real (5 fold): We partition the dataset into folds such that, in each fold, one model is trained on 80% of the sub-datasets and evaluated on the remaining 20%. This ensures that each sub-dataset appears in the test fold exactly one out of 5 models trained.
Synth: A model trained on the combination of the 4 synthetic datasets with scales 1.0, 10.0, 1000.0 and 10000.0.
At test time, the above models face increasingly stronger distribution shifts. In terms of generalisation performance to previously unseen data distributions we can view the Real (all) model as an upper- bound baseline for Real (5 fold) and Synth.
8

---

Published as a conference paper at ICLR 2026
Nog Dree or the Medaa
10"
101
Traiing Type
and (x1)
RAM)
tade lo minutas (8 (lssitt)))
tralic, Noarly (8.1k-10)
choel deates (B.Te (33)
sole Kuscomos (7 Spvdd))
clostd souther (8 (00)
tralic wertly (8.S (eu))
cangerdday (2.40 (2.4e-i1)) 7e-111))
sed cop (3 (e4eill))
sind Aaprnds (1SSev (8)
Sumponttien (4 lovet)) te-s
Cu (4 (6-5)
rimppe (eSSeill))
Heae Hondkastatnn (8.2x 2012))
fed and (8:2x-(0)
pedesitinn (2 (246
pentlyyy (1 (33)
weteererrrraaaeeee (2 183
inseiaan renotll (2 (2-508 (0-1
(8. (2-3
lagali wetleaeettt tratii seeely (1 for
ot.3015 (s. (e1)
dlerve drrunndd (91)
toution quarttyy (s la-
WindyDa (S. (t-Sexttt))
uakor oveely (5)
on dertt (8 S0x
tourise, yoars (1.Se (54)
Won (2.00-194)
westher [1
and dtaly (1.
(5
Aur AoPaaeepertt Autlerratae
Montiyt Diw.iiiddiiiaaaaae
Sub Dutact (Aog. Mad of LLM predittion)
Figure 6: Absolute error the median across different sub-dataset. Comparison of generalisation across models trained on different data.
In Table 4 we report the average coverage of the CI across all training types. We observe that the Real (all) model demonstrates good performance, with the empirical coverage of LLM samples closely matching the expected coverage. The Real (5 fold) model demonstrates a slight downgrade in performance. Interestingly, while the Synth model underperforms, it still demonstrates good generalisation for some of the sub-datasets as we can see on Figure 6 This figure shows the distribution of the absolute error of the predicted median all sub-datasets. The x-axis is sorted by increasing order of magnitude of the datasets defined by the average of the median of LLM samples. We note that the sub-datasets in our collection cover different ranges of values (with individual LLM samples varying in magnitude from 10-3 to 10²²). We suspect that this primarily why the probing model struggles to generalise across some datasets.
the median of LLM samples across
? Key Insights. The probing model exhibit some, albeit limited, generalisation across context lengths. When applied to real-world datasets, the model achieves accurate empirical coverage and demonstrates partial transferability to unseen data distributions. Cross-dataset generalisation possible, but challenged by large variation in scale and distribution.
6 DISCUSSION, LIMITATIONS AND FURTHER WORK
While most probing studies of LLMSs focus on predicting categorical outputs in natural language tasks, we instead target numerical prediction, which is particularly challenging due to high variance in output magnitudes. To address this, we introduce novel probes that decompose the task into magnitude classification and scaled value regression. Our findings show that LLMs encode rich numerical information about their predictions before autoregressive decoding. Training lightweight probes on hidden states allows us to recover both point estimates (mean, median, greedy outputs) and uncertainty. This suggests that much of the LLM's 's "reasoning" over numerical outputs occurs during input processing, with autoregressive decoding primarily simply surfacing the predictions. Beyond offering insight into how LLMs handle regression, our results also open a practical path: enabling uncertainty-awe aware numerical prediction without the computational cost of repeated sampling.
Despite these promising results, several limitations remain. First, our approach requires access to internal model activations, even though it does not involve fine-tuning the LLM itself. Second, while our probing models exhibit some generalisation ability, they are still model-specific, requiring retraining for each architecture or tokenisation scheme. Third, training and evaluation requires approximating the LLM's predictive distribution via empirical sampling, which is an imperfect and computationally expensive proxy.
Future work could extend this framework to a broader range of structured data and prediction tasks, including univariate or multivariate regression and time-series forecasting, as well as multi-step ahead prediction tasks. While in this work we focused on quantifying the spread of the LLM distribution using quantile regression (which allowed us to obtain the estimates of the confidence intervals), alternative methods can be considered (e.g. Bayesian neural networks) which might offer a more fine-graines description of the distribution. Finally, motivated by our generalisation results, key next step is the development of universal probing model that can be applied off off-the-shelf to a given L.M across diverse tasks and domains.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
We thank Claudio Fanconi for helpful suggestions on the design of the magnitude-faccooriee probing model. We also acknowledge the financial support of AstraZeneca (Julianna Piskorz) and Eedi (Katarzyna Kobalczyk). This work was supported by Azure sponsorship credits granted by Microsoft's Al for Good Research Lab.
REFERENCES
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matthew Dixon, Ronen Eldan, Victor Fragoso, Jianfeng Gao, Mei Gao, Min Gao, Amit Garg, Allie Del Giorno, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Wenxiang Hu, Jamie Huynh, Dan Iter, Sam Ade Jacobs, Mojan Javaheripi, Xin Jin, Nikos Karampatziakis, Piero Kauffmann, Mahoud Khademi, Dongwoo Kim, Young Jin Kim, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi Li, Yunsheng Li, Chen Liang, Lars Liden, Xihui Lin, Zeqi Lin, Ce Liu, Liyuan Liu, Mengchen Liu. Weishung Liu, Xiaodong Liu, Chong Luo, Piyush Madan, Ali Mahmoudzadeh, David Majercak, Matt Mazzola, Caio César Teodoro Mendes, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Perez-Becker, Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac, Liliang Ren, Gustavo de Rosa, Corby Rosset, Sambudha Roy. Olatunji Ruwase, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning Shang, Hiteshi Sharma, Yelong Shen, Swadheen Shukla, Xia Song, Masahiro Tanaka, Andrea Tupini, Praneetha Vaddamanu, Chunyu Wang, Guanhua Wang. Lijuan Wang. Shuohang Wang, Xin Wang. Yu Wang, Rachel Ward, Wen Wen, Philipp Witte. Haiping Wu, Xiaoxia Wu, Michael Wyatt, Bin Xiao, Can Xu, Jiahang Xu, Weijian Xu, Jilong Xue. Sonali Yadav, Fan Yang, Jianwei Yang. Yifan Yang. Ziyi Yang. Donghan Yu, Lu Yuan, Chenruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna Zhang. Yi Zhang, Yue Zhang, Yunan Zhang, and Xiren Zhou. Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone, August 2024. URL
org/ abs/ 14219 arXiv:2404. 14219 [cs].
Mubashara Akhtar, Abhilash Shankarampeta, Vivek Gupta, Arpit Patil, Oana Cocarascu, and Elena Simperl. Exploring the Numerical Reasoning Capabilities of Language Models: A Comprehensive Analysis on Tabular Data. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the
Association for Computational Linguistics: EMNLP 2023, pp. 15391-15405, Singapore, December
2023, Association for Computational Linguistics. doi: 10. 10.118653v1/2000 /20/33//112222/2/2 URL
www.:/22...iinnneeenlmp.. 1028. 2023, findings wwwenddiing-eenllp....m wm..p.0028
Guillaume Alain and Yoshua Bengio. Understanding inteninniiitttttteeeia layers using linear classifier probes, November 2018. URL BRL.htttp//aaxiv.ooot
01 644 111.v:161.001644 [stat].
Amos Azaria and Tom Mitchell. The Internal State of an LLM Knows When It's Lying, October 2023. URL
arXiv:2304.13753 [cs].
Nora Belrose, Igor Ostrovsky, Lev McKinney, Zach Furman, Logan Smith, Danny Halawi, Stella Biderman, and Jacob Steinhardt. Eliciting Latent Predictions from Transformers with the Tuned Lens, November 2025. URL htl.httpp/// htap:://arxivvv [cs].
2ta.////33..m 08112. arXiv:2303.08 12
DeepSeek-AL. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025.
Rakshitha Godahewa, Christoph Bergmeir, Geoffrey I. Webb, Rob J. Hyndman, and Pablo Montero- Manso. Monash Time Series Forecasting Archive, May 2021. URL httt::://arxiii.orrggg abs/2105. 06643 arXiv:2105.06643 [cs].
Siavash Golkar, Mariel Pettee, Michael Eickenberg, Alberto Bietti, Miles Cranmer, Geraud Krawezik, Francois Lanusse, Michael McCabe, Ruben Ohana, Liam Parker, Bruno Régaldo Saint Blancard, Tiberiu Tesileanu, Kyunghyun Cho, and Shirley Ho. Val: A Continuous Numerical Tokenization
10

---

Published as a conference paper at ICLR 2026
for Scientific Language Models, December 2024. URL 02989 arXiv:2310.02989 [stat].
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshom, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang. Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Eytimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang. Jiccao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid EI-Arini, Krithika lyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins. Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli. Mathew Oldham, Mathicu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic. Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang. Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh. Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman. Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do. Vish Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang. Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi. Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic. Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mcjia. Ce Liu, Changhan Wang. Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty. Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Elconora Presani, Emily Hahn, Emily Wood, Bric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpem, Grant Herman, Grigory Sizov, Guangyi, Zhang,

---

Published as a conference paper at ICLR 2026
Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang. Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Kegian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang. Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj. Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang. Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy. Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li. Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiaojian Wu, Xiaolan Wang. Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang. Yossi Adi, Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang. Zhiwei Zhao, and Zhiyu Ma. The Llama 3 Herd of Models, November 2024. URL http:://axxiii orgg abs 2407 21783 arXiv:2407.2177 [es].
Nate Gruver, Mare Finzi, Shikai Qiu, and Andrew Gordon Wilson. Large Language Models Are Zero- Shot Time Series Forecasters, August 2024. URL WtRLhhtt:///axttt http:/::///axiiooo//abba..//// /at.b////31008881 arXiv:2310.07822 [cs].
Stefan Hegselmann, Alejandro Buendia, Hunter I Lang, Monica Agrawal, Xiaoyi Jiang, and David Sontag. TabLLM: Few-shot Classification of Tabular Data with Large Language Models, March 2023. URL
org. /abs////////////////////////a/a/ /2200.2001222 10723 arXiv:2210.10722 [cs].
Julien Herzen, Francesco Lässig, Samuele Giuliano Piazzetta, Thomas Neuer, Léo Tafti, Guillaume Raille, Tomas Van Pottelbergh, Marek Pasieka, Andrzej Skrodzki, Nicolas Huguenin. Maxime Dumonal, Jan Košcisz, Dennis Bader, Frédérick Gusset, Mounir Benheddi, Camila Williamson, Michal Kosinski, Matej Petrik, and Gaël Grosch. Darts: User-Friend]] Modern Machine Learning for Time Series. Journal of Machine Learning Research, 23(124):1 6, 2022. ISSN 1533-7928. URL / org. www.rrrrggppppppse2/2. http:/papprp//222110pp
html
Roger Koenker and Kevin Hallock. Quantile Regression. Journal of Economic Perspectives,
15(4):143-156, December 2001. ISSN 0895-3309. doi: 10. 10.1257/jep. 14.4.143. URL https
Boshko Koloski, Andrei Margeloiu, Xiangjian Jiang, Blaž Škrlj. Nikola Simidjievski, and Mateja Jamnik. LLM Embeddings for Deep Learning on Tabular Data, February 2025. URL http:
7/ wwwp.xwwwwww.b/////0002 11596 arXiv:2502. 11596 [cs] version: 1.
12

---

Published as a conference paper at ICLR 2026
Jannik Kossen, Jiatong Han, Muhammed Razzak, Lisa Schut, Shreshth Malik, and Yarin Gal. Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs, June 2024. URL
http::/arxivv org/abs/ 2406 15927 arXiv:2406.1190 159277 [es].
Dmitrii Krasheninnikov, Richard E. Tumnr, and David Krueger. Fresh in memory: Training-order recency is linearly encoded in language model activations, September 2025. URL http://
arxiv. org/ aww.ab///555 14223 arXiv:2509. SXXV:25099112233 [es].
Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah, and Joshua Batson. On the Biology of Large Language Model. Transformer Circuits Thread, 2025. URL pub/ htt/2/22//attiibtioooo graphs bio.ooioooooooooooooo html
Oscar Obeso, Andy Arditi, Javier Ferrando, Joshua Freeman, Cameron Holmes, and Neel Nanda. Real-Time Detection of Hallucinated Entities in Long-Form Generation, September 2025. URL arXiv:2509.0353 [es].
Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, and Yonatan Belinkov. LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hal- lucinations, May 2025. [cs].
URL http: httt:::/axxiv.org/abb///////////
wwwgabs///.....ooooo 2410. 02707 arXiv:2410.02707
Koyena Pal, Jiuding Sun, Andrew Yuan, Byron Wallace, and David Bau. Future Lens: Anticipating Subsequent Tokens from a Single Hidden State. In Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL), pp. 548-560, 2023. doi: 10. 11663/v1/2023 conll- URLhttp: http://aaxiv.../.. .rg. bbrgg/bbs/a0 abs/abssl1111 04897 arXiv:231 11.2397 [cs].
Dario Di Palma, Alessandro De Bellis, Giovanni Servedio, Vito Walter Anelli, Fedelucio Narducci, and Tommaso Di Noia. LLaMAs Have Feelings Too: Unveiling Sentiment and Emotion Repre- sentations in LLaMA Models Through Probing, June 2025. http:///:/ .rg. abs/ arXiv:2505. 16491 [es].
2505. 1 91
James Requeima, John Bronskill, Dami Choi, Richard E. Turner, and David Duvenaud. LLM Processes: Numerical Predictive Distributions Conditioned on Natural Language, December 2024.
URL http: http:://axvvv..o org/ wwtpooggg.bb////oeeee 2405. 12856 MiXi::2405.12855 [stat].
Sarthak Roy, Ashish Harshavardhan, Animesh Mukherjee, and Punyajoy Saha. Probing LLMs for hate speech detection: strengths and vulnerabilities, October 2023. URL http:::////.i...rrrrrtt
abs abs/23100 2210 0. 12860
arXiv:23 12860 [cs].
Eli Schwartz, Leshem Choshen, Joseph Shtok, Sivan Doveh, Leonid Karlinsky, and Assaf Arbelle. NumeroLogic: Number Encoding for Enhanced LLMs' Numerical Reasoning, March 2024. URL
http://arxiv. org. abs 2404 00459 arXiv:2404.00459 [es] version: 1.
Aliaksandra Shysheya, John Bronskill, James Requeima, Shoaib Ahmed Siddiqui, Javier Gonza- lez, David Duvenaud, and Richard E. Turner. JoLT: Joint Probabilistic Predictions on Tab- ular Data Using LLMs, February 2025. URL http::/aazii.orr//ab///0000 wwwp://ggg/abbaaeeeeooo 1XXv:2502..1177 [stat].
Aaditya K. Singh and D. J. Strouse. Tokenization counts: the impact of tokenization on arith- metic frontier LLMs, February 2024. URL htt.://p..aaalv..rr///////22.2.2 1490: [es].
Alessandro Stolfo, Yonatan Belinkov, and Mrinmaya Sachan. A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysis, October 2023. URL httt:://arxi> wr../rgg///..... abs///b///// 2305 15054 arXiv:2305. 15054 [cs].
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cris- tian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu,
13

---

Published as a conference paper at ICLR 2026
Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, lliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama Open Foundation and Fine-Tuned Chat Models, July 2023.
org abs 2307 09288 arXiv:2307. .09288 [es].
Robert Vacareanu, Vlad-Andrei Negru, Vasile Suciu, and Mihai Surdeanu. From Words to Numbers: Your Large Language Model Secretly A Capable Regressor When Given In-Context Examples, September 2024. URL httt:://aaaaiieee org/ abs 2404. 07544 arXiv:2404.07544 [es].
Eric Wallace, Yizhong Wang, Sujian Li. Sameer Singh, and Matt Gardner. Do NLP Models Know Numbers? Probing Numeracy in Embeddings, September 2019. URL http::/aaxiv...//
abs/1909 07940 arXiv: 1999999994 [cs].
Jiaxin Wen, Pei Ke, Hao Sun, Zhexin Zhang. Chengfei Li, Jinfeng Bai, and Minlie Huang. Unveiling the Implicit Toxicity in Large Language Models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 1322-1338, Singapore, December 2023, Association for Computational Linguis- tics. doi: 10.18653/v 122.633311/22..mmllpl.. main.84. URL
1a-p-mmaingg ww..aniin....mm 847]
Hao Xue and Flora D. Salim. PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecasting, December 2023. URL
arXiv:2210.08964 [stat].
Shu Yang, Linbo Wang, and Peng Ding. Causal inference with confounders missing not at random, February 2019. http:://p///
wwwp/as../////ooooo..//.../....r /tt.s/////2... 03951 arXiv:1702.0399 [stat].
Yongchao Zhou, Uri Alon, Xinyun Chen, Xuezhi Wang, Rishabh Agarwal, and Denny Zhou. Transformers Can Achieve Length Generalization But Not Robustly, February 2024. URL
http://arxiv/ www./////////eo.oe/...//ee.eeae/
awtpab////2 22tp:/2022.00220 09337 arXiv:2402.09371 [es].
Angela Yaqian Zhu, Nandita Mitra, and Jason Roy. Addressing positivity violations in causal effect estimation using Gaussian process priors. Statistics in Medicine, 42( 2023, ISSN 1097- 0258. doi: 10. 10.1000//im..600
[tL//tttes//onnn.nellleeeee....e _eprint: http://oollllaaaagygy..oooooioii
/bbs/
10 1002/00/0 s im. 9600
14

---

Published as a conference paper at ICLR 2026
APPENDIX CONTENTS
Related Works
15
Details of the Experimental Sctup
16
B.1
Assets and Licensing Information
16
B.2 Computer infrastructure used
16
B3 Details of the datasets)
17
Details of the probing models
18
Additional Experimental Results
19
C.1
Comparison against vanilla MLP probe
19
Results with other LLMs
20
Computational Costs and Interence Time
25
Layer Ablation Study
25
C,5 Mean Absolute Loss per quantile
25
A
RELATED WORKS
LLMs for Structured Data. With the advances in the performance of LLMs, several methods
have been proposed which utilise LLM's pre-training and prior knowledge make predictions
structured data, namely tabular data (e.g. Requuema et al. 2022 Hegselmann et al. 2023 Shysheya
al. 2025 Vacareanu et al. 2024) and time series data (e.g. Gruver al. [024 Xue & Salim 2023).
These works show that L.LMs can serve competitive predictive models even without task-specific fine-tuning. This is particularly evident the small sample regime, where large-scale pre-traine pre-training, access to prior knowledge and ability to condition textual information allows LLMs to match or outperform the performance of purpose-burt regressors.
Numerical Predictive Distributions LLMs. When used as regressors, LLMs can provide not only point estimates but also full predictive distributions, reflecting their stochastic nature. To elicit continuous distributions over numerical outputs, Gruver al. (2024) and Requeima et al. (2024) propose autoregressive approach that generates logit values bins, which are then scaled to form a valid probability distribution. Access to such distributions is crucial for downstream tasks requiring uncertainty quantification, including decision-making under uncertainty and Bayesian optimisation. However, these methods are computationally intensive, as they require multiple sequential queries to the LLM construct single distribution (e.g., p(123.4) (23))) (.12))(((11)))pppp)))))) 113))p( (p0pp1((())))__ (223))). This motivates us to explore alternative approaches to eliciting numerical predictive distributions from LLMs.
discretised numeric
Discrepancy between number generation and autoregression. As next-token predictors, LLMs are not explicitly trained to understand the value of numbers. Due to their autoregressive nature, early tokens encode digits before key decisions like decimal placement (that determine number's magnitude) are made. This can lead to surprisingly poor performance simple numerical tasks (Yang al., 2019; Akhtar al. 2023 Zhou al., 2024; [chwwarzz al., 2024). To address these limitations, several works have proposed alternatives to standard autoregressive decoding for numerical predictions. For instance, Golkar et al. (2024) introduce a special [NUM) token, replaced post-hoc with continuous value predicted by learned regression head-though this requires retraining the model. Others (Singh & Strouse 2024 Schwartz et al. 2024) investigate number- specific tokenizations to improve numerical accuracy of LLMs. In contrast, ask whether one can bypass autoregressive decoding in pre-trained LLMs by directly reading out the predictive distribution from the internal representations.
15

---

Published as a conference paper at ICLR 2026
Probing and LLMs. Probe classifiers, simply probes, models trained uncover specific properties directly from the intermediate representations of neural models Alain & Bengio (2018). R). In the context of LLMs, growing body of works from the area of mechanistic interpretability studied what kind of properties can be directly recovered from the internal representations of the LLMs (without the need for decoding), and where within the internal representations they located (e.g. which layer). Amongst others. prior work has probed LLMs for properties such as factuality of the generated responses Obeso et al. (2025); Orgad al. (2025 y Azaria & Mitchell (2023), semantic entropy Kossen et al. (2024), toxicity Roy et al. (2033); ; Wen et al. (2023), text sentiment Palma et al. (2025) and even training order recency Krasheminnikov primarily on training probe classifiers, in this work we focus probe regressors, proposing our magnitudeefacctrissed probe as method for handling targets with large range of continuous values (spanning several orders of magnitude). By focusing on continuous values, we uncover that the hidden states of the LLMs uncover much more fine- ann-grained information than suggested by previous works (which probe primarily for binary-or binarised -proiittdd-roprttess)
(2025). While the previous works focus
Complementary findings from mechanistic interpretability suggest that, even in purely textual settings, LLM hidden states encode representations of tokens that the model is most likely to generate several tokens ahead (Pal et al.) 2023 Lindsey et al., 2025 Belrose et al.) 2025). In contrast to these works, show that similar results hold also when using LLMs for numerical predictions, allowing uncover not only the marginal distributions tokens at each of the steps ahead, but also the properties of the entire numerical predictive distribution of the LLM (mean, median, quantiles).
Probing numeracy in LLM embeddings. A number of prior works give evidence that simple probing models can be used to learn numerical values encoded in the LLM embeddings. Wallace al. (2019) has shown that the value of number can be successfully decoded from its encoded word embedding (e.g., "71" + 71.0.). [Solloo et al. (2023) identified specific layers in LLMs that store numerical content, recoverable via simple linear probes. while Zhu et al. (2023) demonstrated that intervening on these layers alters generated outputs. More recently, Koloski et al. (2025) showed that L.M embeddings can serve as effective covariates in downstream regression models.
Taken together, these results support the hypothesis that it should be possible to train probes that efficiently approximate the numerical predictive distribution of the LLM, motivating our work.
B DETAILS OF THE EXPERIMENTAL SETUP
B. ASSETS AND LICENSING INFORMATION
The following existing assets used to produce the experimental results:
Monash dataset
Godahewa et 2021
Darts dataset Herzen al. 2022
Llama-2-7B model
Touvron 2023
Llama-3-8B model
Grattatiori et al. 2024)
Phi-3.5-mini model Abdin et al. 2024
DeepSeek-RRI-Distil-LLllmmm model DeepSeek-A] (2055)
B.2 COMPUTER INFRASTRUCTURE USED
Hardware. All experiments were conducted using 2 separate NC24rs instances and one NCSOadis H100 v5 instance on the Microsoft Azure cloud platform. These instances part of Azure's ' GPU-optimised virtual machine series, with their hardware specifications summarised in T
Generating the synthetic dataset for one scaling factor Duale E {1, 10, 1000, 10000) took no more than 10h. Training one probe model took more than 4h.
16

---

Published as a conference paper at ICLR 2026
Table Azure Virtual Machine Specifications
Specification
NC24rs v3
NC80adis H100_ _5
vCPUs
24
80
System Memory (GiB)
448
640
GPU Model
4x NVIDIA Tesla V100
2x NVIDIA H100 NVL
GPU Memory (per GPU)
16 GiB
94 GiB
Total GPU Memory
64 GiB
188 GiB
GPU Architecture
Volta
Hopper 12.x
CUDA Version
11.x
CPU Model
Intel Xeon E5-2690 v4
AMD EPYC Genoa
Local Storage
2.9 TB
7.1 TB
B.3 DETAILS THE DATASETS
B.3.1 DETAILS OF THE SYNTHETIC TIME SERIES DATASET
We generate synthetic dataset comprising time series derived from a family of parametric functions, each evaluated over a fixed domain and perturbed with controlled noise. The purpose is to simulate diverse temporal patterns, inducing varying levels of uncertainty in the LLM's predictions.
We use a set of base functions defined over the interval æ E [0, 60). discretised into 120 equidistant points. The functions are summarised in 6 For each function and value of a, we generate clean series y = f(a æ), and then apply:
Additive Gaussian noise with variance o² E f0.0, 0.0,, 0.13.
Vertical scaling by ~ (1, Dscate)
Vertical translation by d ~ (((-Daalte Dwate)
From each transformed series, we sample 10 different subsequences for each length n {3, 5, 7, 10, 13, 15, 17, 20, 25, 35, 40), with random starting offsets. Each subsequence is used as a candidate training input x;- Inputs are serialised as floating-point strings with p decimal places (we p for Decale 1.0, p for Dicale 10.0, p 2 for Dscate 1000.0 and p 1 for Dscale = 10000.0). This results 33600 generated raw time series for each value of Dscale.
Concatenation, filtering, and final splits. We then use the raw generated datasets to create four seale- specific datasets which are bounded in value and exhibit balanced distribution across magnitude orders. Firstly, after querying the LLM to generate 100 predictive samples for each input sequence, we keep only those series for which 100 predictive samples were successfully generated. After this step, we combine all raw datasets and derive balanced subsets at each scale level by binning examples x; according the median value of the LLM prediction, mydedian] using 8 logarithmic bins [10-3, 10*). We cap the number of samples in each bin at 12,000; bins may contain fewer samples the initial data generation process does not yield sufficient number of instances, particularly for bins corresponding to the smallest magnitudes. To avoid outlier values we also filter the data to maintain only the series x; for which
mediiam
Iy greody
< D Dxcule-
Thus, the final size of each scale-specific dataset is seueeddependent. with approximate sizes of 18k, 30k, 48k, and 60k for Daaae E (1.0, 10.0, 1000.0, 10000.0), respectively. Finally, for each dataset. we create train/validation/test splits with proportions 0.8/0.1/0.1.
B.3.2 MONASH DATASET
Data Loading: We use the data from the Monash dataset, preprocessed by Gru- ver et al. (204) and available from lntts:://rrve. httpc/glle..o/.fee///ee/ooeo Each sub-
1sKrpWbD33R BerooWDDLvvLi 3LvLO e5 dataset file contains tuples of the form (train, test), which are concatenated to form complete univariate time series.
17

---

Published as a conference paper at ICLR 2026
Function name
Formula
d-range
sin
sin(x)
[0.5, 6.0]
linear_sin sin
0.2 sin(x) 430
[0.5, 6.0]
sinc
sinc(x)
[0.05, 0.2]
xsine
x #-30
sin (x: 30)
(0.3
50
beat
sin sin(x) (z) sin (
[0.1,
(a-28 2)
gaussian__wal wave random
cos (10 - 2))
[0.01. 0.1]
1(11,1)
[0.0, 1.0]
Table 6: Functions used to generate time series data, their mathematical forms, and the range of the time-scaling parameter a.
To ensure computational tractability, each series is subsampled (via strided slicing) to contain at most 1000 time steps.
Resampling:
Series Selection: For each dataset, maximum of 50 time series are selected random to control the number of examples used during training.
Subsequence Generation: From each selected series, we extract multiple training subse- quences of varying lengths (3, 5, 7, 10, 13, 15, 17, 20, 25, 30, 35,35, 40). For each length, we generate up to 10 training subsequences, sampled at different offsets.
B.3.3
DARTS DATASET
* Data Loading: We use the data from the Darts dataset, available from the dart python package. We use the following sub-datasets: AirPassengerrsattsset, AusBeerDataset, GasRateCO2Datasst,, MonthlyMilkDataser,,, SunspotsDataset, WineDattset,, WoolyDataset, HeartRateDDttaet.
Resampling: To ensure computational tractability. the series for the datasets Sunspots- Dataset and HearlRateDataset are subsampled (via strided slicing).
Series Selection: For each dataset, all available time series are selected.
Subsequence Generation: From each selected series, we extract multiple training subse- quences of varying lengths (3, 5, 7, 10, 13, 15, 17, 20, 25, 30, 35, 40). For each length, we generate up to 10 training subsequences, sampled at different offsets.
B.3.4 LLM GENERATION SETTINGS
We generate the LLM hidden states from LLMs available through the huggingface library. For each of the input time series, obtain 100 samples from the LLM, generated autoregressively. as well as the greedy generation. During generation for -lama-2277,, as its tokeniser encodes each digit separately,
narrow down the generated tokens just the digits, decimal point and +/- signs. For obtaining the random samples, we use tempe rature=1. 0 and top_p=0.___.. 95. We exclude from the final dataset samples for which generation failed least once (i.c. the obtained generation was not a valid number). such that each time series in the final dataset has exactly 100 valid LLM samples.
B.3.5 TRAIN- WALINAAIO----TEEEE SPLIT
Before training, we split each of the datasets in 80% training dataset, 10% validation dataset and 10% test dataset. Unless otherwise stated (in the generalisation experiments), these splits are random. We do not apply any scaling or transformation to either the LLM embeddings (which are inputs to our model) or the outputs.
B.4 DETAILS OF THE PROBING MODELS
Our magnitude- factorised regression models, used both for the purpose of point prediction and for the purpose of quantile regression have the hyperparameters reported in Table and Table We train
sagnitude-factorieed
18

---

Published as a conference paper at ICLR 2026
the model using the ADAM optimiser. Details of the implementation be found in codebase, available at mtlttps://githule hcwp://aaiiaakkaball////////aaae 11m.
Hyperparameter
Description
Default Value
min_mag mag
Minimum exponent for base-10 magnitude
3
scaling (as used by furder)
mag_mag
Maximum exponent for base- 10 magnitude
log (Daale)
scaling
alpha
Weight for classification loss component
100.0
beta
Weight for regression loss component
50.0
K
Top- K exponents taken into consideration
(see Equation
hidden_layers
Number of hidden layers in feature extrac-
hidden_dim
Dimensionality of hidden feature represen- 512
tation
activation fune Lorden_state____
Activation function used in the MLP GELU
A list of the hidden states H. to use as imput
[25, ,,, 32
Table 7: Model-specific hyperparameters for the magnitude-factorissdd models.
Hyperparameter
Description
Default Value
learning_rata
Learning rate for the optimiser
10-5
weight_decay
L. regularization weight Learning rate scheduler step size Learning rate scheduler gamma Number of samples per training batch
1.0
scheduler_sste_siiee
100
scheduler_gamms
0.5
batch_size
1024
max_epochs
Number of training epochs Patience for the early stopping
600
patience
200
Table 8: Optimiser and training-related hyperparameters.
C ADDITIONAL EXPERIMENTAL RESULTS
C.1 COMPARISON AGAINST VANILLA MLP PROBE
In Table 9 we compare our magnitude-factee factorised probe with a vanilla MLP trained on the same hidden representations. Both models single hidden layer of size 512 and learning rate of 10-5 Evaluation is conducted on Llama-2 using the synthetic time series dataset with scale D = 1.0, which provides the narrowest range of magnitudes among those study. Even in this relatively simple setting, the magnitude-factoriia reducing MSE by 41%. 33%, and 42% for greedy, mean, and median predictions respectively. These results highlight the importance of probe design: a negative finding with one architecture does not necessarily imply that information is absent from the LLM'' hidden states may simply reflect the limitations of the probing model.
probe achieves substantial improvements over the baseline.
Table 9: MSE of the values predicted with the magnitude-factoriaa (MFP) model vs. vanilla MLP
predicted value
MFP probe (ours)
MLP probe (log-scaling)
MLP probe (no log-scaling)
greedy
0.0150
0.0400
0.0255
0.0061
0.0091
0.0092
median
0.0058
0.0101
0.0100
19

---

Published as a conference paper at ICLR 2026
C.2 RESULTS WITH OTHER LLMS
We provide results for the key experiments in the main paper with more LLMs. As the tokenisers of models outside of the Llama-2 family do not encode digits separately, do narrow down the generated tokens during decoding. For obtaining the random samples, we use temperature= and top 11te_pp0000000 95. We perform repeated sampling until for each time series, we obtain 100 LLM samples Vo2 P (x,). For Llama-3.2-3B 3.2-BB, as the model contains less layers, we also concatenate the representations from across the last 8 layers, which amounts to layers ,
Targat meam
Targat: madian
Prediced calle (oe sala)
Tanget gnady
Ausmayy by expurent acrons Largets
Podem Ressery
nd
Rd
Prokkced
Prason R
Parxen R. 0.56
Porsen F. 0.33
60
22
-1
Trae valu (lotu scal)
Tax válaa Denus scda)
Tre valas (bou saik)
Truc siposor:
(a) Llama-2-7B
Targat maan
Targat: madiann
Prodicadd vallen Dricga soaky)
Tagee gnadly
Assunny by sogurent actONs Largets
Pederin Reaurity
04
Proliced
Pueson R 0.00
Porne R. 0
Paras 0.2
True (lorr scal)
(osts scda]
(bius scak)
exposne
(b) Jlama-3-8B
Targat: madiian
Prodiceed oolen Dhigs mcae)
gnady
1.0
Largels
Pietrton Arnerry
Sa
Proliced who Dit.
(c) Llama-3.2-3B
Target: meon
Torget: median
Target gready
Accuracy by exponent scroes targets
Prediced odln Docap soae)
Pietrttn Ne aey
0d
Profecod
Puevon
Poevi
Paran
Trae valu (lons
Tru valce Doats scale]
Trae valae (heus salk)
The exposse
(d)
Target: meon
Torget: median
Target gready
Accuracy by exponent bcross targets
ad Dorcp sode)
Prederon aor
Ga
0:1
Proliced
Presos & 3.90
Porxi R 0
Poraet 0.22
60
22
-1
Trae valu (lota sol))
Tax vilce (eaus scalo]
Tra value (bepu saak)
Tru exposne
(c) DeepSeck-PR
Figure Predicted vs. sample mean, median and greedy prediction (on login scale). Results analogous to Figure
20

---

Published as a conference paper at ICLR 2026
Target: mean
Target: median
Target: greedy
Prediced volue
0.0
-0.5
1.0
-1.0
-0.5
0.0
0.5
1.0
-1.0
-0.5
0.5
1.0
-1.
-0.5
0.5
1.0
True
True
True
(a) Llama-2-7B
Target: mean
Target: median
Target: greedy
1.0
Prediced volue
0.0
-0.5
1.0
0.5
0.0
0.5
1.0
1.0
0.5
0.0
0.5
1.0
1.0
0.5
0.0
0.5
1.0
True
True
(b) Llama-3-8B
Target: mean
Target: median
Target: greedy
1.0
Predicted value
0.0
-0.5
-1.0
1.0
0.5
0.0
0.5
1.0
1.0
0.5
0.0
0.5
1.0
1.0
0.5
0.0
0.5
1.0
True
True
(c) -lama-3.2-3D
Target: mean
Target: median
Target: greedy
1.0
Predicted value
0.0
-0.5
-1.0
1.0
0.5
0.0
0.5
1.0
1.0
0.5
0.0
0.5
1.0
1.0
0.5
0.0
0.5
1.0
True
(d) Phi-3.S-instruct
Target: mean
Target: median
Target: greedy
1.0
Predicted value
0.0
0.5
1.0
0.5
0.0
0.5
1.0
1.0
0.5
0.0
0.5
1.0
1.0
0.5
0.0
0.5
1.0
True
(c) DeepSeck- Mae-Seck-RR-Disseee RI-Distill-Llama 8B
Figure Predicted vs. sample mean, median and greedy prediction on the dataset with scale Dacale = 1.0.
21

---

Published as a conference paper at ICLR 2026
Sample Efficiency of Mean Estimation
Sample Efficiency of Mean Estimation
0.25
MSE w.r.t. Test Value
LLM Sampling (mean)
MSE w.r.t. Test Value
0.150
LLM Sampling (mean)
0.20
Probe
Probe
0.125
0.15
0.10
0.100
0.05
0.075
0.00
15
10
20
Number of LLM Samples
Number of LLM Samples
(a) Llama-2-7B
(b) Llama-3-8B
Sample Efficiency of Mean Estimation
Sample Efficiency of Mean Estimation
0.16
MSE w.r.t Test Value
LLM Sampling (mean)
MSE w.r.t. Test Value
LLM Sampling (mean)
0.14
0.14
Probe
Probe
0.12
0.12
0.10
0.10
0.08
0.08
15
20
25
15
20
Number of LLM Samples
Number of LLM Samples
(c) -.ama-3.2-3B
(d) Phi-3.5-instruct
Sample Efficiency of Mean Estimation
MSE w.r.t. Test Value
LLM Sampling (mean)
0.12
Probe
0.10
0.08
10
15
20
25
Number LLM Samples
(e) DeepSeek Biniieekk-R--iiie c1-DDitill-Llama--3
Figure 9: Sample efficiency on the task of one step ahead predictions. Results analogous to Figure 4
The shaded regions mark 95% confidence sets, computed using 100 bootstrap samples. We use D = 1.0.
22

---

Published as a conference paper at ICLR 2026
Table 10: MSE for the predictions on the dataset with scale D 1.0, reported for all models.
Results analogous to Table
(a) Llama-2-77
(b) Llama-3-88 3-BB
Vi target
D (ours)
X
X:
Vi target
Dž (ours)
X
X;
21,m
mean
0.006
0.256
0.035
0.085
mean
0.007
0.253
0.047
0.093
median
0.006
0.260
0.041
0.087
median
0.012
0.264
0,061
0.106
greedy
0.015
0,273
0.065
0.109
greedy
0.016
0.255
0.072
0.122
(c) Llama-3.22 3.2-3B
V. target
07 (ours)
X;
mean
0.007
0,260
0.0484
0.092
median
0.013
0.271
0.062
0.104
greedy
0.018
0.267
0.075
0.119
(d) Phi-3.5-minniinssruut
(c) DeepScek-RI RI-Distill- Distill-Llama-S8h
y target
0f (ours)
x
x;
Zi,n
yr target
0 (ours)
%;
mean
0.008
0,248
0,042
0.100
mean
0.009
0.247
0.045
0.110
median
0,012
0.252
0.047
0.104
median
0.013
0.253
0.056
0.120
greedy
0.020
0.270
0.060
0.113
greedy
0.020
0.264
0.069
0.135
Table 11: Coverage of the for all models. Results analogous to Table
(a) Llama-2-42 2-7B
(b) Llama-3-8B
50%
90%
95%
a dataset
50%
90%
95%
dataset
1.0
52.0 0.4
0.3
95.5 + 0.2
1.0
90.4 0.6
0.4
94.7 0.3
10.0
52.7 0.5
96.1 0.3
0.2
10.0
53.8 0.6
9.4
0.2
1000.0
0.3
0.3
95.7
1000.0
51.7
91.2
9.2
10000.0
48.2
95.4 0.2 0.2
10000.0
50.8
95.3 91.1
(c) Phi-3.5-mini-instrec
Q dataset
50%
90%
95%
1.0
49,9 + 0.5
89.3 + 0.4
94.3
10.0
0.4 0.5
0.3
1000.0
88.0+0.3 4.4
9.3
10000.0
47.3 0.3
0.3
93.
23

---

Published as a conference paper at ICLR 2026
Pastonet scale: 1.0
Datvve stale: 10.0
Darasst ecae: 1000
Dataver scale 10000
Predited
10me
Speartan Ai 0.00
Spsammn 0.90
Spuarman 0.00
Spearran A0 0.00
Sample IQR
(a) Llama-2-7B
Dottoset scale: 1.0
Dataset scale:
Cotnect scale: 1000
Dataset scale: 10000.0
Prelicted
10ra
Spauraa AR 0.88
Pramoo
Sptomaa 0.00
0:75
0.03
SpaarEzR 12
Pammos
Sample KQR
(b) 1lama-3-88
309
001
Preficted
10"4
Paoroo IP
Ploross F:
Pronsa 03
10m1
Sample IOR
(c) Phi-3.5-instruct
Figure 10: Predicted IQR vs. Sample IQR (median adjusted). Results analogous Figure
24

---

Published as a conference paper at ICLR 2026
C.3 COMPUTATIONAL COSTS AND INFERENCE TIME
Crucial for understanding the validity of our approach is the comparison of the inference cost of the proposed probe vs. autoregressive sampling. Ignoring the cost of encoding the input time series with the LLM (as this cost is shared by both methods) we note that decoding each token with the 7B- 7B-parameter LLMs (such as Llama-2) requires 14B FLOPS, which means that, for a model with digit-by-digit tokenization obtaining n samples of a 5-digit number requires X 70B FLOPS. In comparison, using our magnitude-factorisi of the LLM and thus has input of size 8 x 4, 096 = 32, 768, 1 hidden layer of dimension 512, and a maximum of k = 9 separate heads corresponding to each of the magnitude bins) incurs the constant cost of 32, 768 512 + 512 X 9 & 17M FLOPS for the magnitude classification model and 32, 768 x 512 + 512 x Re 17M FLOPS for the regression model, so 34M parameters in total per one statistic. If we wanted to predict, say. 7 different quantile values, we would need models of the same size, requiring 234M FLOPS. Thus, the required number of computations for using the probe generating even LLM sample is far lower.
probe (which we assume uses 8 concatenated layers
We further validate these estimates by comparing the times needed to generate n samples from the LLM vs. the time needed obtain single median estimate from probe. We the experiments using the Llama-2-7B model, averaging the results over 100 random time series samples consisting of 20 data points each. We run all the timing experiments on one instance of a H100 GPU. In this setting, inference with trained probe model (including the process of obtaining the hidden representation of the LLM) takes 0.034 = 0.006s. We include the times required for obtaining n LLM samples in Table 12, These results confirm that even generating single sample via autoregression is roughly 47x slower than running the full inference pipeline with our probe.
Table 12: Average time to generate n LLM samples.
n samples
1
10
20
50
100
time (8)
1.59 + 0.08
1.62 + 0.07
1.71 + 0.07
1.83
0.07
2.35 + 0.08
3.28 + 0.08
C.4 LAYER ABLATION STUDY
In this section, we ablate the choice of the layers H. used by probing models by training individual layers of the LLM, letting H = {E} for each E E [16...., 32). Our results provide further insights into how information about the LLM's numerical predictive distribution is spread across the layers.
Figure shows the results for the probing models from section For the target of the greedy LLM prediction, we see that the most relevant information is contained in the last layers of the LLM, although generally the variations between the layers are not that significant. We further observe that concatenating information from across different layers leads to significant performance improvements. For the mean target, which is predicted with a higher accuracy, we observe that information relevant for predicting the magnitude and the scaled value of the target is distributed less uniformly, concentrating in the 30th layer.
Table 12 shows analogous results for the quantile regression model from section 3.2 The observed pattern for median prediction is similar to the one mean prediction from Figure In terms the uncertainty information, we find it to be more uniformly encoded across the layers and again that concatenating information across several layers leads to an improved probing performance.
C.5 MEAN ABSOLUTE LOSS PER QUANTILE
In addition the results presented in Section we report the mean absolute error between the the empirical quantile q and the predicted quantile q" for all quantile levels. Results are shown in Table [3 We observe that the absolute errors are higher for quantile levels further from the median.
25

---

Published as a conference paper at ICLR 2026
Magnit.ds Prediction Accursoy
Final Prediction on MSE
Magnitude Accuracy
Firal MSE
0.80
AMlascea Concrnanttdd Probe
0000
Abfrrien Concaionated P-obz
691
0.114
Layer Abtatios
Consiterated Probe
8:33
0.97
tum6
6/00
0.000
(a) Prediction of groedy
(b) Prediction of y:
mean
Figure 11: Layer ablation for Llama-2-7 on the probing model from section
Layen ALAaCono
[ankloud w. serele IQRC
Conciterated Pribe
Phoron
0.005
0.66
0000
0.006
0.01
Consaterattd
Layer AbriiA
Figure 12: Layer ablation for Llama-2-78 on the quantile probing model from section 3 MSE on the median (left, lower is better) and the Pearson R between predicted IQR and sample IQR (right, higher better).
Table 13: Quantile regression absolute error per quantile. Values represent the mean absolute error between the empirical quantile q" and the predicted quantile g" for all quantile levels.
quantile level 7 dataset
0.025
0.05
0.25
0.5
0.75
0.95
0.97
1.0
0.58
0.40
0.16
0.05
0.15
0.40
0.60
10.0
3.12
2.16
0.45
0.28
0.49
1.91
3.26
1000.0
111.16
61.54
14.83
12.10
14.30
46.04
115.71
10000.0
843.11
380.06
119.80
100.41
118.90
483.33
1172.27
26


Reviews:
[{"review_id": "K4qL8MMvIt", "reviewer": "Reviewer_Mkh9", "summary": "This paper addresses the question of whether LLM's hidden representations encode distributional information about their numerical predictions rather than relying on autoregressive decoding when it comes to utilizing LLMs for time-series forecasting regression tasks. They Introduce a probing model that separately predict the order of magnitude ( treated as classification ) and the multiplicative residual to predict the mean, median and the greedy predictions from the LLM internal representations. They further extend their results to predicting multiple quantiles of the underlying distribution utilizing quantile regressors. They lastly validate generalization capabilities of the underlying predictors.", "questions": "1. How sensitive are the probe's predictions to the choice of layers used ? It would be interesting to analyze which layers contribute the most to recovering the numerical predictions, and whether the same choice of \"best\" layers is consistent across tasks and different settings studied in the paper under \"generalization\". \n\n2. One form of generalization that is not studied in the experiments is whether a probe trained on llama2 for example generalizes ( or not ) to different models from the same family ( for example llama3 ). Results discussing this would be interesting.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. the paper is very well written. All experiments all well-motivated, clearly explained, and results are well presented. \n\n2. The idea of eliciting numerical predictions and uncertainties from hidden states is timely, and can possible be extended to tasks beyond time series forecasting. \n\n3. The experiments are sound, and different aspects such as different kind of generalizations are well considered, thus are comprehensive. I found the calibration analysis interesting extension of the experiments and that it strengthens the credibility of the results. \n\nLastly, and to summarize, the paper is well written, and the findings are interesting, informative, and of interest to a broader community. The evidence that distributional information about LLM predictions are recoverable from hidden states is enlightening followed by sound experimental evidence.", "weaknesses": "Most of the weaknesses are well acknowledged in the paper. These include (i) accessing hidden states may decrease the practicality of the method, (ii) relying on extensive autoregressive sampling for training the probes. \n\n1. While the paper positions itself only within the context of regression and time-series forecasting, it does not sufficiently discuss prior work that also show hidden states encode future outputs and uncertainty in other modalities and tasks. There is a growing literature that shows similar behaviors that probe internal states in the context of analyzing sentiment, factuality, intent, jailbreaking, chain-of-thought reasoning and etc. The related works would benefit from addressing these works as well for better positioning the contribution within a larger context.\n\n2. The experiments in section 2.2 focus on [-1,1] range but its not very clear how the method behaves for larger value ranges. although the authors mention testing other ranges, the results don't show how performance and calibration changes with scale, and it would be interesting to see how performance metric across different ranges change; whether larger numeric ranges introduce instability, or if normalization essentially removes that issue.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DrvxuQuTB7", "reviewer": "Reviewer_x9bF", "summary": "The paper introduces a set of regression probes to predict the distributional properties of LLM-based time-series forecasters. By training the probes on synthetic data, the probes can predict quantities such as the greedy prediction, mean, and median with good accuracy without autoregressive decoding in both ID time-series and, to a less extent, OOD time-series.", "questions": "- Can the proposed approach be generalized to predict distributional properties of a future sequence, rather than a single value?\n- How much total runtime or compute saving do the probes provide over decoding from the LLM across the experiments?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- I appreciate the careful design going into designing the parameterization and loss functions of the probes, which is important for handling values with large ranges.\n- The empirical result shows good agreement between the probe predictions and actual values obtained from decoding.\n- The computational efficiency of the probe over decoding / sampling is appealing.\n- Overall good presentation.", "weaknesses": "- A main finding of the paper is that LLM’s predictive distribution is encoded in its internal activations. This statement seems trivially true. The predictive distribution (jointly over all future tokens) is fully determined given the hidden states of the LLM (KV cache) as a direct consequence of the model architecture.\n- The probes only make predictions about a single next value, rather than a future sequence. In practice, time-series forecasters are used to make extended, variable-length predictions over a non-trivial horizon, where this approach does not trivially generalize over to. In other words, the probe only shortcuts autoregression over the digits, but not time steps.\n- I'm not convinced that the probes provide significant computational speedup over directly decoding from the LLM. In both approaches, most of the computation is in running the LLM forward pass on the input sequence. Decoding from the LLM is much cheaper, especially for predicting only the very next value. Thus, the savings from the probes can be negligible relative to the overall cost.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5Ifx1Lnjve", "reviewer": "Reviewer_hDqn", "summary": "The authors explore the ability to extract the predictions of an LLM dependent on its hidden states of its final layers using a variety of methods. For their main experiment, they adopt a mantissa+exponent floating point style method of calculating the regression model, with the mantissa trained as a regression problem and the exponent trained as a classification problem. Only the exponent is trained first, then the mantissa is used to fine-tune the results. They then compare this against the point estimates of the greedy, median, and greedy methods of the LLM. They additionally train a quantile-regression probe using pinball loss to quantify uncertainty.", "questions": "If LLMs generate results autoregressively and the model does not work as well for the greedy approach, does this mean that it would still be hard to capture the results of the LLMs purely from the hidden states?\n\nHow much contribution does the floating-point formulation actually bring for the probe, compared to just predicting the value directly? If it is significant, then how would this method compare to using it against a standard regressor (without LLM) with the same floating-point formulation?\n\nWhy is the LLM sample error so high in the beginning for Figure 4?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Strengths\nThis is an interesting dive into whether both 1. point estimates and 2. uncertainty can be recovered from the LLM's hidden state.\n\nThey show advantages against standard LLM sample-based prediction in Figure 4, along with improved results over GP in the tables for the time-series regression task.\n\nThey also demonstrate some generalization properties, which can be instrumental when dealing with distribution shift.", "weaknesses": "Though the author's main goal seems to be moving towards sidestepping autoregressive generation, they only explore one-step prediction in the current results. Additionally, for some of the main other results, they calculate statistics and last step predictions, it may be worth it to show average MSE across time rather than MSE against the average in time.\n\nCurrently, the experiments seem to be all done on synthetic datasets. While this is useful for the uncertainty metrics, having some error-based analysis on some standard real-world time series datasets could strengthen the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yDgigdPRwr", "reviewer": "Reviewer_8jM1", "summary": "In an empirical paper, authors target numerical prediction in LLMs by decomposing prediction into magnitude classification and scaled value regression. The key contribution is learning the magnitude and a normalized value separately, and using the predictions before LLM token generation. \n\nWhile it is hinted that the idea enables uncertainty-aware numerical \nprediction without repeated sampling, evaluation is limited to maximum Pinball loss and IQR prediction in generated data.\n\nWhile the methods empirically work well, we gain no insight regarding the LLM representation behind.\n\nThe paper is well written and source code is provided anonymously.\n\nWhile the paper presents convincing empirical results for the applicability of the decomposed numerical prediction learning in LLMs, I miss theoretical or architectural explanations. Also, the uncertainty considerations are shomewhat limited.", "questions": "The discussion of Table 1 could be more elaborate. Do I understand well that the LLM raw output is much worse than the mean or median of several of its outputs? I see no explanation, not even in the Appendix.\n\n3.1 pinball loss: why the maximum over quantiles? Why not the integral, as in CRPS [Alexander Jordan, Fabian Kruger, and Sebastian Lerch. Evaluating probabilistic forecasts with scoringrules. Journal of Statistical Software, 90(12):1–37, 2019, Diane Bouchacourt, Pawan K Mudigonda, and Sebastian Nowozin. Disco nets: Dissimilarity coefficients networks, Neurips'16]?\n\nIn the training procedure of eqs (6-7), how can you handle if a multimodal distribution has modes of different order of magnitude? Isn't eq (6) too restrictive?\n\nWhile the main goal is certainly not to provide the best model for distribution learnting, could you compare other methods that learn the distribution, e.g. [Shengyang Sun, Changyou Chen, and Lawrence Carin. Learning Structured Weight Uncertainty in Bayesian Neural Networks. AISTATS'17]. \n\nFig. 6 only shows MAE, it would be interesting to see e.g. Pinball Loss, compared to non-LLM models for learning distributions.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "+ New contribution towards improved numeric prediction capabilities of LLMs\n+ Very well written paper\n+ Source code is available", "weaknesses": "- No theoretical and/or architectural explanation of how the LLMs represent the numeric range and scaled value\n- The uncertainty part could be strengthened\n- Comparison with methods that learn the distribution of regression problems (e.g. Bayesian NN, Mixture Density Networks), while not absolutely necessary, could make the contribution more valuable", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)