Introducing Background Temperature to Characterise Hidden Randomness in Large Language Models
Authors: Alberto Messina
Stefano Scotta
Venue: Accepted by TMLR ()

Abstract: Even when decoding with temperature $T=0$, large language models (LLMs) can produce divergent outputs for identical inputs. Recent works align in highlighting implementation-level sources of nondeterminism, including batch-size variation, kernel non-invariance, and floating-point non-associativity. In this work, we formalize this behavior by introducing the notion of background temperature $T_{\mathrm{bg}}$, the effective temperature induced by an implementation-dependent perturbation process observed even when nominal $T=0$. We provide clean definitions, show how $T_{\mathrm{bg}}$ relates to a stochastic perturbation governed by the inference environment $I$, and propose an empirical protocol to estimate $T_{bg}$ via the equivalent temperature $T_n(I)$ of an ideal reference system. We conclude with a set of pilot experiments run on a representative pool from the major LLM providers that demonstrate the idea and outline implications for reproducibility, evaluation, and deployment.

Published in Transactions on Machine Learning Research (02/2026)
Introducing Background Temperature to Characterise Hid- den Randomness in Large Language Models
Alberto Messina"
alberto. messina@rai. RAT - Radiotelevisione Italiona, Centre for Research, Technological Innovation and Experimentation (CRITS)
Stefano Scotta"
Reffaoo.cootaa@nre Reviewed on OpenReview: https: // openreview. net/ pormfo id= bzOhe4bARF
RAI Radiotelevisione Italiana, Centre for Research, Technological Innovation and Experimentation (CRITS)
Abstract
Even when decoding with temperature T = O, large language models (LLMs) can produce divergent outputs for identical inputs. Recent works align in highlighting implemenlation- level sources of nondeterminism, including batch-size variation, kernel non-invariance, and floating-point non-associativitty In this work, we formalize this behavior by introduc- ing the notion of background temperature Tug, the effective temperature induced by an implementaaiion-deeeedet perturbation process observed even when nominal T 0. We provide clean definitions, show how Tug relates to stochastic perturbation governed by the inference environment I, and propose empirical protocol to estimate Thg via the equivalent temperature (1) of an ideal reference system. We conclude with set of pilot experiments run on a representative pool from the major LLM providers that demonstrate the idea and outline implications for reproducibility, evaluation, and deployment.
1 Introduction
A common assumption in LLM deployment is that setting the decoding temperature to T = 0 (greedy de- coding) ensures determinism. However, empirical evidence shows output variability persists under nominally deterministic settings. The recent work in He & Lab 2025 argues that nondeterminism in LLM inference often arises from practical systems issues such as varying batch sizes and the lack of batch-invariant kernels, along with floating- poiit non-associativvtt and reduction-order effects. Despite growing awareness of this phenomenon, current. approaches to quantifying nondeterminism rely isolated and often incompatible metrics exact-match rates, edit distances, entropy changes, perplexity-basee based measures (see Terven et al. 2222)) and Yun et al. (2025)). While these capture specific facets of instability, they provide fragmented picture and are of difficult usage to compare among models, providers, or hardware environments. This lack of an integrated approach prevents practitioners from reliably characterizing, monitoring and benchmarking the effective randomness of deployed systems.
To address this, we introduce the notion of a background temperature (Tug). This quantity summarizes the effective stochasticity induced by the models' inference environment, even when the configured decoding temperature is zero. By how it is defined, Tog abstracts the specific stochastic sampling implementations of the various models, which allect each model's logits before decoding, instead providing the temperature of an idealized and shared reference system whase output- variaii-ariybilityy statistics best match those observed under real deployment conditions. This approach offers a unified, interpretable scalar that aggregates all sources of implementation noise into a consistent scale. Instead of tracking a bundle of heterogeneous metrics, practitioners obtain a single descriptor of inference instability, comparable across models, providers, and hardware.
"These authors contributed equally to this work.

---

Published in Transactions on Machine Learning Research (02/2026)
The paper is organized by first presentation of an heuristic definition and estimation protocol of background temperature, followed by rigorous mathematical formalization. Finally, we validate the approach through extensive experiments.
Contributions. (i) A concise formal model that addresses the phenomenon of nondeterminism stochastic effect on the output probability; (ii) formal definition of background temperature; (iii) the outline of a practical protocol to estimate it; (iv) set of pilot experiments illustrating and validating the concept.
2
Related Work
2.1 Sources of non-determinism
LLM outputs can vary for several reasons. First, stochastic decoding methods (e.g., temperature, top-k, top-p) intentionally introduce randomness via sampling. Second, floating-poin non-associativity makes large reductions (dot products, softmax, layer norms) sensitive to kernel implementations, parallelism patterns, and hardware, so small numerical differences can be amplified through the network and flip argmax decisions. Third, non batch-invariant kernels combined with dynamic batching cause the numeries for given request to depend batch size and position within the batch, so changing server load leads to different outputs for the "same" prompt. Finally, system-level factors cuntorsssuuhhae such as parallelism configuration, non-deterministic library algorithms, and variation across training runs or checkpoints- introduce additional run-to-run variability even when the model architecture and code are nominally unchanged. The recent work by Thinking Machines Lab He & Lab (2025) provides a systems- firssttttt analysis of LLM nondeterminism, emphasizing batch-size variation and batch-invariant kernels for inference; they also explain how floating-pon non con-associativity and reduction ordering contribute to variability. A key, and sometimes underappreciatedd driver of this variability is the effective batch size seen by the serving stack. Modern inference servers dynamically batch concurrent requests for throughput, and many GPU kernels used in the transformer forward pass particularly matrix multiplications, attention, and normalization layers are not batch-invariant: their floating- point reduction order changes as function of batch size and, in some cases, of the position of a sequence within the batch. Because floating-point arithmetic is non-ossociative, different reduction orders induce small but systematic numerical differences, which can then be amplified by subsequent layers and eventually flip argmax decisions in the decoder, producing different token sequences for the same logical query. In this view, the inference engine "run-to-run deterministic" given a fixed batch configuration, but from the perspective of individual user facing time-varying server load, the stochasticity in batch composition translates directly into apparent nondeterminism in the model's oulputs.
In addition to this work, similarly, several recent studies have quantified non-determinism in large language model outputs even under settings intended to be deterministic (c.g. temperature T = 0, fixed scods). For example:
Atil et al. 2025 systematically evaluate multiple LLMs configured under deterministic settings across zero-shol and few-shot lasks. They observe large accuracy variations (up to 15%) across runs with the same input, and show that even the string outputs are often not identical.
Song et al. (2024) explore how evaluation practices often ignore variability arising from different decoding configurations (greedy vs sampling). They show that even for greedy decoding, evaluation metrics vary, and that alignment methods can help reduce sampling variance.
Ouyang et al. 2025 analyze code generation benchmarks and show that many coding tasks produce dilferent code oulputs across repeated prompt invocations, even when using T = 0. This confirms that deterministic temperature settings do not guarantee output consistency.
Yuan et al. 2025) show that changing system-level factors (GPU type, batch size, numeric precision) alter LLM outputs even under greedy decoding.
2

---

Published in Transactions on Machine Learning Research (02/2026)
2.2 Batch- sichhiinvrrian kernels
Batch-invariant kernels are numerical primilives whose outpuls for given logical example do not depend on the size of the enclosing batch, the example's position within that batch, or the way batches are assembled at serving time. Concretely, a batch-invariant matrix multiplication, RMSNorm, or attention kernel guarantees that running a single example in isolation, or as part of any larger batch, yields bitwise identical activations for that example, assuming fixed hardware and ware versions. Achieving this property requires carefully constraining parallelization and reduction strategies: instead of allowing the compiler or kernel library to switch between different reduction trees (e.g., data-parallel vs. Split-K / stream-K) depending on batch size, batch-invariant kernels enforce fixed, example-local reduction order that is reused across all batch shapes. This often trades some peak throughput for predictable numerics, but in practice the overhoad can be modest when kernels are engineered to maintain sufficient, parallelism without changing reduction topology. Recent LLM serving systems have begun to expose explicit "deterministic" or "batch-invariant" " modes, which swap in such kernels for the critical operations (RMSNorm, matmul, attention) decouple user-visible outputs from dynamic batching and to enable reprodueible inference at scale.
While prior work largely documents the existence and magnitude of non- non-determinism, there remains gap in formalizing this behavior in terms of an equivalent temperature transformation functional and in proposing standard protocols to measure the effective background randomness. Our work addresses this by introducing the notion of an equivalent. temperature (T) and its expectation Tog- In the next sections, we transition from formal definitions to a concrete empirical protocol aimed at estimating an equivalent temperature (1) induced by implementation noise, and ultimately the background temperature.
3 Heuristic Definition and Empirical Estimation
This section introduces Tig heuristically and outlines its estimation protocol. We start by defining the role of temperature in LLMs to establish the intuition behind Tbg. followed by concrete measurement procedure. A rigorous formalization is deferred to Section 4
3.1 Preliminaries and Heuristic Definition
Let D denote the vocabulary of LLM, with cardinality [D]. For any token sequence E Dirl, we write I for its t-th token, and 7<1_ = {T² j < i} for the subsequence of previously generated tokens. At each generation step / E (l,...../71) |r]}, the model produces a vector of logits 2, = {=0.0 : 0 E D} E R PI depending on the context Ttt, These logits are real-valued scores, one for each token E D. By applying the softmax function to 2t, we obtain probability distribution over the vocabulary:
exp(21,0) P.( P.(0 | 75) ΣnED exp(2..))
for all 0 D.
(1)
A temperature parameter T 2 0 is typically introduced to control the concentration of this probability distribution. The temperature acts by modifying the logits prior to the softmax by a scaling factor 1/T. The resulting lemperature-adjusted distribution is
exp(21,/ -0) (-)) E exp(2tn/T)
for all 0 E D.
(2)
For T 0, the distribution concentrates its mass on the most likely token, effectively becoming a degenerate distribution (a Dirac delta) over arg maxge D (0). Conversely, as T + 0o. the distribution flattens and converges to the uniform distribution PT - D). See also Li et al. 2025 for details.
It is convenient to interpret temperature transformation applied to the original distribution. We therefore define a temperature functional
Fy R |PI + IR [D]
(3)
such that, given the base probability vector P. P(() [0, 1I one obtains the adjusted distribution pf (f) by Pf (r.)). In the idealized limit, Fo corresponds to greedy decoding, assigning unit mass
3

---

Published in Transactions on Machine Learning Research (02/2026)
to the most probable token, while Fr with T > produces probability distribution used for sampling the next token in language models. In real LLM deployments, the effective distribution obtained at temperature T= O may exhibit nondeterministic behavior due to implemennatioon-deeeeeett factors. As observed by the authors in He & Lab 2025),, even nominally deterministic docoding (T = 0) can produce variability in the generaled loken sequence. Let denote the inference environment, which includes hardware characteristics, backend implementations, numerical precision, batch composition, concurrency, reduction ordering, and other low-level sources of variation (see He & Lab| (2025) and Shanmugavelu et al. (2024), for more details). Let FT denote the temperature transformation functional realized by the system under environment I. We model these effects as a perturbation of the ideal functional Fr. Specifically, we assume that there exists a perturbation mapping from probability distributions over D to probability distributions over D such that, at =
(P.) = = cP()))
=
(4)
Even if the changes given by 6 are small in magnitude, regions of the probability simplex where two or more tokens have similar mass are highly sensitive: arbitrarily small changes may alter the arg max and therefore change the emitted token at step t.
We interpret the effect of 6 producing distribution that behaves approximately if sampling were performed at a nonzero temperature, even when T = 0 has been selected. We therefore postulate the existence of an equivalent temperature T, (2) such that
(P((P) = (P(P) = (T.)
(5)
The background temperature of an LLM implementation is the expected equivalent temperature over all the possible inference environments I, so, houristically, it is given by
Tog == Ejez [Tn(.)).
(6)
Intuitively, Thg quantifies the implicit randomness present in real systems even when the user explicitly requests deterministic next-token selection. The quantity Tog thus captures implementation-leve nonde- terminism and paves the way for effective measure of intrinsic variability in practical LLM decoding pipelines.
3.2 Estimating TI) and Tug Empirically
The heuristic definition in equation G involves an abstract expectation over inference environments, which cannot be observed directly. To make (I) and Tug empirically measurable, we replace the ideal reference system with a quasi-ideal baseline: a controlled inference setup in which known sources of nondeterminism batch-dependent kernels, floating-point procision issues, concurrency, reduction order, and framowork-spociffc flag- are minimized. Prior works shows that such configurations significantly reduce output variability under zero temperature, making them suitable as reference systems. Running the models locally guarantees a stable inference environment.
The empirical estimation protocol proceeds as follows. First, one selects diverse and representative prompt set 11, including general generation prompts (short or long, with common or rare vocabulary), task bench- marks such as QA (e.g., truthful qa Lin et al. (2022), SQuADPPicee & Cote (2025), TriviaQAJJohhi et al. (2017)), summarization, classification, or code generation, edge or adversarial prompts (e.g., long contexts, rare tokens, top-k ties), and so on.
Second, repeated inference is run at T with the system under test,, while systematically varying the inference environment I along axes known to influence nondeterminism. These include batch structure and co-batching, concurrency and load, hardware/ handware/backend backend types (GPU/CPU, numeric precision, kernel imple- mentations), and numerics such as reduction order, deterministic flags, or fused versus unfused kernels (see He & Lab (2025) Shanmigavelu et al. (2024) Ravi et al. (2025) for details). For remote systems, prolonged and repeated operation can be used to statistically sample the distribution of inference environments.
Third, under the stable baseline environment, the same prompts are run across grid of temperatures establish a mapping between temperatures and output-variability statistics.
4

---

Published in Transactions on Machine Learning Research (02/2026)
Fourth, variability metrics are computed across multiple runs to capture the system's nondeterminism. Metrics can include exact-match rates, first-divergence indices, edit distances, distributional divergences such as KL or JS divergence, and entropy of next-token distributions.
Fifth, the distributions variability metrics are compared between the system under test and the reference system. The temperature of the reference system that minimizes some chosen distance between distributions is taken the equivalent temperature for that environment. Repeating this across prompts and environ- mental samples yields an empirical estimate of (T).
Finally, aggregating the equivalent temperatures across sampled environments and prompts produces an estimate of the system's 's background temperature Tug capturing the intrinsic nondeterminism in real-world LLM deployments. Uncertainly bounds can be derived from the variabilily across prompts, environments, and reference models. This protocol allows measurement of implicit randomness in LLMs without requir- ing access to perfect oracle, while providing guidance for system design and operational mitigation of nondeterminism.
4
Mathematical Formalization
4.1 Rigorous Definitions
In this Section provide rigorous definition of Tug. generalizing it to the case in which the temperature of system under test is not set to 0 but to a generic r >> O. First of all we introduce all the necessary notation that we are going to use.
4.1.1 Notation
We denote by E the set of all possible LLMs Let S denote the set of pairs = (), where m C and a the provider through which m is accessod. Note that if there is no provider meaning that the LLM is run locally provider is the set I of all possible inference environments in which a system S can be executed. The temperatures used in the LLM configurations are denoted by T E [0, Tmax], where Tiax is finite arbitrarily large value in Rt. We denote by II the set of all possible prompts.
we simply denote (m, 0) m. Closely related to the models accessed through a
Then we introduce the concept of variability measures, which are functions that take as input a set of responses produced by an LLM and return a scalar (or vector) value that quantifics the variability among them. The of all variability measures is denoted by V. The empirical distribution of the values of E V computed for each prompt in IT over N answers generated by E L at temperature T E [0, Tmxx] is denoted by (t) (v, II). Analogously, the empirical distribution of the values of V computed for each prompt in II over the N answers generated by s E S at temperature T E [0, mmax] in the inference environment E I is denoted by (f, 1) = of (v. it, f). Note that both f4((v) and of (v) are elements of rod(y)) where cod(v) denotes the codomain of Lastly, for cach e V, denote by d, function that evaluates the distance (or similarity) between variability distributions computed with For example, if v has codomain N de could be the Kolmogorov-Smirnov distance (see Wilcox (010)), which quantifies the distance between two distributions.
4.1.2 Definitions
Hereinafter, let us assume that all the sets IT, D. I, E and S are finite and discrete. This could seem restrictive in general, but in practice, is trivially true. First, we define precisely the equivalent temperature T, introduced in Section 3.1
Definition 4.1 Consider the probability space (Q, F.P), where Ω = =xIVxxX_ J 20, and p is the uniform probability measure on Q. For any & L, S, E V, I I, T E [0, T max], and chosen distance function
5

---

Published in Transactions on Machine Learning Research (02/2026)
du, we define the equivalent temperature TV T (I, u) the volue of the random variable
Tg, :: - [0, Imax]
(1 () + T2 (1,0) [M (1,
Σ
t
(7)
(E,
where
(I, argmin d, (ff(()), (2 Te(0,Tmaxl
(8)
and the set of minimizers (1, dol (o(f(((), (v, is continuous on the compact interval [0, Tmax)).
v) is assumed be nonempty (which holds, for instonce, if +
In words, T2, v) is defined as the mean of all temperatures in [max] that minimize the distance between the variability distributions { II) and (v, II, I). Given the assumptions it is casy to prove that Ts,- is measurable random variable. Now, can finally define rigorously the background temperature of system.
Definition 4.2 The background temperature Tgg(s) of a system s E S run at temperature E [0, Tmaxl is the value of the funclion
Tog S [0, Tmax] - [0, maxl
(9)
(s, 7 (h((s) == EpITs, (I,
the point (s, T). Where the expectation is taken with respect to the probability meosure p introduced in Definition 4.1
Definition 4.2 is quite general and takes into account the possibility estimating background temperature for systems run at non temperaturos, this goes beyond the intuitive idea given Section 3.1 but it generalize that concept and will be useful for next Sections.
4.2 Estimation Procedure
Clearly, when want to estimate the background temperature for system S, working with the sets used in Section 4.1 is practically impossible. To make estimation tractable, we restrict ourselves to finite subsets of the original domains: a reduced set of reference LLMs L C L; a subset of prompts II C II; a limited set I of inference environments; a finite discrete subset of temperatures C [0, mmax]: selection of variability measures C Thus, the distributions f+( and (v), have to be intended as ff II) and (v, II, I), for T.r O, f E C, ve
Then, adapting the definitions in Section 4.1 the estimated equivalent temperature is defined
(1,e) MM (I, (1.0)
Σ t, where M (1,0) = argmin d, (ff (),, (10)
Tee
That is, T (I, is the mean of all discrete temperatures in ② that minimize the distance between the empirical variability distributions (v) and (v, I). Consequently, the estimated background temperature is given by the empirical mean
(s)(
EEΣ VEI
Ne. (T,
(11)
I I V
Clearly, the larger the sets C, and V, the more robust the estimate Th (s) will be.
The focus through this work will be on estimating the value of 70 bi (s) for different systems but we will also the estimates Th (s) for generic temperatures for validating the pipeline.

---

Published in Transactions on Machine Learning Research (02/2026)
5 Experiments setting
The experiments were conducted using the following scttings (see notation in Sections 4.1.1 and 4.2
= II", consisting of the first n prompts in TruthfulQA Lin et al. (002 dataset. Each prompt was used = 100 and N = 32 times for cach system under test and reference model, respectivcly, limiting the answers to 32 tokens.
= (0,0.01,.... 0.19, 0.2, 0.25, 0.45, 0.5,0.5,0.8 o............................... 0.9, 1}.
The reference LLMs considered are C (smoll, llama, mistral}, where smoll, llama and mistral 1eea--3..-----ntttiii and Mistral-
stand for, respoctively, SmolLM3-B8 3 Bakouch et al. (2025
7B-Instruct-vO.3 (Jiang et al. (2023)
In most of the experiments, considered single inference environment I, corresponding to the conditions under which the experiment was run. Unless otherwise specified, the set I therefore consists of this single element
The distance that we use between the distributions of the variability metric values is always the Kolmogorov Smirnov distance and we denote it by dxs. This choice is appropriate because it quantifics discrepancics between onc-dimensional distributions, and all variability metrics in our set produce one-dimensional real values (see Section 5.1
In the following Section, we define rigorously the variability metries, i.e. the set V, that we used. The set S of the systems under lest will be specified for each experiment.
5.1 Variability Measures
We used the following variability measures to quantify the variability within the N answers given by the LLMs each prompt. See Figure | for more in-depth analysis them. Throughout this section let denote for each prompt I1, the generated answers by Ar = (3,).
5.1.1 Maximum Exact Match Fraction
Let Pa be the frequency the most common answer in Ar. The maximum exact match fraction for the answers given π E II defined as
MEMF( = PT N
(12)
which measures how often the model produces exactly the same answer. MEMF the case which all the answers are identical, and it MEMF 1/N if all the answers are unique.
5.1.2 Average Normalized Levenshtein Distance
Let Lev(a, b) denote the Levenshtein distance between two lists tokens, i.e. the number of elements to change, add or remove from a make identical The average normalized Levenshtein distance for the N answers given to Il is defined
Lev a
AvgLev(T)
Σ
(13)
Tomaz
2
ISKKIEN
where Lmaz is a normalization factor. Note that in our context Lmaz 32, indeed, being the answers capped to 32 tokens, the maximum value of Lev(aç, az); for any i,j E (1,..., N} and क E II, is 32. goes from 0, when all answers are identical to where they are all completely different.
Instruct

---

Published in Transactions on Machine Learning Research (02/2026)
5.1.3 Average Longest Common Subsequence Distance
Let LCS(a, b) denote the length of the Longest Common Subsequence between two sequences of tokens a and b. We define the corresponding normalized distance as
LCS(a, b)
DistLCS(a,b)) == min [a), [6])
(14)
Thus, DistL b) = 0 for identical answers and approaches 1 when the two answers share no common subsequence. For cach prompt II, the average LCS distance is defined
AvgI.CS(m)
Σ
(aF,
(15)
(2)
ISiRiSN
1.0
SmolLM3-38
Llama-3.233B-instiie AvgiCS
SmolLM3-38 Avglov
Mistral-78 wwt.------ttrut-o..3333.
SmolLMB-33 AvgLCS
Mistral-78-instrut--00 AvgLev
3.2. 38-Instruct-
Mistral-78-Instreee ca-innstt-sst AvgLCS
0.8
Laama332--88-nnste AugLev
0.6
metric
0.2
0.8
Temperature
Figure 1: Mean values of the metries defined below function of the temperature, computed over the answers produced by the reference models L the prompts in I7 each sampled 32 times.
6 Experiments and Results
In this section, we present some experiments to validate the theory presented in this work. The pipeline described Scction 4.2 under the experimental sctting in Scction implemented in htttps://gite
6.1 Background Temperature Estimate Across Models and Providers
In this experiment, we estimate the background temperature using the pipeline and the settings presented in Sections 4.2 and 5 for a set of different systems E S run at zero temperature. The goal is determine whether different LLMs, accessed through different providers, exhibit significantly different variability in their answers, and whether the background temperature can capture this property. We used the same prompt set I130 for all the systems considered, and also tested whether significant changes appear when increasing the prompt set in the case (gpt-4.1-nane Azure), in which we also considered 17200 to see if this change modifies significantly the estimates. All results are reported in Table
Estimated background temperatures T (s) vary systematically across systems 8 E S, spanning an order of magnitude from 0 (claudo-sonnet-4 on AWS) 0.288 (DeepSeck-VV V3.1 on Azure). This pattern of variability indicates that the background temperature is not determined solely by the underlying model architecture, but instead emerges from the specific model provider pair. The of DeepSeek-V33 V3.1 makes this especially
8

---

Published in Transactions on Machine Learning Research (02/2026)
Table 1: Values of (s) where s the combination the first two columns. The column "Region" gives additional information about the inference environment considered.
LLM
Provider
Region
7 (s)
Prompt Sct
claude-sonnet-4
AWS
eu-west-3
0.000
1130
grok-3-mini
Azure
swedencentral
0.016
1730
mistral-large-2422-vi.
AWS
us-east-1
0.016
-130
gemini-2.0-flash
Google
0.048
I1:00
gpt-4.1-nano
Azure
eastus2
0.068
1
gpt-4.1-nano
Azure
eastus2
0.087
11:00
DeepSeek-V3-0322
Azure
swedencentral
0.148
I
DeepSeek-V3.. V3.1
AWS
us- we-west-2
0.186
I1
DeepSeck-V3.11
Azure
swedencentral
0.288
I1300
clear: the model deployed on AWS showed Thg (s) = 0.186, whereas the same model on Azure increases 0.288, despite the model and the experimental protocol being identical in both settings. Another interesting observation is that claude-sonnet-4 for each prompt, it produced the same sequence of tokens across all N = 100 iterations, resulting in a background temperature of O. While this does not guarantee determinism for arbitrary prompt sets or token limits, it suggests that the configuration 8 (claude-somnet-4, AWS) represented the most stable system among those tested. Overall, these results indicate that The (s) captures a consistent signature variability that depends on the specific model-provider combination, rather than on the LLM model alone.
AWS behaved perfectly deterministically over the tested prompt set:
6.2 Validation of Background Temperature via Configured Sampling Temperature
To further validate the background temperature as a consistent measure of LLM response variability, we evaluated 7% (s) for multiple values of the configured sampling lemperature across different systems and bg
providers, keeping all other inference settings fixed. We also included an on-premises model, gemma-3-1b-i TTaa 2025 , providing a fully controlled environment that serves as a baseline for comparison against
cloud-hosted systems. The tested systems and the obtained values are reported in Table 2
Table 2: Values of T%g (s) for different values of 7 and systems E S.
LLM
Provider
Region
TV (s)
IT
gpt-4.1-nano
Azure
eastus2
0.00
0.068
I 200
gpt-4.1-namd
Azure
eastus2
0.05
0.081
2000
gpt-4.1-nano
Azure
eastus2
0.10
0.101
200
grok-3-mini
Azure
swedencentral
0.00
0.016
30
grok-3-mini
Azure
swedencentral
0.05
0.032
3
grok-3-mini
Azure
swedencentral
0.10
0.066
30
claude-sonnet-4
AWS
eu-west-3
0.00
0.000
31
claude-sonnet-4
AWS
eu-west-3
0.05
0.018
30
claude-sonnet-4
AWS
eu-west-3
0.10
0.022
30
gemma-3-1b-it
On-premises
0.00
0.000
II 200
gemma-3-1b-s
On-premises
0.05
0.032
II 200
gemma-3-1b-in
On-premises
0.10
0.061
T 200
gemma-3-lb-it
On-premises
0.15
0.093
200
The estimated background temperatures generally do not match exactly the configured sampling tempera- tures, even in fully controlled on- premises settings. However, Ty (s) consistently increases with 7 across all
http:://wugginngaaacccc//oooooo/ 3-1b-it
9

---

Published in Transactions on Machine Learning Research (02/2026)
DeepSeek-V3-0324,
Azure Azure
gpt-4.1-nano Azure
claude-sonnet-4_AN _WW
mistraallarge22402-1100000 AWS
0.150
0.125
0.100
0.075
0.050
0.025
16
18
22
Approximate Time (UTC)
Figure 2: Evolution of Tyg (s) for different & E S (see Table [ for the regions and prompt sets) over the course of a day, highlighting the effect of temporal changes in the inference euvironment.
systems, demonstrating a monotonic relationship:
T2 Th (Tg S.
(16)
For example, claude-sonnet-4 on AWS consistently exhibits the lowest background temperature among cloud- hosted models, confirming the trends observed in the aero-temperature experiments. These results indicate that (s) provides a meaningful and interpretable estimate of the effective stochasticity outputs. Importantly,
the model captures the variability of responses independently of the configured sampling temperature T, which can be inffuenced by provider-specific implementations and the inference environment. Therefore, (s) can be considered more uniform and intrinsic measure of output variability, allowing for meaningful comparisons across models and deployment environments regardless of the temperature setting.
6.3 Temporal Variability of Th
To investigate the stability of background temperature estimates over time, hypothesizing that the time is relevant component of the inference environment, we repeated the estimation procedure for selection of LLMs from different providers at multiple times throughout the day (Figure 2 All models exhibited some temporal fluctuations in Th, (s), except for claude-sonnet-4 accessed via AWS, which remained perfectly stable throughout the day, confirming the results discussod in Section 6.1 Despite these variations, the trajectories of different systems did not intersect, indicating that temporal fluctuations other inference-envirommerr factors. To clarify whether these temporal fluctuations variability of the models to changes in the inference environment, we performed five concurrent estimations of To (s) using gpt-4. 1-nano on Microsoft Azure, keeping all conditions constant (same provider, model, execution time) and using the prompt set 30 The fluctuations observed across these concurrent were
smaller than those induced
due to intrinsic
Table 3: The statistics for 1pt-4.1-nano via Azure (swedencentral). "Same-time" refers to five contemporary estimates, "Day-long" refers the experiment in Section 6.3 and Figure 2
Experiment Mcan m
Std. Dev.
a a/m
Same-time 0.0985
0.0038 0.0085
0.039
Day-long 0.0650
0.131
minimal (Table 3 and considerably smaller than those observed in the non-simultancous measurements.
This demonstrates that the temporal variations seen the first experiment are primarily driven by changes in the inference environment time, rather than by intrinsic randomness of the models under identical
10

---

Published in Transactions on Machine Learning Research (02/2026)
conditions. In general, these results show that temporal aspects relevant components of the inference environment, significantly changing the estimates To (x); i.e. influencing the stability of the answers given by the model.
6.4 Validation of Tig(s) as a Measure of Answer Variability
To further assess whether the estimated background temperature (s)(s) provides a reliable measure of the variability exhibited by system S at a given sampling temperature т, we evaluated the relation- ship between bg and the empirical variability of model outputs
dataset. For this purpose, we considered a prompt set il consisting of the first 100 instructions from the test split of the Hugging- FaceH44/testing self instruct Sattrrcct_sma[ill__ dataset. We selected three systems whose estimated background tem- peratures (sec Table ( allowed for controlled comparison: gpt-4.1-nam deployed on Azure with O, gemma-3-1b-it running on-premises with 0.10, which share similar values of Thg (+) (respectively, 0.068 and 0.061); and gemma-3-1b-l on-premises with 0.15, whose estimated background temperature (0.093) is noticeably higher. This setup enables testing the hypothesis that systems with similar Tig (s) exhibit similar distributions of output variability on given prompt set (different from the one used to estimate Thy (8)), independently of model architecture and deployment environment. Thus, we analyzed the distribu- tions of response variability (computed with the measures in Section 5.1 over the prompt set. if, using each prompt 100 times. Pairwise differences between the variability distributions were quantified using the Kol- mogorov-Smirnor distance dxs (Figure 3 across all variability metrics v E V. Additionally, the Wilcoxon
des MEMF
des AvgLev
dys AvgLCS
gpl
0.09
0.16
gemimdoo.s 15 gemimboo. 10
0.1.2
0:12
0.11
0.14
9pl
gemmaa.o gemmag.15
gpt.
gemmaz.. gemmay.1
gpt.
gemmau1s gemmaa.1
Figure 3: dks between the variability distributions the answers given to the prompts in I1, by the models gpt- 4.1-nano, via Azure at temperature 0, denoted by gptp, and gemma-3-lb-it, on on-premises at temperatures 0.10 and 0.15, denoted respectively by gemma and gemma,.10
signed-rank test (see Wilcox 2010 was applied to assess whether the distributions corresponding to the two models with similar Tg can be considered statistically analogous, while the third model expected to exhibit different distribution. The results, sunmarized in Figure 3 and Table g clearly indicate that models with similar background temperatures produce variability distributions that are significantly closer to each other than to those of the higher-temperatur system. Across all metrics, the Wilcoxon test fails to reject the null hypothesis analogous distributions for the pair with matching TV, (s), whereas comparisons involving the third model consistently reject. it.
Overall, these findings confirm that (s)))) effectively captures the intrinsic variability of responses induced by given sampling temperature. Morcover, the consistency of the results across datasets and metrics demonstrates that background temperature is robust and reliable proxy for measuring answer variability in LLMs.
11

---

Published in Transactions on Machine Learning Research (02/2026)
Table 4: Wilcoxon signed-rank test results for variability comparisons between model pairs across for E V. The model are denoted as follows: gpt-1.1-nano, via Azure at temperature 0, by gPky, and gemma---Ib-it, on-premises at temperatures 0.10 and 0.15, denoted, respectively, by gemmaa and geumman.15 The bold p-values are the cases in which we can not reject the null hypothesis Ho that the distributions are analogous, in the other cases reject Ho.
Metric
Model Pair
Wilcoxon Stat
p-value
MEMF
gPto vs 10
1090.0
0.05342
MEMF
gPto vs gemmåg.15
727.5
0.00008
MEMF
gemma vs gemma0
50.5
0.00000
AvgLev
KPLo vs gemma0,10
1398.0
0.73647
AvgLev
gPtp vs gemma0.15
1097.0
0.03999
AvgLev
g vs gemma8015
33.0
0.00000
AvgLCS
gPLg vs gemm180
1276.0
0.54805
AvgLCS
gPtp vs gemma0.15
939.0
0.01568
AvgLCS
gemma 10 vs gemmag. 15
24.0
0.00000
7 Discussion and Conclusion
In this work, we introduced background temperature (Tbg) quantitative lens implementation- dependent variability in Large Langnage Models. Even under nominally deterministic decoding, different in- ference enviromments induce statistically measurable randomness in the generated outputs. Our contribution is twofold: (i) we formalized Tog as an equivalent temperature that best matches observed variability against a controlled reference, and (ii) we proposed and empirically demonstrated a practical protocol for estimating it through variability metrics computed over repeated generations. Across multiple models, providers, and configured temperatures, our pilot studies show that Thg captures systematic and interpretable differences. First, background temperature varies substantially across model provider combinations, rather than merely across model architectures. For example, the same model deployed via different cloud providers can differ by an order of magnitude, suggesting that infrastruct plays a central role in inference stability. Second, Thg exhibits a monotonic relationship with the sampling temperature T, indicating that it reflects effective output stochasticity beyond the configured temperature parameter. Third, temporal experiments show that Tog is sensitive to time-dependent factors, reinforcing that inference environment includes load patterns and low-level system behavior, not only model and APT configuration. Finally, systems with similar estimated background temperature produce statistically similar distributions of output variability unscen prompts, suggesting that Tog can serve as stable proxy for characterizing response variability. Thus, can conclude that using background temperature, rather than relying on ad hoc observations of nondeterministic outputs, provides a scalar quantity that practitioners can report, monitor, and compare across deployments. It is essential to emphasize that background temperature must not be interpreted as the model's actual sampling temperature. Rather, it is a single scalar that summarizes the net effect of implenentation-induees noise by matching the variability statistics of the tested system to those of controlled reference runs. While alternative metrics such perplexity, diversity scores, edit-distance statistics each capture only fragment of model variability, background temperature offers unified and interpretable measure that aggregates all sources of randomness hardware behavior, software kernels, concurrency effects, and numerical precision into one coherent quantity. The methodology introduced here also comes with important limitations. A primary challenge is the lack of a truly ideal reference configuration: current estimates require comparisons against quasi-ideal, stable setups, which inevitably introduce model-dependent effects. While averaging over multi- ple reference systems helps reduce this influence, future work should focus on identifying well-defined, highly stable reference models and documenting their characteristics. Scaling the approach is another open issue. Although we employed multiple variability metrics, our experiments rely on relatively small prompt sets, and broader, domain-diverse prompt collections will be required for more robust estimates. In addition, back- ground lemperature values depend on several design choices, including the distance metric, the temperature grid, and prompt selection. Establishing shared conventions for these components - particularly through a standardized set of stable reference models and common evaluation scttings would allow background temperature to become a practical and genuinely comparable method for quantifying output variability in LLM systems.
12

---

Published in Transactions on Machine Learning Research (02/2026)
References
Berk Atil, Sarp Aykent, Alexa Chittams, Lisheng Fu, Rebecca J. Passonneau, Evan Radcliffe, Guru Rajan Rajagopal, Adam Sloan, Tomasz Tudrej, Ferhan Ture, Zhe Wu, Lixinyu Xu, and Breck Baldwin. Non- determinism of 'deterministic' Ilm settings, 2025. URL http://///axxx...//aaa//////
04667
Elic Bakouch, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Lewis Tunstall, Carlos Miguel Patíño, Edward Beeching, Aymeric Roucher, Aksel Joonas Reedi, Quentin Gallouédee, Kashif Rasul, Nathan Habib, Clémentine Fourrier, Hynek Kydlicek, Guilherme Penedo, Hugo Larcher, Mathieu Morlon, Vaib- hav Srivastav, Joshua Lochner, Xuan-Son Nguyen, Colin Raffel, Leandro von Werra, and Thomas Wolf. SmolLM3: smol, multilingual, long-context reasoner. https https://hugge
smolmm3 2025.
Horace He and Thinking Machines Lab. ence. Thinking Machines Lab blog, 2025. defeating nondeterminism-ine 11m- inference/ Accessed: 2025-09-15.
Defeating nondeterminnss in 1lm infer- URL https://tbhinkiigaachhee ai/blog/
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL https://aaxii.... org/abgg///.o..ooo ab10
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1601-1611, Vancouver, Canada, July 2017. Association for Computational Linguistics. doi: 10.1663//1/P117-1147 URL https://aclanthool/ org/P17-1147/
Lujun Li, Lama Sleem, Niccolo' Gentile, Geoffrey Nichil, and Radu State. Exploring the impact of tem- perature on large language models: Hot or cold? Procedia Computer Science, 264:242-22 251, 2025. ISSN 1877-0509. doi:
URL httpp://www.sceeecelreeee com/
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods, 2022. URL
Shuyin Ouyang, Jie M. Zhang, Mark Harman, and Meng Wang. An empirical study of the non-determinism of chatgpt in code generation. ACM Transactions on Software Engineering and Methodology, 34(2):1 28, Jamuary 2025. ISSN 1557-7392. doi: 10.1145/3697010. URL http://dx. http:://dxxdd...rgg///// doi. 1tt///10.111//8880 104111141130000
S. Price and D. L. Cote. Document, analysis with lmss: Assessing performance, bias, and nondeterminism in decision making. In ICPRAM 2025: Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods, pp. 207 214, 2025. ISBN: 978-959-758-730-6
Nikita Ravi, Abhinav Goel, James C. Davis, and George K. Thiruvathukal. Improving the reproducibil- ity of deep learning software: An initial investigation through a case study analysis. arXiv preprint, arXiv:2505.03116 2025. URL htttp::/aarxiv.orgee
03165). Accessed: 2025-09-15.
Sanjif Shanmugavelu, Mathieu Taillefumier, Christopher Culver, Oscar Hernandez, Mark Coletti, and Ada Sedova. Impacts of floating-point non-associativity on reproducibility for hpc and deep learning appli- cations. arXiv preprint, arXiv:2408.05148, 2024. URL https://arxiv. wwtpaaawwwv.....graaaaa orggabs////e aa//////./222..../// /200 05148 2025-09-15.
Accessed:
Yifan Song, Guoyin Wang, Sujian Li, and Bill Yuchen Lin. The good, the bad, and the greedy: Evaluation of lims should not ignore non-determinism, 2024. URL
org. ab2g//b2s/400..... 10457
Gemma Team. Gemma 3 technical report, 2025. URL https://arxiv.org/aaaa
13

---

Published in Transactions on Machine Learning Research (02/2026)
Juan Terven, Diana-Margarita Cordova-Esparza, Julio-Alejandro Romero-Gorte Gonzzll,,, Alfonso Ramírez-
Pedraza, and F...A Chávez-Urbiola. A comprehensive survey of loss functions and metrics in deep learning.
Artificial Intelligence Review, 58, 2025. ISSN 1573-7462. doi: https://doi.org
0000000/010022221-1111111 1007 w11111/00///00///. 025- 11198- 7#citeas
URL
www.://..rrnnnn.ooooooooooaaeeee com/ artaiaatiileea https//nr.../artaiiee////ee/
Rand R. Wilcox. Fundamentals of Modern Statistical Methods. Springer New York, NY, 2010. doi: https: // //doi.org/ 10tp:/1110000//8-144419-9----039
Jiayi Yuan, Hao Li, Xinheng Ding, Wenya Xic, Yu-Jhe Li, Wentian Zhao, Kun Wan, Jing Shi, Xia Hu, and Zirui Liu. Understanding and mitigating numerical sources of nondeterminism in 1hm inference, 2025. URL
Longfei Yun, Chenyang An, Zilong Wang, Letian Peng, and Jingbo Shang. The price of format: Diversity collapse in LLMs. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Findings of the Association for Compulational Lònguistics: EMNLP 2025, pp. 15454 15468, Suzhou, China, nov 2025, Association for Computational Linguistics. doi: 1..1865//v12022 findings findings-emmlp.8866
URL httpe://cclaataoooggoror///2222/ http:///22.....iin..ee.e. emnlp.836)
14


Reviews:
[{"review_id": "KshoYY9Tps", "reviewer": "Reviewer_9HtF", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "To quantify intrinsic variability in LLM outputs — even under greedy decoding, where responses are expected to be deterministic — the authors introduce the notion of background temperature  Tbg and propose proxies for estimating it using a reference LLM. Although identifying a stable and reliable reference model poses challenges, I believe their analysis represents a meaningful step toward establishing comparable metrics for LLM certification.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The metrics and experimental setups are generally well presented. The authors’ main contribution is the introduction of the background temperature, which is evaluated across several use cases. While I would have appreciated deeper analysis or more substantive conclusions from the comparisons, it is fair to say that the authors deliver on what they claim: they introduce the metric and report results accordingly.", "audience_yn": "Yes", "audience_explanation": "LLM certification is a deep and impactful area where the presented results can be of interest.", "requested_changes": "- Out of the comparison of the metric in Table 1, can we draw certain conclusions about how the LLM architecture/training influences the background temperature?\n- Can the authors elaborate in more detail why the batch size influences LLM output distribution in model validation/test? This is referered to a source of variability in LLM outputs but I don't see how it may influence. Similarly, I miss some introduction to batch-invariant kernels. I believe the paper would really improve if a new section summarizing sources of variability in LLMs is introduced, detailing all these aspects.\n- Please elaborate some conclusions out of the comparison in Table 1. How the LLM structure/architecture/training may influence the background temperature.", "broader_impact_concerns": "I do not have any specific concern in this line.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Z5ngMIozTv", "reviewer": "Reviewer_YqG6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces the concept of \"background temperature\" ($T_{bg}$) to formalize and quantify the nondeterminism observed in Large Language Models (LLMs) even at a nominal decoding temperature of $T=0$. The authors propose that implementation-level factors (e.g., floating-point arithmetic, kernel choices) induce a stochastic perturbation on output probabilities, which is equivalent to decoding from an ideal, deterministic model at some non-zero temperature. The key contributions are the formal definition of this background temperature, a detailed empirical protocol for its estimation using a reference model, and a set of pilot experiments applying this protocol to several proprietary LLMs.", "claims_and_evidence_yn": "No", "claims_explanation": "The central claims of the paper are not supported by sufficiently convincing evidence due to a combination of significant methodological limitations, a lack of theoretical justification, an insufficient scale of experimentation, and issues with the presentation of the evidence.\n\nFirst, the empirical protocol makes a key methodological choice that, while pragmatic, significantly complicates the interpretation of the results. The authors propose to estimate the background temperature of a target model (System B, e.g., `gpt-4.1-nano`) by comparing its output variability at $T=0$ to the variability of a completely different reference model (System A, e.g., `SmolLM3-3B`). The authors acknowledge the difficulty of obtaining a \"quasi-ideal\" reference of the same model. However, the chosen alternative of using a different model introduces a significant confound. This makes it difficult to isolate the implementation-dependent nondeterminism of System B, as the final measurement reflects a combined effect of: (1) the actual system noise of B, (2) the intrinsic differences between the models' output distributions, and (3) the differing sensitivities of each model's architecture to temperature scaling. The authors' own results illustrate this challenge: the estimated $T_{bg}$ for `gpt-4.1-nano` changes from 0.05 to 0.10 simply by switching the reference model. The proposed solution of averaging these results is a heuristic that highlights the sensitivity of the measurement to the reference choice, rather than resolving the ambiguity. This makes it challenging to interpret the final number as an intrinsic property of the system under test.\n\nSecond, the paper lacks theoretical grounding for its central claims. The core modeling assumption, stated in Equation 7 ($F_I^0(P) \\approx \\epsilon_I(P) \\approx F_{T_n(I)}(P)$), posits that implementation-dependent noise acts *as if* it were a temperature transformation on the logits. This is a strong claim presented as an axiom without any theoretical proof or derivation. It is entirely possible that system-level perturbations induce changes to the probability distribution that are not well-approximated by a simple temperature scaling. The claim in Section 6.5 that the estimator $\\hat{T}_bg$ will converge to the true $T_{bg}$ is also stated without proof, leaving it as an assertion.\n\nThird, the pilot experiments are somewhat small in scale and narrow in scope to support the paper's broad conclusions. The primary experiment uses 200 prompts, while subsequent experiments use only 30. Such sample sizes may not be sufficient to yield robust estimates. Moreover, the evaluation relies exclusively on a single, coarse metric: the exact-match fraction. This metric is useful but cannot capture more subtle distributional shifts. The paper proposes a richer set of metrics in Section 6.4 but fails to employ them, missing an opportunity to provide more convincing evidence.\n\nFinally, the presentation of the evidence could be improved. The paper's layout, particularly with pages 8 and 9 consisting entirely of figures, disrupts the narrative flow. While visual aids are crucial, integrating them more closely with the relevant text would improve readability. This current layout separates the results from their description and could suggest that the supporting textual analysis is less substantial than the visual presentation.", "audience_yn": "Yes", "audience_explanation": "The problem of nondeterminism in LLMs, even in supposedly deterministic settings, is a significant practical issue for reproducibility, evaluation, and deployment in high-stakes applications. Practitioners and researchers in the TMLR audience are acutely aware of this challenge. The paper's proposal to frame this issue in terms of a \"background temperature\" is novel, intuitive, and provides a potentially powerful conceptual tool for reasoning about and measuring this hidden randomness. While the current execution has some notable limitations, the core idea and the problem it addresses are highly relevant and would certainly be of interest to the community. The extensive discussion of factors causing nondeterminism and the outline of a measurement protocol, even with its current issues, will spark valuable discussion and could inspire more rigorous future work.", "requested_changes": "**Critical Changes:**\n\n1.  The experimental methodology should be revised to address the significant confound introduced by using a different model as the reference. The current approach makes it difficult to isolate the variable of interest. A critical revision would be to re-run experiments using a protocol that compares the system under test to a \"quasi-ideal\" configuration of the *same* model, as described in the first paragraph of Section 6. If this is not feasible for remote APIs, the paper must more thoroughly discuss this limitation and be much more cautious in its claims, framing the results as a model-pair-dependent metric rather than an intrinsic property of the test model.\n2.  The mathematical formalism needs to be tightened. The definitions in Sections 4 and 5 rely on imprecise \"$\\approx$\" symbols (e.g., Equation 7). The concept of an \"equivalent temperature\" should be defined more rigorously, for example, by formally stating that it is the temperature that minimizes a specific divergence metric, which connects the theory more directly to the empirical protocol.\n3.  The definition of background temperature in Equation 8, $T_{bg} := \\mathbb{E}_{I \\in \\mathcal{I}}[T_n(I)]$, lacks rigor. The expectation is taken over a set $\\mathcal{I}$ without defining a probability measure on it. Please specify the distribution over which the expectation is taken or clarify how this expectation should be interpreted.\n\n**Strengthening Changes:**\n\n1.  The pilot experiments are limited to a single metric (exact-match fraction). The paper would be much stronger if the experiments were expanded to include some of the other metrics proposed in Section 6.4 (e.g., distributional divergence, edit distance). This would help assess whether the $T_{bg}$ estimate is robust to the choice of metric.\n2.  The layout of the paper should be improved to better integrate figures with the text. Having two full pages (8 and 9) dedicated only to figures disrupts the reading experience and separates the evidence from its interpretation.\n3.  The finding that `claude-sonnet-4` has an estimated $T_{bg}=0$ is very interesting and warrants more discussion. Does this truly imply it is deterministic, or is it a limitation of the measurement protocol or the specific prompts and metric used? Exploring this further would add significant value.", "broader_impact_concerns": "The work focuses on the technical problem of reproducibility and does not appear to have direct negative broader impact concerns. If this method were to be used to \"certify\" models as having low background temperature, it would be critical that the measurement protocol be sound and standardized, as a flawed protocol could be used to mislead users about the reliability of a model. However, this is an issue of correct application rather than an inherent ethical problem with the research itself. The paper is transparent in its methods, and no Broader Impact Statement seems necessary at this stage.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "SxlNXs2d1W", "reviewer": "Reviewer_1Q9j", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "DISCLAIMER: This reviewer is not an expert on LLMs.\n\n**Summary**  \nThis paper proposes to quantify the non-determinism of LLM even at sampling temperature $T=0$ via the notion of background temperature. This is supposed to be the temperature at which the outputs of an ideal LLM, deterministic at $T=0$, would match the outputs of the LLM in question. The paper further proposes  a pipeline for estimating this background temperature and runs pilot experiments.\n\n**Strengths**  \n- S1: The phenomenon of non-determinism is interesting and very relevant to trust and safety concerns around LLMs.  \n- S2: Aiming to quantify this non-determinism is interesting and the proposed method of computing a background temperature seems novel. \n\n\n**Weaknesses**  \n\n\nThis paper seems to be in a *very* preliminary state. Experiments are not thoroughly conducted, the presentation is suboptimal , and the paper structure is very unusual to say the least. \n\n*Experiments:*  \nIn Section 6 the authors give a blueprint of a possible pipeline for computing background temperature. But the supporting experiments are very minimal and fall short of many desiderata in section 6. Only 2 reference models are used, only one variability metric, exact-match rate, only 200 non-diverse prompts (all from truthful_qa) are used. Overall, 4 LLM models are investigated, but their findings do not get compared or discussed (e.g. the claude model was even perfectly deterministic). Moreover, the few conducted experiments are not even consistent: The ChatGPT experiment used 200 prompts, but for gemini, grok and claude only 30 prompts were used.  \n\nBefore comparing different remote and reference models, it would be good to first verify the plausibility of the proposed pipeline. For instance, one could take a model, e.g. llama, that can be run deterministically for $T=0$, and use it both as reference and remote model (in the remote setting with some temperature $T>0$). This way one could check if the number of prompts, the variability metrics, the number of responses etc are suitable to reliably recover the true temperature under which the remote model was run. If successful, one could take this setup one step further and put the remote variant in a non-deterministic environment and check how the estimation of the true temperature changes (both when remote is run at $T=0$ and $T>0$). Such baseline experiments seem important to then trust background temperate estimations across different model types. \n\nIn sum, the experiments do not support the pipeline suggestions and are also too limited for meaningful scientific contribution. \n\n\n*Conceptual issues:*   \nWhy can we model the temperature of one LLM by that of another / a set of other reference LLMs? Why is this even meaningful? Are even the vocabularies the same?\n\nWhat is the upshot of using background temperature (with all the non-canonical choices and references involved) when one could just compute perplexity or other diversity measures of the outputs of the remote LLMs at $T=0$?\n\n*Presentation:*   \nThe paper uses lots of notation that is often hard to follow and sometimes not explained. For instance, how are $\\hat{z}$ in Eq (3) defined? What is  $n$ in eq (7)? Why is $\\hat{z}$ indexed by the generation step $i$ and not the token index $t$ in Eq (3)? Why is the deterministic decoding in eq (2) equivalent to non-deterministic decoding with other logits $\\hat{z}$ eq (3)?\n\nMoreover, some parts of the proposed pipeline remain rather vague. E.g. how should one engineer prompts so as to elicit near-ties among the top-k tokens?\n\n*Other unusual aspects*:    \nThe entire paper only has 10 references. The introduction is extremely short, shorter even than the abstract. It does not contain any motivation for the relevance of the work, no examples, and only minimal contextualization. Sections 4,5 are (like the introduction) only about 10 lines long (or a quarter of a page), but sections 6 and 7 are 3 and 5 pages long, respectively. This seems very unbalanced. The figures are huge compared to their information content. There are several odd stylistic choices, such as starting a sentence with \"In fact, even if, in general,\"", "claims_and_evidence_yn": "No", "claims_explanation": "The notion of background temperature remains vague, since the proposed pipeline for computing it contains several possible choices and vague descriptions. The experiments are minimal and insufficient to corroborate the proposed pipeline.", "audience_yn": "No", "audience_explanation": "The general questions is very interesting and the proposed approach might have some merit. But in its current very preliminary form, I doubt that much can be gained from this paper, unfortunately.", "requested_changes": "See weaknesses. In particular: \n\n- Run thorough experiments that come close to the intended pipeline. \n\n- Verify the pipeline in settings where true temperatures are available. \n\n- Clarify the intended pipeline and its notation. \n\n- Compare to simpler diversity measurements.\n\n- Fix basic formatting aspects of the paper, such as section lengths, figure sizes, number of references.", "broader_impact_concerns": "I do not think a Broader Impact Statement is necessary. While non-determinism in LLMs is of broad impact, this paper tries to quantify it, which rather addresses concerns than raising new ones.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)