Self-Supervised Laplace Approximation for Bayesian Uncertainty Quantification
Authors: Julian Rodemann
Alexander Marquard
Thomas Augustin
Michele Caprio
Venue: Accepted by TMLR ()

Abstract: Approximate Bayesian inference typically revolves around computing the posterior parameter distribution. In practice, however, the main object of interest is often a model’s predictions rather than its parameters. In this work, we propose to bypass the parameter posterior and focus directly on approximating the posterior predictive distribution. We achieve this by drawing inspiration from self-training within self-supervised and semi-supervised learning. Essentially, we quantify a Bayesian model's predictive uncertainty by refitting on self-predicted data. The idea is strikingly simple: If a model assigns high likelihood to self-predicted data, these predictions are of low uncertainty, and vice versa. This yields a deterministic, sampling-free approximation of the posterior predictive. The modular structure of our Self-Supervised Laplace Approximation (SSLA) further allows us to plug in different prior specifications, enabling classical Bayesian sensitivity (w.r.t. prior choice) analysis. In order to bypass expensive refitting, we further introduce an approximate version of SSLA, called ASSLA. We study (A)SSLA both theoretically and empirically in regression models ranging from Bayesian linear models to Bayesian neural networks. Across a wide array of regression tasks with simulated and real-world datasets, our methods outperform classical Laplace approximations in predictive calibration while remaining computationally efficient.

Published in Transactions on Machine Learning Research (03/2026)
Self-Supervised Laplace Approximation for Bayesian Uncertainty Quantification
Julian Rodemann
rodemann @ccispa..nnnnncccom de
Rational Intelligence Lab, CISPA Helmholtz Center for Information Security
Department of Statistics, LMU Munich
Alexander Marquard
alezander. warquard de
Thomas Augustin
thomas Ustal. uni-mmenchenn.dde de
Department of Statistics, LMU Munich
Michele Caprio
michele. saaheld..pprio@mmnnneeseee..r...
Department of Computer Science, The University of Manchester
Reviewed on OpenReview: https: // openreview. net/ forum? id= T8w8L2:3JG
Abstract
Approximate Bayesian inference typically revolves around computing the posterior param- eter distribution. In practice, however, the main object of interest is often a model's predic- lions rather than its parameters. In this work, we propose to bypass the parameter poste- rior and focus directly on approximating the posterior predictive distribution. We achieve this by drawing inspiration from self-training within sclf-supervised and semi-supervised learning. Essentially, quantify Bayesian model's predictive uncertainty by refitting on self-predicted data. The idea is strikingly simple: Tf a model assigns high likelihood to self-predicted data, these predictions are of low uncertainty, and vice versa. This yields de- terministic, sampling-froe free approximation of the posterior predictive. The modular structure of our Self-Supervised Laplace Approximation (SSL.A) further allows us plug different prior specifications, enabling classical Bayesian sensitivity (w.r.t. prior choice) analysis. In order bypass expensive refitting, we further introduce an approximate version of SSLA, called ASSLA. We study (A)SSLA both theoretically and empirically in regression models ranging from Bayesian linear models to Bayesian neural networks. Across wide array of regression tasks with simulated and real-world datasets, our methods outperform classical Laplace approximations in predictive calibration while remaining compulationally efficient.
1 Introduction
Despite all the morits of Bayesian mothods, one of their notorions shortcomings is the fact that in many applications the posterior turns out to be computationally intractable. Let @ denote the parameter space of interest and S denote the sample space. Let denote a generic element of O, and D be a collection of elements of S corresponding to the collected evidence. By Bayes' theorem, we have that
p(0 D) p(D ())(() p(D)
P(D ()T(O) Jo PDD ())
(1)
where for maximum generality assumed θ and S to be uncountable, that p(0 D) is the probability density function (pdf) of posterior P(- D) Pp, p( 0) is the pdf of likelihood Po, and =(0) is the pdf of prior H Oftentimes, the denominator in equation ] can only be solved numerically. As a consequence, to
Notice that pDD is the pdf the marginal likelihood.

---

Published in Transactions on Machine Learning Research (03/2026)
obtain the posterior, the scholar needs to resort to approximation techniques such as Variational Inference (VT), Markov Chain Monte Carlo (MCMC) or Laplace approximations, see Section 2
Csually, these approximations require a large computational overhead, which makes Bayesian techniques slower than their frequentist counterparts. This is especially true in Machine Learning (ML) and Artificial Intelligence (AT) applications, where Bayesian Neural Networks (BNNs) are much slower to train than classical NNs. In this work, we propose technique to approximate the posterior predictive p(D | 0, D), where D denotes unseen test data, that allows us to overcome this type of shortcoming. Loosely inspired by recent work martingale posteriors (Fong et al. 2023 Lee et al., 2023 Moya & Walker 2025 we forego explicit calculation of the (parameter) posterior and focus on the posterior prodictive distribution. The posterior predictive is a crucial quantity in practical Bayesian inference: It informs the scholar about the distribution of the predictions resulting from the posterior distribution of 8. constitutes one major, if not the main advantage of Bayesian inference over frequentist procedures. The latter lack inherent methods of quantifying predictive uncertainty, because 0 is not treated as random variable.
Our approximation is based on the well-known Laplace approximation method; this means that the posterior predictive is approximated by an unnormalized Gaussian distribution. We note in passing that this is not necessarily a severely restrictive assumption: For example, it is customary in Variational Inference (VI) to approximate the true posterior with its closest (according to the KT, divergence) Gaussian distribution.
The proposed self-supervised Laplace approximation is based on a simple, yet far-reaching insight from reciprocal learning (Rodemann et al.] 2024 Rodemann & Bailie 2025). or more specifically, from self- and
We can learn something from refitting self-predicted data. Contrary to self- and semi-supervised learning, however, we do not aim at increasing predictive accuracy, but approximate the predictions' uncertainly.
semi-supervised learning (Chapelle al. 2006 Zhu & Goldberg 2009 Jing & Tian 2021 :
Augmenting a likelihood with an additional (pseuddo-obsservation is a standard Bayesian device under con- ditional independence assumptions. Our contribution is not the use of psoudo-obseevattons per se, but the resulting closed-form Laplace approcimation of the posterior predictive evaluated at the model's self- prediction. Concretely, we derive a tractable approximation of log p(g((1 In+1, that decomposes into a difference of three interpretable increments: (i) log-likelihood increment, (ii) log-prior increment, and (iii) curvature (log-determinant) increment. This decomposition yields a practical predictive sensitivity signal: the larger the change induced by Ba+1, the higher the predictive uncertainty assigned to that prediction. To avoid ambiguity with broad uses of the expression "self-supervised learning", we interpret the construction as a self-training based Laplace approximation and refer to the mothod accordingly.
In a nutshell, we propose to quantify uncertainty of predictions by refitting on these very predictions. Coneretely, we investigate how a model's likelihood/loss changes we ceteris paribus refit the model on the enhanced data set. Intuitively, the lower the model's likelihood of self- predicted data, the higher the model's predictive uncertainty.
The remainder of this article is structured as follows. In Scction 2, we briefly review key methods in approximale Bayesian inference. Section 3 introduces our Self-Supervised Laplace Approximation (SSLA) method, detailing the theoretical foundations. All proofs are relegated to Appendix A. Section 4 presents extensive experiments: First, we validate our approximation on classical conjugate- models for regression problems, then we demonstrate the method's performance in heteroscedastic regression tasks using neural networks, comparing against established Bayesian inference approaches. We conclude with evaluations on various real-world datasets, underlining practical applicability and robustness of our approach. Section 5 concludes the paper, summarizing findings and potential directions for further research.
2 Background and Related Work
Approximate Bayesian Inference (ABI) lackles the practical issue of computing posterior distributions that are typically analytically intractable. In particular, the central tools in Bayesian Deep Learning (BDL), viz. Bayesian Neural Networks (BNNs), often face significant computational challenges, nccessitating efficient approximations. This section briefly discusses BNNs, and reviews key approximation techniques, including Variational Inference, Markov Chain Monte Carlo (MCMC), and Laplace.
2

---

Published in Transactions on Machine Learning Research (03/2026)
Bayesian Neural Networks. BNNs incorporate Bayesian inference into neural network training by mod- eling the network weights probabilistically. Prominent approaches involve explicitly setting priors on weights (Blundell et al. 2015) or implicitly via dropout techniques (Gal & Ghahramani] 2016), interpreting dropout as a form of approximate Bayesian inference. Techniques like Stochastic Gradient Langevin Dynamics (Welling& & Teh 2011 provide scalable inference options.
Variational Inference. Variational Inference (V1) approximates the posterior distribution through opti- mization, minimizing the KL divergence between the true posterior and set of "simpler" distributions (Blei et al. 2017). Classical methods include mean-field variational Bayes (MFVB) Beal 2003) and importance- weighted autoencoders (TWAE) (Mnih & Gregor 2014). Deep generative models, notably Variational Au- toencoders (VAEs) (Kingma& & Welling, 2014), extended these frameworks, incorporating hierarchical mod- 2017) and amortized inference (Kingma & Welling 2014). Techniques such as stochastic variational inference (SVI) (Beal & Ghahramani] 2000) and black-box variational inference (BBVT) (Ran- 2014] improve scalability but struggle keep pace with increasingly complex architectures. Ortega al.
eling (Higgins et al.
ganath ct al.
)024) merge variat ional inference and Laplace approximations by variational sparse Gaussian
Process approach, enabling sub-linear training time.
Markov Chain Monte Carlo (MCMC). MCMC techniques provide theoretically exact posterior sam- pling by constructing Markov chains whose stationary distributions represent the (true) posterior. hods such as Hamiltonian Monte Carlo (Neal, 1993) and the No-U-Turn Sampler (NUTS) (Hoffman & Gelman 2014) are widely employed. However, computational demands limit their practicality in large-scale scenarios
(Papamarkou et al. 2022). partly addressed by Wiese et al. (2023): Sommer et al. 20024 2025
Laplace Approximation. Laplace approximations efficiently approximate posterior distributions with Gaussian distributions centered at the posterior mode, leveraging the posterior's local curvature. Recent advances, such as Kronecker-factored Hessian approximations (Eschenhagen et al. 2023), improve computa- tional efficiency significantly, see also Antoran et al. [2022]; Bouchiat et al. (2023) Daxberger et al. (2021a
Eschenhagen et al. (2023); [mner et al. (2021b); Cinquin et al. (2024). Integrated Nested Laplace Approx-
imations (INLA) (Rue et al. 2009 Martino Riebler 2019) extend these methods to handle multimodal
posterior landscapes more robustly.
The basis for Laplace approximation is Laplace's method, introduced by Pierre-Simon de Laplace in 1774 [Taplace 1986) It serves as tool for approximating integrals of the form
e (f(x) da,
(2)
where fxx is a twice-differentiable function, C > 0 is a constant, and the boundaries a and may potentially
extend to infinity. In the context of Bayesian statistics, Laplace approximation usually denotes either the
estimation of the normalizing constant So p(D) ())( (marginal likelihood) [Llorente et al. 2023) using
Laplace's method,
the approximation of the posterior distribution p(0 D) Teerney & Kadane 1986
through a Gaussian centered at the maximum posteriori estimate, see Laplace (1986) Schwarz 1978
Tierney & Kadane (1986) for initial works and Konishi & Kilagawa (2008) Llorente et al. (2023) Turkman
et al. (2019) for modern textbook proofs of the approximation's 's properties. The Laplace approximation
( D) of p(0 () builds on a second-order Taylor approximation of the likelihood, which then yields a
Gaussian integral (GauB 1877), whose solution gives
(3) where q is the dimension of θ ÓMAP the MAP estimator ÓMAP == arg maxe (E) + log (0)). and postt (OMAP) == - (3(( + log (B)) 0=brAr the negative Hcssian of the log-posterion at 6MAP- In density form, p00 D) & N(O; MAP. (QMAA) The computation of the Hessian was long considered (and to some degree, still is) computational bottlencck in deploying Laplace approximations. A direct computation grows quadratically in d. The Gauss-Newtte approximation has long been the most popular way to circum-
p(0 D) ØL D) = (2/)-
4/2 | Hpost (fMAP) 122 exp (-2(0 0000--0000 0MAP) H (poat(SMAP) - 0MAP))
vent such computational hurdle (Ritter et al. 2018baa a: Kristiadi et al. 2020 Lee al. 2020 Tmmer et. al.
3

---

Published in Transactions on Machine Learning Research (03/2026)
2021b). Recently, however, the Kronecker-factored approximate curvature (KFAC) has attracted increasing
attention due to its scalability and simplicity (Eschenhagen et al. 2023). KFAC relies on block-diagonal factorization of the Hessian (Martens & Grosse 2015
In this work, we shift the focus to the posterior predictive distribution
p(a+22 Tnt1, = / p(0m+1 12n11 0) p(0) D) d0,
(4)
where In+1 is a new input (feature) and 2n+1 is the predicted output (target) with n the cardinality of the collected evidence D. While harder to approximate (at least at first sight, see Section 3), the pos- terior predictive is the most relevant quantity for practical prediction problems addressed via Bayesian methodology.
It provides the user with direct information on the uncertainty of the predictions, rather than on the pa- rameter space. The modus operandi in deploying Laplace approximations in BNNs is to approximate the posterior p(0 D) by some Laplace approximation (2 D) and then use plug-in inn MC-samples from
[ p(0n+1 11 1n+ 0) (r D) d0
(5)
to approximate the posterior predictive distribution [Bosch et al. 2022 Kristiadi et al. 2022
The necessity
of MC-sampling drastically increases the computational overhead. Daxberger et al. 2002 a
summarize and
implement a few alternatives ranging from probit approximation (Sieeeehallter & Lauritzen
1990 for binary
classification to extended probit (Gibbs 1998) or approximating the softmax Gaussian integral via a Dirichlet voftmax-Gaussian
distribution (Hobbhahn al. 2022) for general classification.
This is the regime address in this work: we derive a direct. Laplace approximation of the poste- rior predictive that avoids Monte Carlo integration while remaining applicable to generic deep regression models. We position (A)SSLA as a complementary approach to reduce predictive sampling cost, while ac- knowledging that specialized alternatives exist in particular regimes (e.g., comment-roopaaatie methods, daterministtic/lowvvvainnee VT variants, and Bayesian last-layer models).
In addition, our method is modular with respect to the prior specification. That is, it can incorporate different priors directly in the approximation. This is especially important in the sewly-introduced field of Credal Bayesian Deep Learning (CBDL, Caprio et al. (2024)) building the generalized Bayes rule by Walley 11911. ; see also Walter & Augustin (2009); Rodemann et al. (2023a). There, finite collections of priors and of likelihoods are combined pairwise (thus inducing a combinatorially expensive problem to solve) and VI-approximated to derive a finite set of posteriors, and in turn a finite set of posterior predictives. The convex hull of the latter forms a credal set, i.c. a closed and convex set of probabilities, a contral concept in
Imprecise Probability theory (Walley 1991 Augustin et al. 2014 Troffaes & de Cooman 2014
While there is currently no broadly adopted deterministic method that eliminates the need for MC sampling in modern Bayesian neural networks, there are alternatives for special settings that can reduce avoid brute-force Monte Carlo (MC) sampling at prediction time in certain settings. These mots are not (yet) broadly adopted, general-purpose replacement for MC integration for posterior predictives in modern deep Bayesian regression models. Examples comprise approaches such as probabilistic backpropagation that propagate approximate moments through the network to obtain predictive uncertainties without explicit MC sampling. They rely on strong distributional and independence approximations and historically have not become the default choice for large-scale modern architectures (Ilernández-Lobato & Adams, 2015). Moreover, there exist deterministic (or low-variance) alternatives standard stochastic variational inference that aim to approximate predictive moments with reduced sampling noise; these are nonetheless specialized approximations and are not the commuity-dccfult for obtaining posterior predictives in deep regression BNNs (Wu et al. 2018). Furthermore, restricting uncertainty to the final layer can yield analytic or near- analytic regression predictives, but this constitutes a different modeling choice rather than a general solution
Foor modern references, see c.g. Caprio & Mukherjec (203 Caprio & Scidenfeld (023) Lu al. (024)
Caprio ct
2025;:
Dutta
al. (2025] : Caprio (2025)) Rodemann & Angustin
2021
2022 2024): Jansen al. 2022a
2023
2024
Jansen (025)
Baille & Dem 2025) Frohlich (025).
4

---

Published in Transactions on Machine Learning Research (03/2026)
for full-networt Bayesian regression posterior predictive evaluation (Watson et al. 2021 Harrison et al. 2024
3 Self- Supervieed Laplace Approximation Inspired By Self- Seainnng
In this section, we introduce Self-Supervised Laplace Approximation (SSLA), as well as the idea for its approximate counterpart. As pointed out before, SSLA bypasses the explicit computation of the (pa- rameter) posterior distribution, focusing directly on approximating the posterior predictive distribution. This is achieved by refitting self- reff-predicted data, drawing inspiration from self-supervised and semi- supervised learning. Specifically,, in semi-supervised learning, where the likelihood is jointly evaluated for labeled and (new) self-labcled data Rodemann et al. 2023b c see also Rodemann (2023 2024); Dietrich el al. (2024) and from related work on cautious disambiguation of set-valued labels (Rodemann et al. 2022). As turns out, SSLA allows us plug-in and plug-out priors in a modular fashion. Further approximations allow us get rid of the requirement to refit the model on self- self-predicted data. We call this version the Approximate supe-viuer Laplace Approximation (ASSLA) and introduce it formally in Section 4
draw inspiration from approximately Bayes-optimal eseudo-labeling
Let = V/ where X' is the space of imputs and Y is the space of outputs. Suppose observed data points D = {2 3)) C S, and assume that the (conditional) likelihood is given by I[ P(3: Iz 0) Further, let fo( be some (frequentist) base model that is parameterized by 0 and produces predictions Do+1 given new observations Tn+1- We call Dz+1 = {(x,, }))} i U {(#n+1, bn+1)} the augmented dataset. Consider the (posterior) predictive distribution, Le., the distribution of any Bayesian model's predictions (In+1. (n-1) given training data D,
p(0+11 Ia+1, D)
DD= -)=
p(0n+1 (n+1, p(0 D) d0.
(6)
The main idea behind our approximation is to expand the likelihood p(D () in equation 1 uusing the likelihood of the model's prediction p(βn+1 In+1, 0). This allows us to forgo the need to compute p(0 D) entirely. Define (0) = log | 0) Σ log Iy 0) the classical log-likelihood of D, and
((am41. (en+)1.uu1))00 == log p(un+1 Tn+1, 0) as the log-likelihode of the predicted instance. Further, denote their
sum as E(0)= = f(e, 11) (0) (0). As consequence of Bayes' theorem (equation 1 we can write the
integrand in equation 0 as
p(0.+1 1)+1, p(0) D) exp exp|((0)]=(0) p(D)
Let [(0) -((+(()==
denote the observed Fisher information matrix (the negative Hessian)
and [0) = Vi((0) the observed Fisher information matrix of the augmented dataset, respectively. Further
denote by E arg max, ((0) the maximizer of Ł(0). It holds that ado) expansion of the second order around thus gives
= 0 by definition of A Taylor
86
-==
ē(0) E(t) 2 (0-0) B) ((0) 0).
The inlegrand decays exponentially in 110 0122, where || ||2 denotes the Euclidean norm This allows us to approximate it locally around 0 by (0) se T0) inside the integral with another Taylor series. We refer (Miller) 2006, Scction 3.7) for a detailed treatment of the remainder terms and regularity conditions; Theorem 2) for background on π (0) R 7 (0). We thus approximate p(gn+1
Specifically, (Lapiński 2019
Int1, D) by
exp ((() [(0))=(0) (0) D(D)
/ exp -
1 (0-00 -0) do.
3 As is customary, assume that given inputs x, and parameter 0, the Y, have densitics p(M 2).0) and independent
4This implies that O a subset of Euclidean space If insteed + subset of generic normed space (V, ||l+ll];; substitute | lla with || llv.
5

---

Published in Transactions on Machine Learning Research (03/2026)
The integral Se expl (00 -10-)0000) (e- d is Gaussian integral GauB 1877). Defining Σ [ā(()] and dg as the density of the multivariate Normal distribution N (0, E), we have that
/ exp
- -0-0 00 d0 - / Φr =
(2x) (2 (3()
where q is the dimension of the Euclidean space we are working in, and ||| denotes the determinant operator Combining equation 7 and equation
g
we obtain
(2 9/2 exp [(0) (())
p(0+11 In+l, D) (2Th)
(9)
[(0))_/2p(())
Taking the logarithm of equation
we get
log p(0+1 Tn+1, D)
2 2
log (2T) + (()) + log - log [J() - log (0)
1
(10)
Borrowing from some classical Laplace approximations of the marginal likelihood (Bishop & Nasrabadi 2006
Konishi & Kitagawa
2008
Llorente et al.,
2023
Schwarz
1978 we can approximate log as follows
P(D) = log
p(D () T0)) do
(11)
=. ! log (2m)
1
log [ log 10g
where E arg max, (0),,, see Konishi
Kitagawa
(0008
Section 9.1.3).
Combining equation 10 and
equation we obtain
1 2 log [(0)..
logp(0n+1 In+l: D) ~ ((0) - (()) + log (0) - log (0) log [
(12)
Equation 12 embodies the main idea behind Self-Supervised Laplace: We approximate the posterior predictive at 0n+1 by the change this very Dn+1 causes in the model fit's prior, likelihood, and Fisher information (i.e., all evaluated at the model fit). Note that these changes are driven by a change in parameter (from to 0) as well as in funational form (from l to & and from J to J, respectively). Equation 12 also shows the modularity of our approximation. As the effect of the prior is additive, no refitting under different prior specifications is required. In Appendix we expand on and illustrate such a prior modularity.
On (non-)indeeenddence of the pseuudo-bseervatin:: We emphasize that we defined i(0) C()) (an.ei/ (0), which implies that the log- likelihood must be evaluated twice- once for the existing data D. and once for the new observation (In+1, Dn+1). If we instead wanted to evaluate the likelihood only once- namely, only for the combined dataset DU ((nn11 [n+1)} we would have to assume independence between D and {I 1. However, since In+1 is derived as a function of the dataset D, this assumption of independence does not hold. The same reasoning applies to the extended Fisher info [0).. We thus abstain from such simplification and provide further approximation instead. To sum up, our approximation does not require treating (1++1,0n+1) 1) as independent data point. Indeed, since In+1 itself function of D, any assumption of conditional independence would be incorrect.
Implementation summary: Algorithm I summarizes SSLA, which uses the augmented objective and evaluates predictive uncertainly via likelihood, prior, and curvature increments. Algorithm 2 1 summarizes ASSLA, which replaces with 9 and retaining the leading- order likelihood and curvature effects, see Section 4 right below. both cases, the augmented objective used as a sensitivity probe and not interpreted as the likelihood of an enlarged i.i.d. dataset.
Tn the general case, = dim(V).

---

Published in Transactions on Machine Learning Research (03/2026)
Algorithm SSLA: Self-Supervised Laplace Approximation of the Posterior Predictive
Input: Dataset = {(z,, yi)] test input Ix+1, prior (8), log-likelihood ply x, training objective (D(0) log z).
Output: Approximate rng-posterioo-prediiiiii
LogsssLA( (y In+1, P).
Fit on training data: I E arg mago (f)( + log lo))
2: peed-prrdic choose In+1 as the point prediction at 2n+1 under & (e.g. predictive mean / MAP prediction).
Define P(0) + (D( + log p(n++ 1n)1,
Obtain E arg max ( (f(() log (0)).
Curvatures:
6:
J(Ø) t - V (e) + log π (0)) o=
J(Ø) V (e(0) + log ग (0)) A-
Compute SSLA log-PPD (up to constant):
fSSILA
(y/ 1n+1. D) < (0) (0) log क (0) - log n (0)
10:
+ Hogg(y In+1, 0) & [[oo [J(0] log [O)]]
11: return log PSSLA 2n+1, D).
Input: Dataset D = Approximation of SSLA
Algorithm ASSLA:
test input In+1, prior x(),, log-likelihood log p(y xx, 0), fitted parameter
8, curvature J()).
Output: Approximate
log PASSLA Tn+1, (up to an additive constant inde-
pendent of y).
1: Self-prediction: choose Dn+1 as the point prediction at Ta+1 under θ.
2: Set + 0 (ASSLA's 's defining approximation).
3: Local increment terms at 0:
4:
Ar(y) + log En+1, 0) - log p(00+ Io+1,
(y < log [(0) + Jn+1 (y: | log [J()
5:
where Jo+1 (y: 6) denotes the observed curvature contribution from (Ix+1,9) at 0.
Compute ASSLA log-PPD (up to constant):
log PASSLA (y/ Tn+1, D) Ar(y) As(1).
return log PASSLA ( Tx+1, D).
4 Approximating Further
From an applied perspective, equation 12 requires the computation of the prior, the log- likeliiel and Fisher information. Specifically, these three functions need to be evaluated both at and 0, which means that we have to solve 0 E arg max, ((0) in addition to the usual arg maxg ( = arg max, [E log p(g x 0)], which can be burdensome for large models.
A natural question, then, is whether the computational overhead given by arg max, {(0) can be reduced. The following results allow us to answer positively to such query. The challenge is that simple approximation ~ (Lemnaa) 1h does not sulfice, since the approximation error thereof might be propagated and increased through l and l. We thus need to derive the approximation error for ε(0) ((0). Thcorem finds it for Lipschitz-coontinuuui continuus loss function, which implies that we can approximate ((0) {()) with an approximation error that decreases linearly in
Assumptions.
Throughout the following results, we assume:
(A1) The per-sample lieeliood terms log p(ys Ii, 0) are twice continuously differentiable in neigh- borhood of 0.

---

Published in Transactions on Machine Learning Research (03/2026)
(A2) The observed curvature matrices positive definite in this neighborhood, with cigenvalues bounded below by positive constant.
(A3) The log- prgpproo log (0) twice continuously differentiable with bounded gradient and Hessian in a neighborhood of 0.
Lemma 1. Under Assumptions
it holds that 0 = 0 O(n where O denotes the Bach-
mann Landau big-O nolation. That is, 0z 0 for growing n..
Theorem 1. Assume that the loss is Lipschizz-ontimmuus. and denote by its Lipschitz constant. Then, under Assumptions (A1)-(A3), we have that
Ln + (
=E0)+ +0
n2
12 however, depends on not only through ε(0), but also through the Fisher info J()) and the prior π (0). So we need to take into account the approximation error of I - V2 - -V)(() and (0) x (0). Corollary | takes care of the latter two.
Our approximation in equation
Corollary 1. Assume that the tikelihood's second derivative Vie and the prior are both Lipschitz- continuous. Then, still assuming (A1)-(A3), we have that
Ln +1 n2
I(0)= = -
and
[n +
-(0) + +0
n²
As we can see, Theorem ! and Corollary eliminate the need to compute entirely. That is, we do not have to refit (and optimize) the model for both D and {(In+1, We can restrict ourselves to the parameter vector 6, which can be estimated from the initial training of single non-Bayesian model, e.g., a neural network. We are left to compute the expanded likelihood i evaluated at 0. We also point out how, as a result of the Ganss-Newton approximation (Foresee & Hagan 1997), the observed Fisher information (()) = (0)/n general 0 can be approximated as
((0) ((()))(())
(13)
where G(B) is the Jacobian vector that we obtain from İ. Corollary 2. Given Assumptions (A1) (A3) and Lipschitz loss, the following is true
1 n ((0)
T
Corollary 2 implics that we only have compute the Fisher information for (Za+1 8a41) with given param- eters 0. For the example of a BNN, this requires only one pass through the non- con-Bayesian neural network with weights 0.
The posterior predictive distribution under ASSLA therefore becomes:
1og(p(0.-11Em41) D)) ~ (()) + log (I(0)) {(0) - log (1(10)
(14)
due to the fact that the difference between πt (0) and (0) of order O ((ل see Corollary Recall from this Corollary [ that the functional form of the prior itself does not depend on (In+1, 1n+1). Thus, its
contribution is subleading relative to the likelihood and curvature increments and can be neglected at the
8

---

Published in Transactions on Machine Learning Research (03/2026)
level of approximation used in ASSLA. In contrast, both the likelihood and the Fisher information change at leading order and therefore remain in the approximation. This formulation bypasses the computational
overhead of refitting the model. Due to {(0) = & (=n+110041)) (0) + E(0), equation 14 becomes
1 Og(pp0n+11nn+1 D)) se (( + log (IT01)
1 log (T(])
(15)
2
allowing us to bypass the calculation of the log-likelihood of the training data E()).
Why the prior increment can be dropped in ASSLA:
Equation 12 contains the prior increment
log 10g log (0). In ASSLA, we replace the refit optimum by the original optimum Ô and keep only leading-order terms in the expansion. By Corollary 1 (and Assumption (A3)), the shift A = Ô- is O(n 1)), and since the prior's functional form is unaffected by Da+1, the prior increment is of higher order
log ((0) log (0) O(I = O(
or smaller under the stated bounds. In contrast, the likelihood and curvature increments remain leading- order at the approximation order used in ASSLA, and are therefore retained. This explains the apparent disconnect: the SSLA decomposition contains the prior increment, while the ASSLA simplification drops it as higher-order term at the chosen approximation order.
5 Experiments
We first verify our method in the conjugate case for regression lasks. That is, we use prior distributions which are conjugate to the likelihood and therefore result in posterior predictive distribution (PPD) that can be computed analytically. We can thus determine whether (A)SSLA provides reliable and close approximations of the true underlying PPDs in a controlled manner, before scaling to more complex models. We compare three lypes of conjugale prior models: The normal-normal model, the Poisson-gamma model, and a conjugate Bayesian linear regression. All results and detailed descriptions of the experimental setup can be found in appendix B
To sum up, SSLA and ASSLA are able recover the analytic PPD close to perfect for sample sizes ween 20 and n 100000. Numerical instabilities emerged in our experiments for 1000000 for ASSLA. The reason for these instabilities primarily lies in the curvature term. Table shows that subbraact-two-aggrege. computation in float64 yields max loog -errors on the order of 10 10-11 -10 across all n. confirming that the observed degradation in float32 is due to finite-precision cancellation rather than a methodological limitation of ASSLA. However, SSLA is still able to match the analytic PPD closely in these cases For the normal- normal model, Figure [ illustrates the comparison of the posterior predictive densities for different sample sizes. Similar visualization for the other conjugate cases can be found in appendix B
After having validated our approximation for the conjugate case, we conduct twofold benchmark study, consisting of simulated and real world data.
5.1 Simulated Heteroscedastic Regression in Neural Networks
Building on the validation in conjugate settings, next. consider controlled heteroscedastic regression task to evaluate how well uncertainty quantification methods adapt when the noise variance is input-dependent. We compare SSLA and ASSLA to suite of established approximate Bayesian inference techniques: the classical Laplace approximation (Daxberger et al. 2021a), variational inference including both the mean-fiell field BNN of Blundell al. (2015) and the more expressive VT model of Depeweg et al. (2018), and Hamiltonian Monte Carlo implemented via hamiltorch (Cobb 2023b a sampling-based reference.
Thee transition between the regimes S 10 and n 10th is gradual. Additional experiments (see Table 1 in the conjugate normal-normal model for (2. 10 3- 10 5. 10
7 10 100) show that ASSLA remains stable but exhibits increusing numerical error grows. The degradation is primarily caused by finite-precision cancellation: ASSLA computes predictive increments by subtracting two large aggregated log-likelihood terms in float32, which can collapse small differences to zero. Repeating the same computation float64 eliminates this effect, confirming that the instability is numerical rather than methodological.
9

---

Published in Transactions on Machine Learning Research (03/2026)
Comparison of SSLA and Analytic Posterior Predictive Distributions
ASSLA (Entropy=0.00)
SSLA (Entropy=0.00)
Analytic
20
100
1000
Posterior Predictive Density (PPD)
0.2
0.2
0.2
0.1
0.1
0.0
0.0
50
10000
100000
1000000
0.2
0.2
0.1
0.1
0.0
20
30
40
50
10
20
30
40
10 20
30
40
50
New
Figure Conjugate normal-normal model: Six comparisons SSLA (red) and ASSLA (black) analytic posterior predietive distribution (green) for varying sample sizes n ranging from n 20 to 1000000. SSLA and ASSLA are able to closely match the analytic distributions, highlighted by low entropy scores.
The synthetic data are generated with the ToyHeeterosedastticcae cDataModule from Lightning- -Qa UQ-Boxx Box (Lehmann et al., 2024). Inputs x are drawn from a mixture of three Gaussians (centers Tmii -4, 0, and Tmax with standard deviations 0.4, 0.9, and 0.4, respectively), and targets follow
y (x) +e, c= cos 2 (o,,
(16)
so that the noise amplitude varies smoothly with æ. A multilayer percoptron (Bishop & Nasrabadi 2006 Peng 2017) used as the predictive model; architectural choices, loss definitions, and hyperparameter selection are detailed Appendix C For the Taplace-based methods (including SSLA and ASSLA), the
observed Fisher information is approximated via three covariance representations: diagonal, Kronecker- factored (KFAC), and dense. We additionally report Lightning UQ Box (Lchmann et al. 2024 results in
Appendix D
Figure depicts the posterior predictive means with associated credible bands, and Table reports empirical coverage at nominal confidence levels of 95%, 90%. 75%, and 50%. The classical Laplace approximation exhibits conservative calibration, producing wide, risk-averse intervals. SSLA''s 's uncertainty estimates vary depending on the covariance approximation and suffer from instability calibration.
ASSLA yields smoother and comparatively tighter credible regions, particularly in narrower intervals, but systematically underestimates uncertainty in wider intervals, leading to mild risk-seeking bias. This be- havior reflects the core approximation in ASSLA: replacing the augmented optimum by the original fit Í neglects refitting-iduued curvature changes. When the pseudo-observation would have shifted the optimum non nogligglg,,,,, ASSLA produces overly concentrated predictive densities. Consequently, ASSLA balances computational tractability with competitive calibration moderate regimes, but may under-cover at high nominal levels in settings where refitting effects are substantial. In such regimes, SSLA or more conservative approximations should be preferred.
shouol
10

---

Published in Transactions on Machine Learning Research (03/2026)
72
rums
KL (mean sd)
Max |log] err (mean + sd)
Frac(6=0)
Frac(nonfinite)
100,000
9.41x10 1.3x10
7.57x10 3.1xx10
0.00
0.00
200,000
5.02x10 2.3x10
1.55x10 -2.x110
0.00
0.00
300,000
1.41x10 + 3.3xx10
3.05x10-2 = 5.1x10-4
0.00
0.00
500,000
1.40x10 333xx10
3.06x10 2 =3.1x10-
0.00
0.00
700,000
4.75x10-1 1
5.86x10 2.1x10
0,00
0.00
900,000
4.76x10 2.0x10
5.72x10 2 2.7x10
0.00
1,000,000
4.75x10 9.2x100
5.69x10 2.6x10-3
0.00
0.00
Intermediate regime n E [10 10 in the Normal-Normal conjugate setting: the transition is gradual rather than abrupt. Reported numbers compare a numerically stable reference computation against an ASSLA float32 computation, which illustrates fmite-precision cancellation effects.
Table 1:
Comparison of Credible Intervals Computed by Different Approximation
Observations
Predictions
95% Interval
KSBC
-1.5
+1.3
+0.5
04
8.5
1.
-35
-1.0
+0.
0.
8.5
1.8
1.5
-15
-13
+0.5
0.
8.5
13
15
-15
-23
13
-15
45
1.
-15
8.5
Figure We display the predictive uncertainty intervals for various uncertainty quantification methods, namely SSLA, ASSLA, LA, VI, and MCMC. The shaded blue areas represent the 95% credible intervals, with dots representing observations and the solid blue line indicating the model's predictions.
The flexible variational inference model tracks nominal coverage most closely with minimal systematic devia- tion, whereas MFVI collapses toward the predictive mean and manifests risk-averse behavior. HMC, despite being theoretically grounded baseline, reflects practical linearization limitations in this implementation and delivers intermediate coverage (c.g., approximately 76% at the 75% level and 68% 50%), capturing the underlying structure without extreme dispersion.
These quantitative and qualitative findings expose nuanced trade-off surface: SSLA approximate coverage in certain regimes but is hampered by stability concerns; ASSLA strikes favorable balance in computational elliciency and interval smoothness relative to classical Laplace and HMC, yet its broader underestination in wide intervals requires caution; variational inference provides the most balanced em- pirical calibration; and the standard Laplace method remains the most conservative. Together with the earlier conjugate-case insights, this comparison refines practical guidance for selecting posterior-predicte approximation techniques under heteroscedastic neural regression. (taaeer & Veigal 2023
11

---

Published in Transactions on Machine Learning Research (03/2026)
SSLA
ASSLA
LA
VI
MCMC
CT
KFAC
DIAG
DENSE
KFAC
DIAG
DENSE
KFAC
DIAG
DENSE
VI
MFVI
HMC
95%
88,0
92.0
92.0
76.0
76.0
76.0
100.0
100.0
100.0
92.0
100.0
88.0
90%
84.0
92.0
88.0
68.0
68.0
68.0
100.0
100.0
100.0
92.0
100.0
88.0
75%
76.0
80.0
76.0
60.0
60.0
60.0
92.0
92.0
92.0
80.0
96.0
76.0
50%
72.0
64.0
56.0
56.0
56.0
56.0
68.0
68.0
68.0
64.0
80.0
68.0
Table 2: We report the coverage of different methods at various confidence intervals for heteroscedastic regression, The table shows the coverage percentages for SSLA, ASSLA, LA, VI, and MCMC methods, evaluated using KFAC, DIAG, and DENSE for each approach at the 95%, 90%, 75%, and 50% confidence intervals.
Dataset
Description
Size
Subsampling (n 50)
Concrete Compressive
Modeling the compressive strength of concrete us- Medium ing features.
Strength (Yeh
1998
Wine Quality Cortez
Portuguese "Vinho Verde" wine, with quality scores ranging from to 10.
Medium
& Reis 2009
Bike Sharing
Fanace-
Prediction of total rental bike counts using hourly and daily data.
Medium
T 2013
Airfoil
Solf-Noise
NASA aerodynamic and acoustic test results for airfoil blade sections (Moin et al. 2022
Medium
Brooks & Marcolini
1989:
Auto MPG Quinlan
Predicting the "mpg" attribute using 7 features.
Small
x
1993
Liver Disorders Asun-
Bload test. results used predict daily alcoholic beverage consumption.
Small
x
cion et al. 2016
Daily Demand Fore-
Daily demand forecasting in a Brazilian logistics company.
Small
X
casting Ferreira &
Sassi 2016
Real Estate Valuation
Historical market data for real estate valuation in Small
Yeh
2018
Taiwan.
Table 3: Overview of datasets used for benchmarking ASSLA and SSLA from
UCIMLRepo (Kelly et al.
2025
Subsampling is applicd to modium-sized datascts.
5.2
Case Study on Real-World Data
To assess the empirical viability of (A)SSLA beyond controlled synthetic settings, we apply the methods to diverse set of real-world regression tasks from the UCI Machine Learning Repository (Kelly et al. 2025 These datasets span domains and scales, providing practical stress test for uncertainty quantification under realistic data artifacts and varying sample regimes. Table 3 summarizes the datasets, their modifications, and the subsampling strategy; SSLA restricted to small datasets (fewer than 500 observations) due to its computational overhead, whereas ASSLA employed more broadly with subsampling on medium-sized data maintain tractability.
All datasets are processed through unified pipeline: binary features encoded as 0/1, categorical variables are one-hot expanded, and continuous inputs and targets are standardized to zero mean and unit variance, Several datasct-specific adaptations reframe the raw data into regression tasks or emphasize robustncss: the Wine Quality data incorporates the color attribute and predicts alcohol content rather than quality; Bike Sharing discards the datetime column and predicts normalized ambient temperature; Daily Demand Forccasting Orders is treated as a regression problem via random sampling of train splits; Liver Disorders predicts alkaline phosphatase (ALP) instead of alcohol intake, reflecting a biomedically relevant proxy for liver/bone pathology (Lowe et al. 2025 and Auto MPG omits the non-informative car names field.
12

---

Published in Transactions on Machine Learning Research (03/2026)
Coverage
Dataset
Method
95%
75%
50%
NLL
CRPS 4
SSLA
100.00
91.14
74.68
0.45
0.19
Auto MPG
ASSLA
100.00
82.28
69.62
0.32
0.18
LA
100.00
100.00
94.94
0.99
0.28
SSLA
73.91
44.93
28.99
2.74
0.79
Liver Disorders
ASSLA
49.28
24.64
15.94
5.94
0.85
91.30
66.67
43.48
1.68
0.73
SSLA
98.00
98.00
82.00
0.43
0.19
Concrete Compressive Strength
ASSLA
98.00
90.00
72.00
0.38
0.18
LA
100.00
98.00
98.00
1.00
0.28
SSLA
98.00
88.00
72.00
0.49
0.21
Wine Quality
ASSLA
94.00
84.00
62.00
0.39
0.20
LA
100.00
100.00
90.00
0.99
0.28
SSLA
100.00
100.00
100.00
0.13
0.11
Bike Sharing
ASSLA
100.00
100.00
100.00
0.00
0.09
LA
100.00
100.00
100.00
0.92
0.23
SSLA
98.00
94.00
88.00
0.51
0.19
Airfoil Self-Noise
ASSLA
96.00
90.00
84.00
0.24
0.16
LA
100.00
100.00
94.00
0.97
0.27
SSLA
91.67
91.67
91.67
0.44
0.18
Daily Demand Forecasting Orders
ASSLA
91.67
91.67
83.33
0.34
0.17
LA
100.00
100.00
91.67
1.06
0.29
SSLA
97.59
93.98
79.52
0.96
0.32
Real Estate Valuation
ASSLA
90.36
74.70
50.60
0.74
0.26
LA
100.00
97.59
86.75
1.05
0.32
Table 4: Coverage, NLL, and CRPS results for SSLA, ASSLA, and LA on various UCIMLRepo datasets
The predictive model mirrors the heteroscedastic setting: a multilayer perceptron with two hidden layers of units each and ReLU nonlinearities trained to represent both mean and uncertainty. Hyperparameters are selected via Optuna, searching learning rates in [1 10-5, 10 and batch sizes from 32 256 over 100 trials, with validation negative log-likelihood governing selection. For SSLA we assume standard normal prior on weights, apply last-layer Laplace approximation with post-hoc tuning as in Daxberger et al. (2021a) Chapter 4.1), and approximate the observed Fisher information using KFAC. All remaining training and loss configurations are consistent with the heteroscedastic regression experiment.
Table 4 reports empirical coverage at nominal 95%, 75%, and 50% credible intervals alongside negative log- likelihood (NI.L) and continuous ranked probability score (CRPS), offering both calibration and sharpness diagnostics. The methods exhibit dataset-dependent trade-offs. In Auto MPG both SSLA and ASSLA achieve full coverage at 95%, but ASSLA attains substantially lower NLL than SSLA and the classical Laplace approximation, indicating tighter yet well-calibrated uncertainty. The Liver Disorders task exposes failure mode: both SSLA and ASSLA underperform relative to standard Laplace, with ASSLA showing pronounced undercoverage and inflated NLL, suggesting its adjustment mechanism can over-compress uncertainty when the signal-to-noise ratio is poor.
On Concrete Compressive Strength and Wine Quality, SSLA and ASSLA reach near-nominal high-level coverage while delivering considerably better NLL and CRPS than the overly conservative Laplace baseline, which produces wide intervals that dilute practical informativeness. The Bike Sharing dataset is strong success case for ASSLA: all methods achieve perfect coverage, yet ASSLA yields near-zero NLL and the lowest CRPS, demonstrating its ability to produce highly concentrated, calibrated predictive distributions in favorable signal regimes. For Airfoil Self-Noise and Daily Demand Forecasting Orders, ASSLA consistently improves upon SSLA in efficiency (lower NLL) with comparable coverage, reinforcing its robustness in modium-scale real-world data. In Real Estate Valuation, SSLA slightly edges ASSLA coverage at 95%, but ASSLA achieves a more favorable balance between uncertainty calibration and predictive quality than the conservative Laplace approach, whose inflated uncertainty (and higher NLL./CPPS) reduces sharpness.
13

---

Published in Transactions on Machine Learning Research (03/2026)
These empirical results show a nuanced performance landscape. This kind of multi-criteria evaluation natu- rally leads to comparisons that need not induce total order (Jansen et al. 2024a| b Rodemann & Blocher 2024 Arias et al. 2025a by; we thus discuss the arising trade-offs in what follows. ASSLA frequently delivers uncertainty estimates than classical Laplace, particularly where the latter's risk-aversion would degrade downstream utility. SSLA can sometimes yield marginally higher coverage but incurs higher computational cost and may suffer instability in more challenging regimes. The degradation on Liver Disorders highlights that ASSLA's 's adjustment may overfit uncertainty compression when evidence is weak, indicating regime where fallback to more conservative approximations hybrid regularization might be necessary. Overall, the case study corroborates ASSLA as a practically appealing method for real-world regression: it balances computational tractability with competitive probabilistic calibration across diverse settings while signaling scenarios requiring caution Staber & Veiga 2023
sharper and better-calibrate
6 Limitations and Conclusion
In this work, we proposed to shift the focus of approximate Bayesian inference from the intractable param- eter posterior to the posterior predictive distribution, which is the quantity of primary practical interest for uncertainty-aware prediction. Our Self-Supervised Laplace Approximation (SSLA) quantifies predictive uncertainty by refitting on model-generated (self-predicted) data: predictions that the model assigns high likelihood are identified with low uncertainty, and vice versa. This sulf-super mechanism is modular in the prior, enabling scnsitivity analysis across prior choices. To reduce the computational burden of refitting, we derived an approxinate variant (ASSLA) that leverages asymplotic expansions and local linearization to express the posterior predictive in terms of quantities evaluated at the original fit, avoiding expensive ro-optimization.
Theoretical results characterize the approximation error and justify replacing the augmented posterior mode with the original mode under mild regularity, controlling deviations in the likelihood, Fisher information, and prior. Empirically, we benchmark SSLA and ASSLA across a hierarchy of settings: from conjugate analytic models-whie ground-truth posterior predictives are available-to to controlled synthetic heteroscedastic re- gression and a diverse suite of real-world regression tasks. Comparisons include classical Laplace, variational inference (including mean-field and more expressive variants), and Hamiltonian Monte Carlo, evaluating both calibration (coverage) and sharpness. In controlled settings such as conjugate prior scenarios, both SSLA and ASSLA closely approximate the analytical posterior predictive distribution. In more complex environments like heteroscedastic regression and real-world datasets, ASSLA generally delivers smoother and more computationally efficient uncertainly estimates than SSLA, while traditional methods such as LA often exhibit overly conservative behavior.
However, several challenges remain. SSLA can become unstable on larger datascts or in the presence of prior- data conflicts (Evans & Moshonov 2006 Walter & Augustin| 2009 Marquardt et al. 2023). and ASSLA sometimes underestimates uncertainty in regions with high variance. These findings indicate that while (A)SSLA are promising, they require further refinement to handle all scenarios robustly. First, refining the covariance approximation and addressing numerical instabilities could substantially enhance the reliability of these methods. In some scenarios, it may even be possible to circumvent the full computation of the FIM or to approximate it more efficiently, potentially yielding significant performance gains. Second, given the prior modularity in SSLA, it would be interesting to explore its application within the framework of imprecise probability. In this setting, one could leverage extreme points of prior credal sets to derive an imprecise posterior predictive distribution.
Together, these studies demonstrate that ASSLA in particular often achieves a favorable trade-off wetw computational efficiency and predictive uncertainty quality, producing sharper and belter-calibrated predic- tive distributions than standard Laplace in many regimes, while SSLA provides a more faithful (but costlier) self-refitting baseline.
More generally SSLA/ASSLA do not guarantee that the model likelihood is reliable under arbitrary mis- specification or severe dataset shift: like any Bayesian or likelihood-basor uncortainty quantification method, they inherit the assumptions of the chosen likelihood model. Conditional on the likelihood being mean- ingful local model around 0, SSLAA/SSS. quantify uncertainty primarily through changes induced by the
14

---

Published in Transactions on Machine Learning Research (03/2026)
pseuud-obserratton--bebeee objeectvve and curvature increments- rather than relying the absolute log-likelihood value alone. Under drift, these increments can still surface atypical curvature behavior (e.g., poor condi- tioning and sensitive log-determinants), but any likelihood-based score must be interpreted with caution. In practice, SSLA/ASSLA should be used alongside drift/OO0 detection and, where appropriate, robust likelihood modeling or likelihood lempering.
References
Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next- generation hyperparameteer optimization framework, 2019. URL https://arxiv: htrggabs/19077/ 10902.
Javier Antoran, David Janz, James U Allingham, Erik Daxberger, Riccardo Rb Barbano, Eric Nalismick, and Jose Miguel Hernandez-Lobato. Adapting the linearised Laplace model evidence for modern deep learning. In Kamalika Chaudhuri, Stefanic Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (cds.). Proceedings of the 39th International Conference on Machine Leurning, volume 162 of Proceedings of mlr.
Machine Learning Research, pp. 796-821. PMLR, 17-23 Jul 2022. URL press/ vites//v62/aama wttp/////////2aa...o html html
Esteban Garces Arias, Hannah Blocher, Julian Rodemann, Matthias Aßenmacher, and Christoph Jansen. Statistical multicriteria evaluation of Ilm-generated text. arxiv preprint arXiv:2506. 18082, 2025a.
Esteban Garces Arias, Hannah Blocher, Julian Rodemann, Meiningwei Li, Christian Heumann, and Matthias Aßenmacher. Towards better open-ended text generation: A multicriteria evaluation framework. Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (GEM), pp. 631-654, 2025b.
Arthur Asuncion, David Newman, et al. Liver Disorders. UCI Machine Learning Repository, 2016. DOL: http:///ddoooorg112442222
Introduction to Thomas Augustin, Frank P. Coolen, Gert de Cooman, and Matthias C. M. Troffaes (eds.). Imprecise Probabilities. John Wiley, Chichester, 2014.
James Ballie and Rabanus Derr. Property elicitation imprecise probabilities. arXiv:2007.0887 (last accessed October 27 2025), 2025.
arXiv preprint M. J. Beal. Variational Algorithms for Approzimate Bayesian Inference. PhD thesis, Gatsby Computational Neuroscience Unit, University College London, 2003.
M. J. Beal and Z. Ghahramani. Variational algorithms for approximate Bayesian inference. In Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, 2000.
James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for hyper-parameter optimiza- tion. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K.Q. Weinberger (eds.), Advances in Newral Information Processing Systems, volume 24. Curran Associates, Inc., 2011. URL https://prooediing neurips.cc/pape.. http:////pppaeee.///e//////aa//a
Michael Betancourt. A conceptual introduction to hamiltonian monte carlo, 2018. URL https://arxx org/abs/1701.0244
Christopher Bishop and Nasser Nasrabadi. Pallern recognition and machine learning, volume 4, chapter 4.4.1. Springer, 2006.
David M Blei, Alp Kucukelbir, and Jon D McAuliffe, Variational inference: A review for statisticians. Journal of the American Statistical Association, 112(518):859-8777 2017.
C. Blundell, J. Cornebise, K. Kavukeuoglu, and D. Wierstra. Weight uncertainty in neural network. In Proceedings of the International Conference Machine Learning, 2015.
15

---

Published in Transactions on Machine Learning Research (03/2026)
N. Bosch, J. Grosse, P. Hennig, A. Kristiadi, M. Pförtner, J. Schmidt, F. Schneider, L. Tatzel, and J. Wenger. Numerics of machine learning. Technical report, Tübingen AT Center, 2022. URL wwwrriicccnmemiicc.cmmm 2raarriss.rrreeeccnnn222222222 _tte_/ueeee_______ of Machine BtrchineLLearing//
Kouroche Bouchiat, Alexander Immer, Hugo Yèche, and Vincent Fortuin. Linearized Laplace inference in neural additive models. In Fifth Symposium on Advances in Approrimate Bayesian Inference, 2023.
Pope D. Brooks, Thomas and Michael Marcolini. Airfoil Self-Noise. UCI Machine Learning Repository, /tt////o...o/1.244422222 CC5W22C.
1989. DOI:
Michele Caprio. Optimal transport for e-contaminated credal sets: To the memory of Sayan Mukherjee, 2025. URL httpp:://arravv.or///bbbs /410. 03267
Michele Caprio and Sayan Mukherjee. Ergodic theorems for dynamic imprecise probability kinematics. International Journal of Approzimate Reasoning, 152:325 343, 2023.
Michele Caprio and Teddy Seidenfeld. Constriction for sets of probabilitics. In Enrique Miranda, Ignacio Montes, Erik Quaeghebeur, and Barbara Vantaggi (eds.), Proceedings of the Thirteenth International Symposium on Imprecise Probability: Theories and Applications, volume 215 of Proceedings of Machine Learning Research, pp. 84-95. PMLR, 11-14 Jul 2023. URL https: httpps://proceedings. mmrtiinssmm......mmmmmmmmm http:/iings...r..pss////15. press/v215/
caprio23b. html
Michele Caprio, Souradeep Dutta, Kuk Jin Jang, Vivian Lin, Radoslav Ivanov, Oleg Sokolsky, and Insup Lee. Credal Bayesian deep learning. Transactions on Machine Learning Research, 2024. ISSN 2835-8856.
URL httpp:://penrrvvier httprrrevviieeeeefornrnerree Adw-
Michele Caprio, David Stutz, Shuo Li, and Arnaud Doucet. Conformalized credal regions for classification with ambiguous ground truth. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL
httpps://openreviev. htt.://wwww...eeeeeomoorrreee//e
Olivier Chapelle, Bernhard Schölkopf, and Alexander Zien (eds.). Semi-superoised learning. MIT Press, 2006.
Tristan Cinquin, Marvin Pförtner, Vincent Fortuin, Philipp Hennig, and Robert Bamler. Lap-aaplace: Funetion-space priors for the Laplace approximation in Bayesian deep learning. Advances in Newral Information Processing Systems, 37:13897-13926, 2024.
Adam D Cobb. hamiltorch: A pytorch-based library for hamiltonian monte carlo. In Proceedings of Cyber- Physicol Systems and Internet of Things Week 2023, pp. 114 115. 2023a.
Adam D Cobb. hamiltorch: A pytorch-based library for Hamiltonian Monte Carlo. In Proceedings of Cyber- Physical Systems and Internet of Things Week 2023, CPS-IoT Week 223, pp. 114-115, New York, NY, USA, 2023b. Association for Computing Machinery. ISBN 9798400700491. doi: (0.1144//576911433557888..
URL https://doi. org/ 10. 1141111751//111.... 3587528
Cerdeira A. Almcida. F. Matos T. Cortez, Paulo and J. Rois. Wine Quality. UCI Machine Learning Reposi- tory, 2009. DOI: htlps:///doi.orr//0.22/// 111.002//24222555333
Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. Laplace redux-effortless Bayesian deep learning. Advances in Neural Information Processing Systems, 34:20089-200 201033, 2021a.
Erik Daxberger, Eric Nalisnick, James U Allingham, Javier Antorán, and José Miguel Hernández-Lobato. Bayesian deep learning via subnetwork inference. In International Conference on Machine Learning, pp. PMLR, 2021b.
Stefan Depeweg, Jose-Miguel Hernandez-Lobato, Finale Doshi-Velez, and Steffen Udluft. Decomposition uncertainty in Bayesian deep learning for efficient and risk-sensitive learning. In Jennifer Dy and Andreas Krause (cds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 1184-1193. PMLR, 10-15 Jul 2018. URL https:
amw.iiinnnmmmm.m...mmmmmmraere mlr. //tproeeeiggggllllppesss///////p html
16

---

Published in Transactions on Machine Learning Research (03/2026)
Stefan Dietrich, Julian Rodemann, and Christoph Jansen. Semi-supervised learning guided by the generalized Bayes rule under soft revision. In International Conference on Soft Methods in Probabilily and Stalistics. pp. 110 117. Springer, 2024.
Souradcep Dutta, Michele Caprio, Vivian Lin, Matthew Cleaveland, Kuk Jin Jang, Ivan Ruchkin, Oleg Sokolsky, and Insup Lec. Distributionally robust statistical verification with imprecise neural networks. Tn Proceedings of the 28th ACM International Conference on Hybrid Systems: Compulation and Control, HSCC '25, New York, NY, USA, 2025. Association for Computing Machinery. ISBN 9798400715044, doi:
URL https:///oi.orgg 011.://000/11111/771116666601 1145/3716666 3718040
Runa Eschenhagen, Alexander Immer, Richard Turner, Frank Schneider, and Philipp Hennig. Kronocker- factored approximale curvalure for modern neural network architectures. Adeances Newrul Information Processing Systems, 36:33624 33655, 2023.
Michael Evans and Hadas Moshonov. Checking for prior-data conflict. Bayesian analysis, 14):893-90 914,,
2006.
Hadi Fanaee-T.
Bike Sharing.
UCI
Machine
Learning
Repository, 2013.
DOL:
https://doi.org. wwtp://///10222////////03
Anrdina Ferreira Arthur Ferreira Aleister Ferreira, Ricardo and Renato Sassi. Daily Demand Forecasting Orders. UCT Machine Learning Repasitory, 2016. DOT: htt////o...rr/10.224////////
Edwin Fong, Chris Holmes, and Stephen G Walker. Martingale posterior distributions. Journal of the Royal Statistical Society Series Statistical Methodology, 85(5):1357-1391, 2023.
F Dan Foresce and Martin T Hagan. Gauss-newton approximation to Bayesian learning. In Proceedings of International Conference on Neural Networks (ICNN'97), volume 3, pp. 1930 1935. IEEE, 1997.
Peter I. Frazier. A tutorial on Bayesian optimization, 2018. URL https://arxiv.ot wwwpa/wwwwwrrr/////./.22222
Christian Fröhlich. Imprecise Probabilities in Machine Learning: Structure and Semantics. PhD thesis, University of Tübbingen, 2025.
Y. Gal and Z. Ghahramani. Dropout as Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning, 2016.
Carl Friedrich Gauß. Theoria motus corporum coelestium in sectionibus conicis solem ambientium, volume FA Perthes, 1877.
Mark N Gibbs. Bayesion Gaussian processes for regression and classification. PhD thesis, Citesoer, 1998.
James Harrison, Andrew Y. K. Foong, and Richard E. Turner. Variational bayesian last layers. arXiv preprinl Xrx::22404.11 11599, 2024. URL httph:tt///// http:////aaxw...rr/a//////00 wwtp/ba////4. 11599
José Miguel Hernández-Lobate and Ryan P. Adams. Probabilistic backpropagation for scalable learning of bayesian neural networks. In Proceedings of the 32nd International Conference Machine Learn- ing (ICML), pp. 1861-1869. PMLR, 2015. URL http:://hhhl.ooooppppooootttttt
pbp- icm www.immmm1111111.m 2015. pdf
I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, Mohamed, and A. Lerchner. beta- vae: Learning basic visual concepts with constrained variational framework. In Proceedings of the International Conference on Learning Representations (ICLR), 2017.
Marius Hobbhahn, Agustinus Kristiadi, and Philipp Hennig. Fast predictive uncertainty for classification with Bayesian deep networks. In Uncertainty in Artificial Intelligence, pp. 222- PMLR, 2022.
M. D. Hoffman and A. Gelman. The no-u-turn sampler: Adaptively setting path lengths in Hamiltonian
Monte Carlo. Tn Journal of Machine Learning Research, 2014.
17

---

Published in Transactions on Machine Learning Research (03/2026)
Alexander Immer, Matthias Bauer, Vincent Fortuin, Gunnar Rätsch, and Mohammad Emtiyaz Khan. Scalable marginal likelihood estimation for model selection in deep learning, 2021a. URL https:
//arxiv. org. //aratv.org//as/2204...0 httg:/bs/22144.09990 04975
Alexander Immer, Maciej Korzepa, and Matthias Bauer. Improving predictions of Bayesian neural nets via
local linearization. In International conference on artificiol intelligence and statistics, pp. 703 711. PMLR,
2021b.
Christoph Jansen. Contributions to the decision theoretic foundations of machine learning and robust statistics under weakly structured information. arXiv preprint arXie:2501.10195 2025.
Christoph Jansen, Malte Nalenz, Georg Schollmeyer, and Thomas Augustin. Statistical comparisons of classifiers by generalized stochastic dominance. arXiv preprint arXiv:2209, 0010:2709.01177,, 2022a.
Christoph Janson, Georg Schollmeyer, and Thomas Augustin. Quantifying degrees c-admissibility in deci- sion making with imprecise probabilities. In Reflections on the Foundations of Probabilily and Statislics: Essays in Honor of Teddy Seidenfeld, pp. 319 346. Springer, 2022b.
Christoph Jansen, Georg Schollmeyer, Hannah Blocher, Julian Rodemann, and Thomas Augustin. Robust statistical comparison of random variables with locally varying scale of measurement. In Uncertainty in Artificial Intelligence (UAI). PMLR, 2023.
Christoph Jansen, Georg Schollmeyer, Julian Rodemann, Hannah Blocher, and Thomas Augustin. Statistical multicriteria benchmarking via the GSD-front. In Neural Informalion Processing Systems, volume 37, pp. 98143 191199 2024a.
Christoph Jansen, Georg Schollmeyer, Julian Rodemann, Hannah Blocher, and Thomas Augustin. Statistical multicriteria benchmarking via the ged-front. Advances in Newral Information Processing Systems, 37: 98143-98178 2024b.
Longlong Jing and Yingli Tian. Self-supervised visual feature learning with deep neural networks: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(11):4037-4055, 2021.
RE Kass, I. Tierney, and JB Kadane. The validity of posterior expansions based on Laplace's method." essays in honor of george barnard, eds, geisser, js hodges, 1990.
Ramnect Kaur, Xiayan Ji, Souradeep Dutta, Michele Caprio, Yahan Yang, Elena Bernardis, Oleg Sokolsky, and Insup Lee. Using semantic information for defining and detecting ood inputs, 2023. URL https: //axxiv. org/ hte///bb/222.... 11019
Markelle Kelly, Rachel Longjohn, and Kolby Nottingham. The uci machine learning repository, 2025. URL https://archive ics. uci.edu Last Accessed: 19.01.2025, For citation see https://archivve ics. uci
edu/citaati cittciit
D. Kingma and M. Welling, Auto-encoding variational bayes. Proceedings of the International Conference on Learning Representations (ICLR), 2014.
Sadanori Konishi and Genshiro Kitagawa. Information criteria and statistical modeling, Springer, 2008.
Agustinus Kristiadi, Matthias Hein, and Philipp Hennig, Being baycsian, even just a bit, fixes overconfidence in relu networks. In International conference on machine learning, pp. 5436 5446. PMLR, 2020.
Agustinus Kristiadi, Runa Eschenhagen, and Philipp Hennig. Posterior refinement improves sample efficiency in Bayesian neural networks. Advances in Neural Information Processing Systems, 14:30333-303444 2022.
Tomasz M. Lapinski. Multivariate Laplace's approximation with estimated error and application to limit thoorems. Journal of Approrimation Theory, 248 105305, 2019. ISSN 0021-9045. doi: https: //doi.org/10..11//jjjp 00010000000000..0000000003000000 $0021904519301108
URL http:///www.cccaaeetttttoooooooa
18

---

Published in Transactions on Machine Learning Research (03/2026)
PS Laplace. Mémoires de mathématique et de physique, tome sixieme [memoir on the probability of causes of events.]. Stalistical Science, pp. 1666-66, 1986.
Hyungi Lee, Eunggu Yun, Giung Nam, Edwin Fong, and Juho Lee. Martingale posterior neural processes. arXiv preprint arXim:2004.00p 2023.
Jongseok Lee, Matthias Humt, Jianxiang Feng, and Rudolph Triebel. Estimating model uncertainty of neural networks in sparse information form. In International Conference on Machine Learning, pp. 5702-5713. PMLR, 2020.
Nils Lehmann, Jakob Gawlikowski, Adam J. Stewart, Vytautas Jancauskas, Stefan Depeweg, Eric Nalisnick, and Nina M. Gottschling. Lightning UQ Box: A comprehensive framework for uncertainty quantification in deep learning. arXiv preprint arxix:24100055 03390, 2024.
Fernando Llorente, Luca Martino, David Delgado, and Javier Lopez-Santiago. Marginal likelihood compu- tation for model selection and hypothesis testing: an extensive review. SIAM Review, 65(1):3 58, 2023.
Dhruv Lowe, Terrence Sanvictores, Muhammad Zubair, and Savio John. Alkaline Phosphatase. Treasure Island (FL): StatPearls Publishing, 2025. URL
www.wncciimmm.mm.......m.m. mlm.nih. mih. geer/bookks/BKK4590001
Pengyuan Lu, Michele Caprio, Eric Eaton, and Insup Lee, lbel: Zero-shot model generation under stability- plasticity trade-offs, 2024. URL btatpsss//ardt..org/abb///////// 14782
Alexander Marquardt, Julian Rodemann, and Thomas Augustin. An empirical study of prior-data conflicts in Bayesian neural networks, 2023. Poster presented at the International Symposium on Imprecise Probabil- ity: Theories and Applications (ISIPTA). Available at https: sipta. wwg/acceptee-pppere. short-marquard/ (last accessed October 29 2025).
James Martens and Roger Grosse. Optimizing neural networks with kronecker-factored approximate curva-
ture. In International conference machine learning, pp. 1408-2417, PMLR, 2015.
Sara Martino and Andrea Riebler. Integrated nested Laplace approximations (inla). arXiv preprint arXiv:1907.01248, 2019.
Peter David Miller. Applied asymptotic analysis, volume 75. American Mathematical Soc., 2006.
Tom Minka. Divergence measures and message passing. Technical Report MSR-TR-2005-
173, January 2005. CRL wttp:///wwwmmmcooo....mmmm htt.ossottccooo////oooommooo htt.c///rrrn///eeeerraaa/aaaaaaa
A. Mnih and K. Gregor. Neural variational inference and learning in belief networks. In Proceedings of the
31st International Conference on Machine Learning, 2014.
Hassan Moin, Hafiz Zeeshan Iqbal Khan, Surrayya Mobeen, and Jamshed Riaz. Airfoil's acrodynamic coefficients prediction using artificial neural network. In 2022 19th International Blaurban Conference on Applied Sciences and Technology (IBCAST), pp. 175-182. IEEE, August 2022, doi: (0.1109/ibeast5485) 002219990112. URL http: //tx /wtxxxdooi..oo www.:/ooo...r///o//.... 10. IBCAAAS5545
Blake Moya and Stephen G Walker. Martingale posterior distributions for time-series models. Statistical Science, :081):68-2) 80,88800222 2025.
R. M. Neal. Probabilistic inference using Markov chain monte carlo methods. In Technical Report CRG- TR-93-1 University of Toronto, 1993.
Luis A. Ortega, Simon Rodriguez Santana, and Daniel Hernándezz-Lobato Variational linearized Laplace approximation for Bayesian deep learning. In Ruslan Salakhutdinov,, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 38815- 38836. PMLRR 21 27 Jul 2024. URL htttp:://proceeddngs... .ll.. pptps//e2sss////e/ preass/233/ortega22222 ortps//eeeggaa.a.. htmm
/
19

---

Published in Transactions on Machine Learning Research (03/2026)
Theodore Papamarkou, Jacob Hinkle, M Todd Young, and David Womble. Challenges in Markov chain monte carlo for Bayesian neural networks. Statistical Science, 37(3):425-444 442, 2022.
Zhao Peng. Multilayer perceptron algebra, 2017. URL
Arya A. Pourzanjani and Linda R. Petzold. Implicit hamiltonian monte carlo for sampling multiscale dis- tributions, 2019. URL http:://aarrav.orrr//////1111 /
R. Quinlan, Auto MPG. UCI Machine Learning Repository, 1993. DOL: https://ddoi.org11..22444 10.24402/2202 C5859H
R. Ranganath, S. Gerrish, and D. Blei. Black box variational inference. In Proceedings of the 17th Interna- tional Conference Artificial Intelligence and Statistics, 2014.
Hippolyt Ritter, Alcksandar Botev, and David Barber. Online structured Laplace approximations for over- coming calastrophic forgetting. Advances in Neural Information Processing Systems, 31, 2018a.
Hippolyt Ritter, Aleksandar Botev, and David Barber. A scalable Laplace approximation for neural networks. In 6th International Conference on Learning Represenlations, ICLR 2018- C018-Conference Track Proceedings, volume 6. International Conference on Reprosentation Learning, 2018b.
Julian Rodemann. Pseudo label selection is a decision problem. In Proceedings of the 46th German Conference on Artificial Intelligence. Springer, 2023.
Julian Rodemann. Towards Bayesian data selection. ICML Workshop on Data- Centaic Machine Learning Research (DMLR) at ICML 2024, 2024.
Julian Rodemann and Thomas Augustin. Accounting for imprecision of model specification in Bayesian optimization, 2021. Poster presented at the International Symposium on Imprecise Probabilities (ISIPTA). Sympooo
Available at hhtts:://ssippat www.sstpptaaaaaaa..eo orggabsttaatts/1_9-00 .ddff (last accessed October 29 2025).
Julian Rodemann and Thomas Augustin. Accounting for Gaussian process imprecision in Bayesian opti- mization. In International Symposium on Integrated Uncertainty in Knowledge Modelling and Decision Making (IUKM), pp. 22-104. Springer, 2022.
Julian Rodemann and Thomas Augustin. Imprecise Bayesian optimization. Knowledge-Base Based Systems, 300: 112186, 2024.
Julian Rodemann and James Bailie, Generalization bounds and stopping rules for learning with self-selected data. arXiv preprint arXi::2505.07335 (last accessed October 29 2025). 2025.
Julian Rodemann and Hannah Blocher. Partial rankings of optimizers. arXiv preprint arXiv:2402.1666, 2024.
Julian Rodemaun, Dominik Kreiss, Eyke Hüllermeier, and Thomas Augustin. Levelwise data disambigus- tion by cautious superset classification. In Florence Dupin de Saint-Cyr, Meltem Detürk-Escoffi and Nico Potyka. (eds.), Scalable Uncertainty Management 15th International Conference, SUM 2022, Paris, France, October 17-19, 2022, Proceedings, volume 13562 of Lecture Notes in Computer Science, pp. 263 276. Springer, 2022. doi: 10.1007/977-33-31--8884-- 18. URI. https://doi. org/10.1007/ http::://ao..or//1..0077/ 088---8831888--- 5_18
Julian Rodemann, Thomas Augustin, and Rianne De Heide. Interpreting generalized Bayesian inference by generalized Bayesian inference, 2023a. Poster presented at the Thirteenth International Symposium Imprecise Probabilities (ISIPTA). Available at https://isipta23. sipta. org//acooteedpppperss short-rodemann/ (last accessed October 29 2025).
Julian Rodemann, Jann Goschenhofer, Emilio Dorigatti, Thomas Nagler, and Thomas Augustin. Approxi- mately Baycs-optimal pscudo-label selection. In Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, volume 216 of Proceedings of Machine Learning Research, pp. 1762-1773, 2023b. URL httpps://proceedinggs http:/cceeiaaaaaa..eeeeeee/////s wttpppss////////eeemma////// htmml
20

---

Published in Transactions on Machine Learning Research (03/2026)
Julian Rodemann, Christoph Jansen, Georg Schollmeyer, and Thomas Augustin. In all likelihoods: Robust selection of pseudo-labeled data. In International Symposium on Imprecise Probabilities Theories and Applications (ISIPTA). PMLR, 2023c.
Julian Rodemann, Christoph Jansen, and Georg Schollmeyer. Reciprocal learning. Advances in Neural Information Processing Systems, 37:1686-1724, 2024.
Hávard Rue, Sara Martino, and Nicolas Chopin. Approximate Bayesian inference for latent gaussian models by using integrated nested Laplace approximations. Journal of the Royal Statistical Society: Series B (Stalistical Melhodology), (1):3311 -92, 2009.
Gideon Schwarz. Estimating the dimension of a model. The Annals of Statistics, 6(2):461-460) 1978.
Emanuel Sommer, Lisa Wimmer, Theodore Papamarkou, Ludwig Bothmann, Bernd Bischl, and David Rügamer. Connecting the dots: Is mode-connectednnss the key to feasible sample-based inference in Bayesian neural networks? arXiv preprint Xiix24422001488 2024.
Emanuel Sommer, Jakob Robnik, Giorgi Nozadze, Uros Seljak, and David Rügamer. Microcanonical langevin ensembles: Advancing the sampling of Bayesian neural networks. arXiv preprint urXiv:2502063338 2025.
David J Spiegelhalter and Steffen L Lauritzen. Sequential updating of conditional probabilitios on directed graphical structures. Networks, 20(5):579 605, 1990.
Brian Staber and Sébastien Da Veiga. Benchmarking Bayesian neural networks and evaluation metrics for regression tasks, 2023. URL
Luke Tierney and Joseph B Kadane. Accurate approximations for posterior moments and marginal densities. Journal of the Americon Statistical Association, 81(393).82 86, 1986.
Matthias C. M. Troffaes and Gert de Cooman. Lower Previsions. Wiley Series in Probability and Statistics. John Wiley & Sons, Chichester, UK, edition, 2014. ISBN 978-0-470-72377-7. doi: 10.1002/97811118722222 URL
M Antónia Amaral Turkman, Carlos Daniel Paulino, and Peter Müller. Computational Bayesian statistics: an introduction, volume 11. Cambridge University Press, 2019.
Peter Walley, Statistical reasoning with imprecise probabilities. Chapman & Hall, 1991.
Gero Walter and Thomas Augustin. Imprecision and prior-data conflict in generalized Bayesian inference. Journal of Statistical Theory and Practice, 3(1):255-271, 2009.
James Watson, Andrew Y. K. Foong, Ryan P. Adams, and Richard E. Turner. Latent derivative bayesian last layer networks. In Proceedings of the 38th International Conference on Machine Learning (ICML), pp. PMLR,, 2021. URL hntpps://proceeddnng: mmr...mmm.m......mmm.mm....... prop:p//////////ttoooo/////oooo
pdf
M. Welling and Y. W. Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th International Conference on Neural Information Processing Systems, 2011. URL https: //ww..cB toronto.
Jonas Grogor Wiese, Lisa Wimmer, Theodore Papamarkou, Bernd Bischl, Stephan Günnemann, and David Rügamer. Towards elficient meme sampling in Bayesian neural networks by exploiting symmetry. In Joinl European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD) 2023, volume 14169 Machine Learning and Knowledge Discovery in Databases: Research Track, pp. 459-474. Springer Nature, 2023. doi: 10.1007 017777978-----88 00.1007/998-33081-44441222
Anqi Wu, Jingwei Zhang, Qing Li, and Weiming Hu. Deterministic variational inference for robust bayesian neural networks. arXiv preprint arXiv:1810.0900 2018. URL
03958
I-Cheng Yeh. Concrete Compressive Strength.
UCI Machine Learning Repository, 1998. DOL:
21

---

Published in Transactions on Machine Learning Research (03/2026)
I-Cheng Ych. Real Estate Valuation.
UCI
Machine
Learning Repository, 2018.
DOI:
https://do
Xiaojin Jerry Zhu and Andrew B. Goldberg. Introduction to semi-supervised learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 3(1):1-130, 2009.
A Proofs
Proof of Lemma
By definition, & and 0 satisfy
Vln(O) + v log (0) O, Vε(0) + V log (0) 0,
where {(0) = ( + logr(0++1 In 1, 0). Subtracting the two equations and applying a first-order Taylor expansion of the gradicnts around yields
0 vi(0) + V log ग (0) vē(0) (o) J(0) 0),
for some on the line segment between and
By Assumptions (A1)-(A3), J(Ø) is positive definite with eigenvalues bounded below, hence invertible with uniformly bounded inverse. Moreover,
Vē(0) + log (() (l)(O) + Vlog + logp(0at1 In+1. 0) = =V 1a+1, 0),
which is O(1).
Since [(() = (n), it follows that
0- 0 - -J(0= logp(0n-1 xn+1, - Onn
which proves the claim.
Proof of Theorem 7 We work under assumptions (A1) (A3):
(A1) (0) (0) with each twice continuously differentiable in a neighborhood of ê.
(A2) The observed curvature JD( is positive definite with eigenvalues bounded below by Amin 0.
(A3) The prior log-density log (0) is twice continuously differentiable with bounded gradient and Hessian in neighborhood of 0.
In addition, we use the Lipschitz condition on the single-point term stated in Theorem 1.
First, notice that & contains f, that is,
ε(0) == (140-114441) E()).
In order to simplify the computation of ?(0), we expand around its maximizer 0, so that
(() (()) + 0110 012)
(17)
By Lemma we know that since D U ( (n+1)) and D differ in only one sample, the difference i 0 is order O(n-1) Combining this with equation 17 we have
1(0) (1m.t).0m-)) (0) + -2),
which entails
22

---

Published in Transactions on Machine Learning Research (03/2026)
E(O) 2 l (mn - e(o).
(18)
We then ask ourselves (1 can be approximated by (14)))) Recall that the difference @ - i is of order D(n-1), as DU (u{(1=+1 Bo+1)} and D differ in only one sample. In addition, the likelihood function ((n++1.244))) is Lipschitz-contimuuus in 0. This holds per assumption of Tipschitz-contimmour loss.
Now consider the Lipschitz bound
( (10))1 - L. 110 1|1,,
(19)
where L is Lipschitz constant. Exploiting the fact that 0 - 0 is of order O(n Equation equation 19 entails that
4
(1.44) (en41 (n.)) +
(20)
where the remainder is negligible as + Hence, equation 20 implies that
(( ()) & (1m))1.b.))(0)))
(21)
Combining equation 21 with equation 18 we have
E(0) (0))) E(θ)
(22)
and, by-product,
((0) ~ E(O).
We express Equation [22] E(θ) E(θ), equivalently in asymptotic (Bachmann-Landauu notation
+ +0 3)= ==(=)))- + 0 {()) = E()) 0 1
( Ln + n2
by combining equations
20 and
18
Proof of Corollary 7 This corollary is proved under the same regularity conditions as Theorem 1, namely (A1) (A3). Recall from the proof of Theorem ! that contains fp. Thus the same holds for their derivatives if they exist. Further recall that DU ( In+1)) and D difler in only one sample. So the difference 0-0 is of order On--)). By requiring Tipschitz-comtinuity for the second derivatives and the prior, we can apply the same reasoning as in the proof of Theorem [ Hence, it follows that
[L ( n2
--'(0//
and
Ln + 1 n²
T00 = -()) + 0
23

---

Published in Transactions on Machine Learning Research (03/2026)
simply
=(0)= - -O)(n) -0)(0)/n and -(0) ~ T(0).
Proof of Corollary Immediate from Theorem and Corollary
1
A.1 Highest Density Region Approximation
Notice that the integral Se exp[-3 -000--00000000000-00000000000000 00 in equation is Gaussian integral, and 0 follows multivariate normal distribution (Kass et al. 1990). In addition, is well-known that any marginal distribution of multivariate normal distribution is again multivariate normal. As consequence, our approximation of p(8++ 1nt1, D) is multivariate normal and thus symmetric around its mode. This implies R( is symmetric around given 1 E y, i.e., R(f²) = b, 9 + b). This simplifies the solution for pa. We have
+ J
Plžnt1 E R(g²) Int1, D] 211
p(Dnt1 1at1, D)dy 2
m(pr)
1
8+6
p(Ont1 Dntt,DDdy 2
8-b
Approximating p(dn+1 | 1x+1, by Equation equation 12 we obtain
0+b
exp
log
n
ē(0)
1
log [z(0]] log Itp (0) - - + log (0)=(()- log n m0)) dy
h-
2
1
2
04b
exp {(0)dy
h-h
(1 a) exp
q log
+ log [(0)
1 log Itp (0) + ( 10g log (00
n2 + 1
84b
Lg.2 (0, y. x)dy
b-b
(1
(x) exp
log
log [(6]
log ITe (0) + (0) log log 7 (0)
(23)
n n+1
2
24

---

Published in Transactions on Machine Learning Research (03/2026)
/++
exp
30
log
2m
ε(σ)
1
log [z(0)] log ITp (0) + (0) + 3 log π (0) dy
8-b
2
2
94b
exp d())dy
6-b
39 log
2n
2 a) exp
+
log [I(0]] log ITe (0) 2) 3log m(0)
2
184
fg.z (0, y. dyy
8-b
2 - a)- exp
39
log
2π
+
log [I(Ø)] log [Te (0) - 20 3 log (0)
(24)
2
2
where 1j = (y),..., Mn. 1m+1) and T = Iny fn+1). Note that per Lemma we have log (()
(A) = 0-
Recall that we selected the 12-soss. Then, we have that
10 C(B, y, x.dy
g0+
(y -((A dy
8-b
8-b
(0 b- [xx 01)3
-3 - -b- ((x
1
1 b)} = 2 2/3
(25)
3
Plugging equation
25 into equation
24
get that b is lower bounded by
3 (1 a) exp
3q
2:T
log
ē(() + log [[(] + log Ie, (θ) 2l - 3log 7 (
(26)
2
2
Since we are looking for the smallest possible region R(p"), we want the smallest possible value of Such smallest. possible value is exactly the lower bound in equation 26 which we denote by b².
25

---

Published in Transactions on Machine Learning Research (03/2026)
We can now derive p":
f = p(0++1 = - ++ 6" In+1, D)
39
2T
0 + b² - -S(Ô, x)
exp
log
log [I(Ø) log [z. + (0) + 3 x (0)
2
2
3q
2T
1
exp
log
log
[I(O)]
log Itp (0) + (6) + m())
2
2
3 (1
31
1m
1
a) exp
log
log [I(6) + log II,, (0) -0 3 log m(0)
2
n
2
exp
34 log
2m
1
log
[I(0)]
log [T,, (0) (0) + 3log m()]
2
n
2
3
Q
2m
1
log [I(0] log ITo (0) - _ 3 - log m0))
V
a) exp
2
log
+ 6
6
exp
3q
log
2π
1
log
[ī(0]
- log ITp (0)) (0) + 3 m()
2
71
2
3
2T n
1 3
log Itp (0) - 3 =ED(0)- 2 2l0 (0)
V 2
a)
exp
y log
+ log [I(0] +
exp
3q
log
2m
1
log [ī(Ō]]
- log Itp (0)) + 20) + 3og T m())
2
n1
2
9
(1
a) exp
q
log
2T
log [((0]
log Iep (0) +
2
(0) - log π (0)
2
6
3
3
Now that we found pr, we focus on finding P2 We have
logp((n+1 In+1, D) log(p²)
39 log
1x
1 _ log [I())
{(0)= log(f")
log x (0) +
n
+ log ITe (0) - p (
- 2 log क (0) -
39
2T
log yy 100,x)1 2 log(Ø")
log
log π (0) + こ log [0)
2
+ log
ITp (0) p 2 log n (0).
By Lemma]
1 this entails that
39
1nT
log y - 200,))) 2 log(0a)
log
loo (0)+ log I(0)
2
n
+ log Itp (0) - p
- 2 log 7 (0)
39
2xT
= =log(0")
log
+ 2
log [[0) + log (0)] 101-221 2k) -3log (())
2
Notice that thanks to our choice of the loss, we have that both Fisher information matrices I(0) and Ie, (0) do not depend on Pu+1. Similarly, the normalization term 3g
2 log( 2x and the prior values log (() and log (0) do not depend on 0 The Highest Density Region (HDR), then, given by
Neither directly through y nor indirectly through ē arg max, E(O)-
26

---

Published in Transactions on Machine Learning Research (03/2026)
R(6²) y eyy::p 1 -0((00
39
2T
1
(27)
60 exp
log
log [I(0)] log ITp -0) 20) log (0)
2
n
2
where we group in с constants that do not depend on the prior. We can rewrite the expression for pa we derived before as
0 v a) exp
log (0)
3
Plugging this value in equation 27 we obtain p2:
v/211 -a) exp [c =llog((0) log x (0)
22 f² exp
-ooo (0)
3/ 9 (1 3)3 exp
4c
2 log n m(6)
V
which concludes the argument. The HDR, then, is
9
4c
R(p)) y EV:: 1
a) (1 exp
- 2 2og क T(6)
3
or, equivalently,
1c 3
R(p²) y y: 1 =(0, exx
z
V
7
exp
(0)
Further Details on Experiments
We evaluate our method across synthetic and roal-world benchmarks, comparing against strong probabilistic and deterministic baselines. Metrics cover both accuracy and uncertainty quality, including negative log- likelihood, Brier score, expected calibration error, and aut-of-distribution (OOD) detection (via AUROC) We ablate the contribution of cach component of the approach and study sensitivity to hyperparameters and model scale. Robustness is assessed under distribution shift and label noise, and we report computational overhead Beaining//nference time and memory) relative to baselines. Overall, the method consistently matches or exceeds baseline accuracy while delivering better-calibrated uncertainties and competitive OOD performance modest additional cost. Code reproduce the experimental setup can be found at https: //github. //gittub..com/rodenan///// sssla
B.1 Conjugate Prior Analysis
Conjugate Prior Analysis allows compare (A)SSLAs performance against the analytic PPD.
In the following, we brielly state each of the models and provide the experimental insights.
For modern treatment of OOD semantic information, we refer the interested reader Kaur al. 2023
27

---

Published in Transactions on Machine Learning Research (03/2026)
B.1.1 Normal-Norme Normal Model
Consider a set of observations Xi...., Xn ~ NN,, (ree) where orrg is assumed to be known. The prior for the mean parameter μ defined as N M(Jo TO), where μl and To are also known. The posterior predictive distribution for a new observation Xnews given X, is then defined
new X NN on irue))
(28)
where
Mu Σxi Oirue
0
irue
0
2
;
An =
on
(29)
In experiment, we employ a true data-generating process that assumes parameter values of Ptrue 4.0
and Otrue v2.0, and the prior distribution is specified N(po 4.0, 1.0). Observations are sampled from ((ttrue: drwe), and the posterior predictive distribution is computed analytically according to Equation 28
For SSLA and ASSLA, the log-likelihood and observed Fisher information matrix play a critical role in the approximations. Under our assumptions, the observations X, M(trruer strue) for {l...., nu are independent and identically distributed. The log-likelihood is given by:
{(p) = log ((u) log II
f(()m)) = Σ
log f(X.Ip)
1=1
1=1
(30)
- log (og=²
Σ (X, - 22) 20
The observed Fisher information is defined as the negative second derivative of the lieliikel:ooo::
J(H) Ol(u) 8 (X, n)
(31)
8u. 8u 0²
Figure provides comparison of the approximation quality of SSLA and ASSLA across various sample sizes (n = 20....,.7 1.000.000). For most cases (n = 20,..., 100.000), both SSLA and ASSLA closcly match the analytic posterior predictive distribution. This is further validated by the measured entropy between each approximation method and the analytic posterior predictive distribution, which remains close to zero across all cases. These results confirm the ability of both methods to reconstruct the true distribution effectively.
B.1.2 Poisson-Gamma Model
Let X1....,y XN ~ Poi(A): X ~ Gammala, 8). Then A/. ~ Gamma Σ X; + ta, + 3). The posterior predictive distribution for a new observation Xnew follows Negative Binomial distribution:
Xnow! X ~ Bin (r=Ix-)))
n+ Σ X n ++++1
with the definition of r being the number of successos, p being the success probability.
For the model setup, we assume a true data generating process of X; ~ Poi(Atrwe) for E (l,..., with a true rate of Atrue = 3.0.
The prior distribution is specified as Gamma distribution À Gammala 6.0, 2.0) where and B correspond to the prior's shape and rate parameters, respectively.
9 low entropy indicates that the approximations do introduce significant information loss relative the analytic solution. An entropy of tells that we either technique interchangeable.
28

---

Published in Transactions on Machine Learning Research (03/2026)
Comparison SSLA Analytic Posterior Predictive Distributions
ASSLA (Entropy=0.00)
SSLA (Entropy=0.00)
Analytic
Posterior Predictive Density (PPD) Posterior Predictive Density (PPD)
100
1000
0.25
0.20
0.15
0.15
0.05
0.05
0.00
10
10
100000
1000000
0.20
0.20
0.30
0.25
0.15
0.20
0.10
0.15
0.10
0.05
0.05
0.00
0.00
10
2
10
2
10
Figure 3: Illustrated are the analytic (green) and approximated (SSLA red, ASSLA black) PPDs for varying sample sizes n, ranging from n = 20 to n = 1.000.000. The results indicate close alignment for smaller and moderate data sizes, while deviations for larger sample sizes (n = 1.000.000) become noticeable for ASSLA.
We again derive the log-likell likeliodldood and the observed Fisher Information which are given by
{(A) Σ (og)))(A) A) Σ (X,!)
(32)
and
Σx, I(A)
(33)
X2
respectively.
The experiment evaluates the approximation quality of SSLA and ASSLA across different sample sizes (n 20...., = 1.000.000), see Figure 3 Observations are sampled from the true Poisson distribution, and the posterior predictive density for a new observation Xnew is computed using the three methods, (a) SSLA, (b) ASSLA, (c) Analytically.
C Heteroscedastic Regression: Further insights
C.1 Architectural Design
We use a Multilayer Perceptron (MLP) (Bshho & Nasrabadi 2006 Peng 2017 as the base model due its ficxibility and effoctiveness in modeling non-lincar relationships.
The network architecture is formally defined as:
((x) fo()) W, act (Wa-1 act(... (Wix + b1)) bn-1) + bn
(34)
29

---

Published in Transactions on Machine Learning Research (03/2026)
for standard regression tasks, and as:
((x) log 02 (x)) = fa(x) W act (Wn-1 act(... (W b1)) + bp-1) bn
(35)
for modeling heteroscedastic uncertainty, where the output consists of both the predicted mean and the logarithm of the variance.
We We employ two hidden layers with 50 neurons each and ReLU activation for modeling non-linear relationships.
C.2 Hyperparameter Optimization
To optimize the architectural design of the network, we use Optuna (Akiba et al. 2019 Optuna is a flexible hyperparameter optimization framework that dynamically constructs the search space and leverages Bayesian Optimization (BO) (Frazier] 2018), specifically using the Tree-structured Parzen Estimator (TPE)
[Bergstra et al. 2011 approach for single-objective optimization.
The following parameters were optimized based on validation loss:
Learning Rate: 10-5, 10.5,..., 10-1
Batch Size: 32, 256
The search was conducted for 100 trials, after which the best hyperparameters were selected.
HMC Hyperparameter Tuning For HMC we tune the following hyperparameters:
Step Size
The size steps laken during the simulation of the Hamillonian dynamics. relates the exploration of the sampler (Cobb 2023a
Description:
Search Space: 10-5 ...., 10
Number of Samples
Description: This reflects the total number of samples generated from the posterior distribution 2000
Search Space:
Number of Steps per Sample
Description:
The number of leapfrog steps taken in the Leapfrog integration technique. For further information leapfrog integration, used in HMC refer the reader to Pourzanjani & Petzold (2019).
Search Space: .0.....50 50
Tau In
Description:
This parameter reflects the precision the prior Coob 2023a
Search Space: 0.1,...,10 10
Tau Out
Description: This parameter reflects likelihood output precision Cobb 2023a
Scarch Space: 10,...,500 500
Mass
In HMC the mass is typically encoded as a mass matrix [etancouut 2018 and directly impacts the step size and trajectory of the leapfrog integrator. hamiltorch, diagonal matrix with scaling factor (i.c. the mass parameter) is used (Cobb. 2023a
Description:
Search Space: O.1,...,
30

---

Published in Transactions on Machine Learning Research (03/2026)
BNN-MFVI
For the BNN with Mean-Fiele Variational Inference approximation the following hyperpa-
rameters are tuned
Burnin Epochs
Description: Represents the number of epochs to train before switching to NLL loss
Search Space: 50,..., 200
Number of MC Samples During Train
Description: The number of MC Samples to draw during training when computing the negative ELBO loss
Search Space: 5,...,50 50
Number of MC Samples During Test
Description: The number of MC Samples to draw during test and prediction
Search Space: ,0,...,100 100
Output Noise Scale
Description: The scale of the predicted sigmas
Search Space: ..
BNN-VI The BNN-VT model is proposed in Depeweg et al. 2018) and requires to the hyperparam- eters for the MLP architecture and the hyperparameters of MFVI additionally:
Alpha
Description: This parameter used to minimize the (a-) divergence (see e.g. Minka 2005 between the variational and the analytic posterior (Depeweg al. 2018).
Search Space: 0,...,1
LA Hyperparameter Tuning The Laplace class provided by Daxberger et al. (021a)) enables auto- matic post-hoc tuning of the prior precision using the marginal likelihood method [Immer et al. 2021a
described by Daxberger et al. (2021a| Chapter 3). We follow their recommendation Daxberger et al. 2021a
Chapter 4.1) in applying post-hoc tuning rather than online training.
C.3 Loss Functions
Depending on the regression task and the UQ method, we employ different loss functions:
Mean Squared Error (MSE) Loss: Used for deterministic regression models
LMSE 1 E (us 1xx:))²
Negative Log-Likelihood (NLL): Used for models predicting both mean and variance
1
Σ
(og(o(z)))) + (y) p(x;))²²
DNLL 2
olx)2
31

---

Published in Transactions on Machine Learning Research (03/2026)
C.4 Training Configuration
Optimizer: Adam with learning rate according to the found learning rate hyperparameter
Training Epochs: We used 500 epochs of training with early stopping after 20 epochs of no improve- ment in the validation loss
Batch Size: The batch size is adapted according to the found hyperparameter
Monte Carlo Samples: For (A)SSLA we used 20 samples to reconstruct the ppd of each observation, varying the response y by small margins around the true predicted
We assume a standard normal distributed prior where applicable
D
Additional Experiments
Tables 5 and 6 have some additional results the Lightning UQ benchmarks (Lehunann el. al., 2024) They comprise the Sine RBF (Radial basis function) setup and compare the homoscedastic case to the heteroscedastic RBF regression case. As expected from the theory, SSLA and ASSLA coincide with classical Laplace-style behavior when refitting effects are negligible, thereby validating the asymptotic approximations underlying ASSLA.
Method
RMSE L
NLL 4
Cov90
Cov95 f
Deterministic (plugin)
0.310 + 0.042
0.316 = 0.234
82.0 + 9.9
91.0 + 9.9
Laplace/ Exact
0.310 0.042
0.307 0.219
85.5
92.0 9.9
ASSLA
0.310 = 0.042
0.316 = 0.235
82.0
91.0 + 9.9
SSLA
0.310 0.042
0.307 = 0.219
85.5
92.0 9.9
Table 5: Lightning UQ Box benchmark: Sine (homoscedastic)
RBF regression.
Method
RMSE 4
NLI. 4
Cov90
Cov95 f
Deterministic (plugin)
0.346 + 0.034
0.420 = 0.1
87.0 4.2
89.5 4.9
Laplace/ Exxct Exact
0.346 + 0.034
0.413 0.167
87.5 + 4.9
90.0 4.2
ASSLA
0.346 + 0.034
0.420 = 0.182
87.0 + 4.2
89.5 4.9
SSLA
0.346 + 0.034
0.413 0.167
87.5 + 4.9
90.0 +: 4.2
Table 6: Lightning UQ Box benchmark: Sine (heteroscedastic)
RBF regression.
E Illustration of prior modularity:
We illustrate the prior modularity on simple 1D linear regression toy problem with n 50 and Gaussian noise 0.5. We evaluate SSLA/ASSLA at Trest 1.5. For each prior (N(07 we compute 0 and the self-prediction d. = fa then evaluate the log-PPD approximation at (A) y = -. and (B) y = û. + 1.0. We include a near-noninformative prior 72 == 106.
32

---

Published in Transactions on Machine Learning Research (03/2026)
Prior
SSLA
ASSLA
NO0 0.01)
-0.0259
0.0259
0.1,
-0.0259
-0.0259
N(O, 1)
-0.0259
-0.0259
(0,, 10)
-0.0259
-0.0259
N(0, 10%)
-0.0259
0..259
Table 7: At the self-prediction = Dr, SSLA and ASSLA coincide (up to constants), since the pseudo- observation induces no predictive "surprise" at 9..
Prior
SSLA
ASSLA
SSLA-ASSLA
SSLA SSLAmminfo
Ni 0.01)
-1.6612
-2.0259
0.0647
-0.0633
N((0,011
-1.9303
-2.0259
0.0956
-0.0054
N(O, 1)
-192255
-2.0259
0.1004
-0.0006
WOO, 10)
-1.9250
-2.0259
0.1009
0.0001
NiO, 10")
-1.9250
-202299
0.1009
0.0000
Table 8: Off the self- relf-prediction f. + 1.0, tightening the prior (smaller variance) changes SSLA, while ASSLA is (by construction) much less sensitive at the approximation order whore the prior increment is dropped.
33


Reviews:
[{"review_id": "uksUpR8c3O", "reviewer": "Reviewer_YXaA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes SSLA (self-supervised Laplace Approximation): approximate the posterior predictive directly rather than the parameter posterior by adding a self-predicted pseudo-observation $(x_{n+1}, \\hat{y}_{n+1})$, refitting, and using a Laplace-style approximation to express the posterior predictive distribution as in equation (10) as differences in log-likelihood, log-prior, and log-determinants of Fisher information term.\nThe paper also proposes ASSLA, an approximation that avoids refitting by replacing the augmented optimum $\\tilde{\\theta}$ with the original optimum $\\hat{\\theta}$, leading to simplified formulas.\nNumerical experiments are provided to verify the approach.", "claims_and_evidence_yn": "No", "claims_explanation": "(1) The approach in equations (6) and (12) of adding a pseudo observation is standard in Bayesian models, which is under the usual conditional independence assumptions. The authors frame this as \"self-supervised\", which is unclear.\n\n(2) I am not sure whether the research gap claimed in this paper, \" there is no established method...\" is true.\n\n(3) Equations (12) contains tha log prior terms, and in equations (16)-(17) the priors are dropped completely, which results in a disconnect between claimed distribution and the method.\n\n(4) The mathematical writing and the proofs are very hand-wavy, for example the proof of lemma 1 and the proof of Theorem 1, etc, the conditions for the claims in the proofs are not discussed, for example, $\\tilde{\\theta} = \\hat{\\theta} + O(n^{-1})$ is stated with no conditions ; continuously differentiable functions are all Lipschitz continuous?.\n\n(5) Is equation (3) correct? \n\n(6) Can the authors clarify if $\\hat{y}_{n+1}$ depends on $D$ or not? The evaluation of the likelihood depends on the independence.", "audience_yn": "Yes", "audience_explanation": "The motivation of bypassing parameter posterior approximations and targeting posterior predictive directly is reasonable. The paper aims to achieve fast posterior approximation without heavy sampling. These are interesting areas for the audience of TMLR.", "requested_changes": "I would appreciate the authors addressing my concerns written above and improving the writing of the methodology and the correctness of the mathematical discussions.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "GK4Mne1koF", "reviewer": "Reviewer_yXWb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces two methods, Supervised Laplace Approximation (SSLA) and Approximate Self-Supervised Laplace Approximation (ASSLA). Both aim to perform approximate Bayesian inference while avoiding the direct computation of the posterior parameter distribution, instead approximating the posterior predictive distribution directly. \nIn the case of SSLA, predictive uncertainty is quantified by refitting the model on an augmented dataset that includes the model’s own predictions. However, this solution has additional overhead because of the refitting phase. Therefore, the authors introduce a second method, ASSLA, that avoids the refitting phase by using asymptotic expansions and local linearization to express the posterior predictive distribution.\n\nThe authors compare their methods with three other approximate Bayesian inference techniques using both a synthetic dataset and datasets from the UCI Machine Learning Repository.\n\nThe paper also discusses some scalability and stability issues. The authors report that ASSLA suffers from numerical instabilities when the sample size exceeds n=1.000.000. SSLA, instead, is unstable on larger datasets and in the presence of prior–data conflicts.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims made in the submission are convincing. The authors clearly explain the proposed methods and the results obtained. They also explain the limitations of their methods, briefly discussing them. I believe the paper would be stronger with a more extensive and in-depth discussion of these limitations.", "audience_yn": "Yes", "audience_explanation": "I believe the contribution is sound and that it could be beneficial for researchers working in the field.", "requested_changes": "- The plots can be improved: (1) the font of the ticks, axes, titles, and legends is too small, the authors should try increasing it. (2) In general, there is a lack of consistency across the plots: Figures 1 and 3 have the legend inside each plot, even though the legend is always the same (and it is also hard to read because of the small font). Figure 3, instead, has the legend on top of the plots. The same issue applies to the plot titles: Figures 1 and 3 have a title, while Figure 2 does not.\n- On page 7, the authors refer to Figure 3. However, this figure is in the appendix. Figure 1, instead, is not referred to in the main paper, but it is referred in the appendix. I guess these two references need to be swapped.\n- On page 7, the authors say that SSLA and ASSLA are able to recover the analytical PPD almost perfectly for sample sizes between n=20 and n=100,000. Then, in the same line, they state that numerical instabilities emerge for n >= 1,000,000. What happens in the range between 100,000 and 1,000,000?\n- The authors should elaborate more on the numerical instability issues.\n- The authors state that ASSLA balances computational tractability with competitive probabilistic calibration. At the same time, in Section 5.1, the experiments show that it underestimates uncertainty over wider intervals, leading to a risk-seeking bias. Would it be possible to elaborate more on this?\n- In Section C.2 (page 26 in the appendix), the authors report the tuned hyperparameters. The values of these parameters are written as “1e-5”, I am not sure whether the LaTeX was rendered correctly here.\n- In Section C.2 of the appendix, there are some question marks inside parentheses (?) at the end of some sentences.", "broader_impact_concerns": "I do not have any ethical concerns about this work", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "upMbIdgIpZ", "reviewer": "Reviewer_z9r6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposes the Laplace Approximation method for effectively estimating the posterior predictive distribution for uncertainty quantification in the Bayesian deep learning settings. The main proposal is achieved via a self-supervised learning framework. Multiple evaluations are performed to validate the methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "1. The motivation and background of the work are clearly stated.\n\n2. The theoretical proofs are provided in the Appendices.\n\n3. Multiple case evaluations are performed to validate the method, the experimental configurations are well documented.", "audience_yn": "Yes", "audience_explanation": "1. Laplace approximation in Bayesian deep learning is a highly relevant and active research topic, especially for uncertainty quantification.\n\n2. The idea presented is novel, especially for regression tasks.", "requested_changes": "1. In terms of presentation, I would suggest having the pseudo-code for the algorithms in the main body to improve the clarity of the methods and readability. A high-level conceptual figure of the methods might also be for consideration. Is the regression task the focus of the proposed methods? If yes, it is better to be clear in the main body, including the abstract. Otherwise, a case study on classification may be needed.\n\n2. The method requires the log-likelihood of the predicted instance. How would the method guarantee that the log-likelihood is reliable? E.g., a reliable model (in terms of training metrics) can still make unreliable predictions (for example, in the case of serve data drift), which could result in unreliable log-likelihood?\n\n3. Although multiple experiments are conducted, the tasks are relatively simple and a bit outdated. There are some modern UQ benchmarks worth considering [1]. The comparison baseline methods can also be rich.\n\n[1] Lightning UQ Box: A Comprehensive Framework for Uncertainty Quantification in Deep Learning,\n\n4. It is claimed that *the method is modular with respect to the prior specification*. It will be helpful to include some validations in this aspect, in comparison with some mentioned literature, e.g., Credal Bayesian Deep Learning (Caprioel al. 2024).", "broader_impact_concerns": "N/A", "additional_comments": "N/A", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)