Multivariate Latent Recalibration for Conditional Normalizing Flows
Authors: Victor Dheur
Souhaib Ben Taieb
Venue: NeurIPS 2025 poster ()
Keywords: Uncertainty Quantification
Model Calibration
Multi-response regression
Model Recalibration
Generative Models
Primary area: probabilistic_methods
TL;DR: Latent recalibration learns a radial transform that calibrates normalizing flows, preserving an explicit PDF and improving NLL.

Abstract: A reliable estimate of the full conditional distribution of a multivariate response given a set of covariates is essential in many decision-making applications. However, misspecified or miscalibrated models can lead to poor approximations of the joint distribution, resulting in unreliable predictions and suboptimal decisions. Standard recalibration methods are largely restricted to univariate settings, and while conformal prediction techniques yield multivariate regions with coverage guarantees, they do not provide an explicit form of the underlying probability distribution. We address this gap by first introducing a novel notion of latent calibration, which assesses probabilistic calibration in the latent space of conditional invertible generative models such as normalizing flows and flow matching. Second, we propose latent recalibration (LR), a post-hoc model recalibration method that learns a transformation of the latent space with finite-sample bounds on latent calibration. Unlike existing recalibration methods, LR produces a recalibrated distribution with an explicit multivariate density function while remaining computationally efficient. Extensive experiments on both tabular and image datasets show that LR consistently improves latent calibration error and the negative log-likelihood of the recalibrated models.

Multivariate Latent Recalibration for Conditional Normalizing Flows
Victor Dheur
Souhaíb Ben Taieb*
Department of Computer Science
Department of Statistics and Data Science
University of Mons
Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, United Arab Emirates souhaib. bwwtaieeembaazeeee...e...bebebee ac.
Mons, Belgium
victor. www.duuurouuusu.uuoeuuuuuuuu ac.
Abstract
A reliable estimate of the full conditional distribution of multivariate response given a set of covariates is essential in many decision-making applications. How- ever, misspecified miscalibrated models can lead to poor approximations of the joint distribution, resulting in unreliable predictions and suboptimal decisions. Standard recalibration methods largely restricted to univariate settings, and while conformal prediction techniques yield multivariate regions with coverage guarantees, they do not provide an explicit form of the underlying probability distri- bution. We address this gap by first introducing novel notion of latent calibration, which assesses probabilistic calibration in the latent space of conditional invertible generative models such as normalizing flows and flow matching. Second, we propose latent recalibration (LR),, post-hoc model recalibration method that learns transformation of the latent space with finite-sampl bounds on latent calibration. Unlike existing recalibration methods, LR produces recalibrated distribution with explicit multivariate density function while remaining computationally efficient. Extensive experiments on both tabular and image datasets show that LR consistently improves latent calibration enror and the negative log-likelihood of the recalibrated models.
Introduction
Generating reliable uncertainty estimates is essential for trustworthy decision- making across wide range of applications (Gawlikowski et al., 2023). Multi-output regression problems, in particular, arise frequently in domains such as weather forecasting (Setiawan et al., 2024), energy consumption prediction (Makaremi, 2025), and healthcare resource utilization (Cui et al., 2018). While flexible models like neural networks can achieve high predictive accuracy, their uncertainty estimates are often poorly calibrated, meaning predicted probabilities or confidence regions do not align with empirical frequencies (Guo al., 2017] Dheur and Ben Taieb, (023). Furthermore, most recalibration methods are designed for the single-output setting (Gneiting al., 2007 Song et al., 2019, Sahoo et al., 1021; Kuleshov and Deshpande, 2021 Dewolf et al., 2022 Fakoor et al., 2023
Marx et al.,
2023
Chung et al., 2023 Gneiting and Resin, 2023).
Noting the general lack of methods for assessing and recalibrating multi- multi-output models, Chung et al. (2024) leveraged highest-density regions (HDRs) (Hyndman, 1996) to introduce the notion of HDR calibration and propose the sampling-bass based HDR recalibration (HDR-R) method. Recently, multi- output conformal prediction (CP) methods have also been developed to construct joint prediction sets (Wang al., 20233 Feldman et al., 2023; Fang et al., 2025) Dheur et al., 2025). However, both HDR-R and CP approaches fail to provide an explicit form for the underlying recalibrated probability
Also affiliated with the Department Computer Science, University of Mons.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

distribution, with HDR-R further involving computationally intensive sampling and binning at test. time.
To overcome these limitations, introduce the latent recalibration (LR) method, based new notion of latent calibration, which recalibrates invertible generative models (c.g., normalizing flows (NFs) flow matching (FM)) by operating within their latent space. The core idea is to learn transformation of the latent space such that the resulting model achieves latent calibration. Compared to set-based CP methods and sampling -based based recalibration approaches such HDR-R, R, LR (i) yields fully recalibrated generative model with an explicit multivariate probability density function (PDF), (ii) provides finite-sample latent calibration guarantees, and (iii) enables efficient density evaluation and sampling, which are essential for many applications and support improved decision-making (Klein, 2024).
Our main contributions are:
We introduce latent calibration, a new notion of calibration evaluated within the latent space of an invertible generative model, based on the distribution of latent norms.
We propose LR, recalibration method that yields multivariate predictive distribution with an explicit PDF, offers finite-sample latent calibration guarantees, and remains computation- ally efficient.
We empirically demonstrate, across 29 multi-output tabular datasets and one high- dimensional image dataset, that LR consistently improves latent calibration and reduces negative log-likelihood (NLL). A public codebase is provided to ensure reproducibility!
2
Background
We consider a multi-output distributional regression setting, where the goal is to predict the distri- bution of d-dimensional response variable Y E y C Rd from p-dimensional input vector X E X C IP. We assume access to a dataset D = {(x()) Yo where the samples (XG), yū)
drawn i.i.d. from Pxy, the joint distribution over X V. The dataset is partitioned into three disjoint subsets: training set Dirain. a test set Drest and a calibration set Deal = {(X(), } The true conditional distribution of Y given X = x is denoted by Prix Similarly the true cumulative distribution function (CDF) is denoted Fy max and the cor- responding PDF, assumed to exist for all x E X, is denoted Jyjx=z. More generally, we denote the distribution, CDF and PDF of any random variable A by PA, FA, and fA, respectively. Additionally,
estimates are denoted PA F A. and fA.
2.1 Normalizing Flows for predictive density estimation
Using Diainn our goal is estimate the conditional PDF fy for inputs & E X. NFs offer flexible framework for modeling complex distributions over continuous random variables. Specifically, an y between a latent space Z C IRd and the output space y S Rd. For any y E y and E X, the transformation satisfies T'T'- (y; (1;, x) = y. Given an input X, the NF maps latent random variable 2 E (typically drawn from known base distribution, such as (o, Ia)) to new random variable T(: The resulting conditional PDF computed using the change-of -arvibbell formula:
NF defines learnable bijective (conditional) transformation T Z
fylx= (v) fz f det W
(1)
where fz denotes the density of the latent variable Z. NF's are typically trained by minimizing the NLL over the training dataset using mini-batch stochastic gradient descent. For details on NF architectures, we refer the reader to Papamakarios et al., 202T
For brevity, the main text focuses on normalizing flows, but our method is also compatible with flow matching, as shown in Section L

---

2.2 Statistical calibration
While training with strictly proper scoring rules such as the NLL encourages accurate predictions, does not guarantee that the resulting predictions are reliable or calibrated. mcaning they are statistically aligned with the true distribution of the observations (Gneiting et al., 2007). This issue is particularly relevant under limited data or model misspecification, and has gained renewed attention with the observation that modern neural network classifiers are often miscalibrated and overconfident (Guo et al., 2017).
Probabilistic calibration. For real-valued outcomes (d 1), probabilistic calibration (Gneiting et al., 2007) builds on the probability integral transform (PIT). Denote F Fyjx predictive CDF for Y whose value depends on the random variable X. Assuming F Fyxx== is continuous for any E X, probabilistic calibration requires that
Fxix ~ 1).
(2)
Multivariate calibration. Compared to the univariate case, calibration for vector- valued outcomes has been relatively underexplored. Moreover, assessing calibration in the multivariate setting (d 2 2) is inherently more challenging, as the PIT is no longer uniformly distributed (Genest and Rivest, 2001). While probabilistic calibration can be assessed separately for each dimension, this approach may miss important dependencies between outputs (Chung et al., 2024).
The primary approach to assessing multivariate calibration involves reducing multivariate predictions and observations to univariate summary statistics, and then evaluating the uniformity of the PITs of these transformed values (Allen et al., 2024). Let (X,Y) ~ Px,y and (X, Y) ~ Px Pr and define transformation function (also known as pre-rank function) g X xXy= + IfG g Y) and G = gXx, Y), then by the probability integral transform, the random variable Ú = (G) is uniformly distributed whenever with respect to the transformation g. Chung et al. (2024) introduced HDR calibration as a special case, where g(x, 3) = fxxx== (1). In that case, the corresponding PIT Ú = F'g(G) = HPD frix (Y) the highest predictive density (HPD; Box and Tiao, T990,1992). This form of calibration ensures that HDRs derived from the predictive distribution achieve correct empirical coverage at all nominal probability levels.
d R G. In this case, we say that Prxx is probabilistically calibrated
2.3 Recalibration methods
Recalibration methods adjust a potentially miscalibrated base predictor (e.g., Fylx) to produce an updated predictor (e.g., F ) that satisfies a desired calibration property.
Quantile recalibration. For the univariate case (d = 1), quantile recalibration (QR) (Kuleshov et al., 2018 is recalibration method that enforces probabilistic calibration. Let U = Fyyx (Y), and define Fe. the CDF of Ü, as the calibration map. The recalibrated CDF is given by f = Fg o Frix- By construction, Fyy x (Y) is uniformly distributed over [0, 1). Specifically, for any (0, 1):
ST (a)) = Fo P(F S = (())
(o)
=a)
=a.
HDR recalibration. For d 2 1, Chung et al. (0244 proposed a sampling based HDR recalibration method (HDR-R) R) that targets HDR calibration. Let U (G(G), and define Fg as the calibration map. Given a new input Frest K candidates samples (Y>()) k are drawn from Py|x= with pre-rank values (()) = - -fyIx= (8))), 1,..., K. Based on binning, HDR-R resamples from the set of candidate samples, producing new set of recalibrated samples (Ýr(k) K with pre-rank values (r(()) = -f ()r(k)), This resampling process induces new random variable ý' and its pre-rank ò' - -f (8'). The number of samples in each bin is determined such that
F (- u(0, 1).
(4)
3

---

Table 1: Comparison of calibration notions, the associated calibration statistic O (uniform under calibration). recalibration methods, and related conformal conformity scores.
Calibration notion
Calibration statistic
Recalibration method
Conformal method
Probabilistic (d 1)
xY) Jvp (X) HPD
Quantile recalibration (QR)
DCP
HDR (d 2 1)
HDR recalibration (HDR-R)
HPD-split
Latent (d 2 1)
(e
Latent recalibration (LR)
CONTRA/L-CP
While effective in calibrating HDRs, HDR-R has several limitations: (i) does not produce an explicit recalibrated PDF f x (ii) it generates duplicate samples; (iii) it is subject to discretization errors when estimating FG and (iv) it is computationally expensive, as it requires generating K initial samples for every recalibrated output.
For completeness, we provide an exact algorithm in Section E.2
Estimating the calibration map. In practice, the ideal calibration map F6 is unknown and is estimated based on the calibration statistics (Ūi)_ with Un ~ Fe- A standard estimator is the empirical CDF (a E>= 1(0, < a) but differentiable estimators can also be employed (Marx et al., 2022, Dheur and Ben Taieb, 2024).
2.4 Conformal prediction
Conformal prediction (CP) constructs distribution-free prediction sets R. (X) with finite-sample for any desired significance level E (0, 1) (Vovk et al., 2005 Angelopoulos and Bates, 2021). A common variant, Split CP (SCP) (Papadopoulos al., 2002). partitions the data into a training set Dusin and calibration set D A base predictor trained on Dimins and then a conformity score s : X' x y' R. where lower values indicate better agreement between the model's predictions and the observations. SCP constructs the prediction set by computing the empirical (1 -uuqunttele of the conformity scores evaluated on the calibration set, i.e., (x(X(1), y()),, "...,,,,,,."."". y()),, too), denoted by f-'(1 - a.. The resulting prediction region is (xx) (y E y s(x,v) v) I F -a)). which satisfies the marginal coverage guarantee.
marginal coverage guarantees, i.e., P(Y (x)) > 1
Notably, specific choices of conformity scores correspond to recalibration statistics: Distributional Conformal Prediction (DCP) (Chernozhukov al., al., 2021) uses spcr(x, y) = Fr)x=> while HPD- split (Izbicki al., 2022) uses SHPD-splia (3r, y) HPD frix=r (y): these match the transformations
used in QR and HDR-R, respectively. This highlights a unified framework connecting conformal prediction and recalibration, summarised in Table
3 A New Latent Recalibration Method for Normalizing Flows
We propose new recalibration method, called latent recalibration (LR), for conditional NFs. LR operates the latent space and is specifically designed to achieve our newly introduced notion of multivariate latent calibration.
3.1 A New Notion of Multivariate Latent Calibration
Recall that, given a latent variable Z E 2 with a known distribution and an input x E X' condi- tional NFs estimate the conditional distribution of Y, Fyjx=z by learning a conditional bijective transformation T
y such that the PDF frix of the transformed variable T(Z:z) approx- imates fy|x=x- However, model misspecification significant estimation errors in the learned transformation f can lead to poor calibration of the induced distribution of T(Z:z))
We propose to leverage the simple structure of the latent space and assess calibration directly in this space, notion we refer to as latent calibration. By definition, if the NF is well-specified d
for Fy|x=z, then the inverse transformation 7-1 satisfies (Y; ~ Z, where & denotes

---

approximate equality in distribution. Building this observation, we define norm pz 2 + R.+ over Z (e.g., Pz(z) = ||=]]). The goal test whether (x; Pz(Z).
Since the distribution of Z is known and standard (e.g., standard Gaussian), the distribution of pz (Z) often known in closed-form. For instance, if Z ~O, Ia) and (2 ||=||, then Pz (Z) follows a Chi distribution with d degrees of freedom (xa), whose PDF, CDF, and quantile function can be computed efficiently. As another example, if Z (() is uniformly distributed over the unit hyperball and p=(=) = I| then Pz(Z) follows a Beta(d, 1) distribution.
Definition 1. Consider NF defined by a latent variable Z and a bijective transformation T. For pair (X, Y), define the latent norm w.r.t. T as
L & (X; = P2 (x;
(5)
The NF is said to be latent calibrated w.r.t. Z and the norm Pz if the PIT of the latent norm follows a standard uniform distribution, i.e.,
= == F22(2) ~ ((,
To assess whether model is latent calibrated, we define the latent expected calibration error (L-ECH) as the L distance between the CDF of the PIT variable Ü and the CDF of the uniform distribution:
L-ECE(T)= - L/ (F)(a - al da,
The L-ECE is minimized 0 when 7' is perfectly latent calibrated, and has maximum value of 0.5.
3.2 Multivariate Latent Recalibration
We propose a multivariate latent recalibration method, called LR, which performs a post-hoc adjust- ment of the latent space of a NF to ensure that the resulting model is latent calibrated. Key advantages of LR are that yields a recalibrated distribution with an explicit PDF, remains computationally efficient, and has knite-sample guarantees on latent calibration (see Section 3.3).
Latent space transformation. LR uses the CDF F3 as its calibration map. We define a scalar strictly increasing transformation r RR, - R using the quantile function i which maps the original latent norms 1 E I4, to recalibrated norms as follows:
r(1) = = (Fpz((()))))
(8)
We also define a vector-valued transformation R - 2 based on the scalar transformation r, which maps latent vectors such that pz pz (R(=)) r(p2(=)). When using the Euclidean norm (z(z)= = ||=||, R is a radial transformation:
r(I==|) |||||
R(2)
(with R(0) 0).
The transformation R rescales each vector along its original direction by replacing its norm with r([]=]]). This procedure defines new latent variable Z' = R(Z), and the associated recalibrated NF T.
|||||
Proposition 1. The recalibrated NF f(z':)) X) defined with the new latent variable Z' = R[ZZ) is latent calibrated, i.e. Ú' j=²=hi Fp2(2 ~ 1))
Consider the inverse transformation r-)(1) F2 (F2(1) for latent norm R+. Then, using Pz (x())) r(pz(z)) the following identity holds:
Proof.
(1) = Fr(ez(2)(1) (1) = Fpz(2) (r- = F2()), VEER
(10)
Then, it follows that
P(O' s aa)= = PPF S ()) P())))) a0) =
(11)
5

---

Latent distribution
Predictive distribution
Latent calibration
HDR calibration
1.0
Uncalibrated
0.4
0.4
0.2
0.2
NLL 1/47, ES: 0.90
L-BCB: 0.28
HDR-ECE 0.07
10
1.0
Calibrated
0.8
0.8
1-
0.8
NLL. 0.91, ES: 0.86
L-RCB: 0.02
HDR-BOR: 0.02
0.0
0.0
0.00
0.25
0.50
0.70
1.00
0.00
0.27
0.50
0.72
1.09
Figure Illustration LR for bivariate output. first column shows latent distribution, second column displays the predictive PDF, and the third and fourth columns show reliability diagrams for latent and HDR calibration, respectively. The first row corresponds to an uncalibrated NF, and the second is the same model after LR. Calibration points and their projections in the latent space shown in bluc. The PDF for both the latent distribution and the predictive distribution shown in orange. Level sets of the PIT of the latent levels 0.01, 0.1, 0.5, and 0.9 are indicated with black contours in the second column, and their corresponding preimages are shown in the first column. LR improves both latent calibration (third column) and HDR calibration (fourth column). Additional prediction examples on worl-world datasets are presented in Section G
Recalibrated predictive density. A distinctive feature of our LR recalibration procedure is that produces a recalibrated distribution with an explicit multivariate PDF.
Note that the recalibrated NF can be interpreted as composite transformation f = T' R. applied the original latent variable Z with density fz. typically a standard multivariate Gaussian. Given E X' and y E V, assuming the transformation R is differentiable, the recalibrated predictive density frix-= (y) can be computed using the change of variables formula. Let T (y: x) and
-2') Then, we have
(1) (v) fz (2) /det (V, (V.((=))I det (V,T""(rx=))
(12)
Let us consider the case where Pz (2) = ||= The inverse transformation takes the form R -- z' and the Jacobian determinant of R can be computed efficiently as:
(2')
(1)
d-1
Ör(1)
|eet (V.R(=)))
with / ||2|||
(13)
Øl
A detailed proof is provided in Section C. The term as:
ar(I) Øf
in 13 is computed using the chain rule
dr()) al
(1)) OF (z)
where F (1)
(14)
0r'
To compute (Z) (1)/a1, we leverage the fact that pz (Z) ~ Xd. whose PDF is available in closed-form and be evaluated efficiently.
In practice, Fi is estimated by computing latent norms L; (x(( (0) using samples (x()), ()) from the calibration set Deal- Section D details how this can be achieyed using ker- nel density estimation or monotonic splines, resulting
differentiable estimate F' of Fi. All operations are carried out in log-space to ensure numerical stability. Figure |illustrate LR, with the recalibrated predictive density fri shown the second column of the second row.
3.3 Useful Properties of Multivariate Latent Recalibration
We present finite-sample coverage guarantees for LR and highlight its connections conformal prediction methods. We assume that R depends on an estimate Fj of F;, based on latent norms L Li..., L..

---

Finite-sample coverage guarantees for recalibrated latent norms. Let assume that the estimated calibration map Fú i. maps the i-th order statistic L() of Li...., La within margin X/(n + 1) 2 0 of the target quantile i/ (n + 1), that is,
F ( E
i i+i
(15)
Then, letting = n+I. Theorem 1 of Marx et al.. 2022 yields the following finite-sample coverage guarantee for the recalibrated latent norms:
1+A
P (Fez(2) (ex(Y:x)) S <a)= = (F)( E [a e, te
(16)
where we used (TO) for the first equality and the probabilities are taken over X, Y, and the recalibrated latent norms Li....... Ln.
Equivalence with conformal prediction sets. We observe that the prediction sets derived from the recalibrated predictive density LR coincide exactly with those obtained by the multivariate conformal methods CONTRA (Fang al., 2025) and L-CP (Dheur al., 2025). Specifically, this equivalence holds when LR uses the empirical CDF of the calibration scores L [L,,... L,, +00)
its calibration map, i.e., Fi (1) n+1 Σ 144, < ().
CONTRA and L-CP are conformal methods that construct prediction sets using the conformity score SCONTRA (ar, y) = SL-CP(Z, y) (x( Under this choice of calibration map, for any x E X and
E (0, 1), we have
{u e/ F. (e a) Eye Fe((l( a
(17)
{y E E SCONTRA (I, v) S (a))
(18)
where i (a)= [[ denotes the (1 - a) right empirical quantile of the calibration scores L. This shows that the a-sublevel sets of the PIT of the latent norm of LR (77 correspond exactly to the conformal prediction sets produced by CONTRA and L-CP coverage (8). While this equivalence is notable, it is important to point out that the chosen calibration map FL. being non- differentiable, does not yield well-defined recalibrated predictive density function f fix This
equivalence is summarized in the last row of Table 1
Equivalence of LR and QR in the single-output setting. QR is special of LR where d = 2 = 1], (1, 1), 1 Fylx==(v) and pz(z) z. In this case, R = 7 and thus --=) R (=:z) = Fi Fyxx=r = =f F showing that both methods perform exactly the same transformation.
Related work
Our work builds upon and contributes to generative modeling, calibration, conformal prediction, and methods that combine these concepts the context of multi-output regression. An extended description of related works is available in Section B
Various notions of calibration have been studied, including probabilistic (Gneiting et al., 2007). marginal (Gneiting et al., 2007) and HDR (Chung et al., 2024) calibration. Ziegel and Gneiting (2014) and Allen et al. (2024) also proposed multivariate notions of calibration but, knowledge, no calibration methods for these notions have been proposed.
While traditional CP focuses univariate intervals (Romano al., 2019; Sesia and Romano, 2021
recent multivariate CP methods create flexible regions. HPD-split (Izbicki et al., 2022) uses HPD values scores. PCP (Wang et al., 2023) uses balls around samples. ST-DQR (Feldman al., 2023 selects samples based on a region in a latent space and creates balls around these samples. CONTRA (Fang et al., 2025) and L-CP (Dheur et al., 2025) operate in the latent space of NFs.
Some methods explicitly merge CP and recalibration. Vovk al. (2020) and Vovk al. (0019) developed conformal predictive systems for calibrated univariate distributions. MCC (Marx et al., 2022) unified univariate recalibration methods under CP lens. Our work extends this direction to multivariate outputs via transformation in the latent space.
7

---

5 Experiments
We present an extensive experimental study using 29 tabular datasets widely used in prior research (Tsoumakas et al., 201TT; Cevid et al., 2022 Chung et al., 2024 Feldman et al., 2023] Wang et al., 2023] Barrio et al., 2024 Camehl et al., 2024). Furthermore, while recent work on model recalibration (Chung et al.. 2024; Fang al., 2025) has primarily focused on data modalities with relatively low output dimensionality, we also include a high-dimensional output setting with an image dataset with larger output dimension (Choi al., 2020).
5.1 Datasets
Tabular datasets. The tabular datasets range in size from 103 to 50,000 data points, with the number of input features (p) varying from to 368, and the number of output variables (d) ranging from 2 to 16. A detailed summary of these datasets provided in Table in Appendix [A] Following the protocol of Chung et al. (2024), we use 65/20/15 split for training, validation, and testing. All input features and output targets are normalized to have zero mean and unit variance on the training set. Experiments are repeated 10 times with different random splitting. For each run, we compare the same base model with or without recalibration.
Image dataset. We use the AFHQ dataset (Choi et al.. 2020), which consists of high-resolution animal face images. The imput X f0. 1, 2] indicates one of three classes (cat, dog. or wild animal), and the output is a 256 x 256 RGB image E 3 = [-1, 1|3x x56 resulting in an output dimension of 196,608. We follow the standard split with 14,630 training instances and 1, 1,500 test instances. To improve sample quality, Zhai et al. (2024) add Gaussian noise ~ (00, 0.07*) to each image y during training.
5.2 Experimental setup
(Non-recalibrate) base model. For the tabular datasets, we consider convex potential flows (Huang et al., 22020), masked autoregressive flows (MAFs, Papamakarios, Pavlakou, et al., 2017) and flow matching (FM, Lipman et al., 2002). A notable difference from the setup of Chung et al. (2024) is that their predictive distributions are restricted multivariate Gaussians with diagonal covariance, whereas NFs can model dependencies between output dimensions. For the image dataset, we use the TarFlow model (Zhai al., (024), a transformer-basse achieves state-of-the-at likelihood performance. As is standard, all aforementioned NFs use a latent variable Z ~ (O, T). Details on these base models are provided in Section [ with hyperparameter tuning details for convex potential flows in Section E Results for MAFs and FM are deferred to Sections and In the following, we denote the recalibabddd base model as BASE
conditional NF pre-trained on AFHQ. which
Compared methods. For our latent recalibration method, LR, we use the Euclidean norm Pz (z) || and estimate Fz using kernel density estimation with a Gamma kernel: details are provided in Section D For both tabular and image datasets, we compare LR with the base model BASE, Additionally, we include HDR-R for tabular datasets only, as it becomes computationally prohibitive for TarFlow. For tabular datasets, following Chung et al. (2024). the recalibration map is learned on the validation set. This avoids using additional data for calibration and ensures a fair comparison with BASE, but sacrifices finite-sampn sample guarantees. For the image dataset, since no separate calibration set available, calibration is performed on the training data. This also sacrifices fnite-sample guarantees, but we will show below that it still leads to substantial improvements in calibration.
Error metrics. We consider several error metries compare the different methods. For the tabular datasets, we evaluate model calibration using the latent expected calibration error (L-ECE) and the HDR expected calibration error (HDR-ECE). Both metrics range from 0 (best) 0.5 (worst). Predictive accuracy is assessed using two strictly proper scoring rules: negative ligelikeldoood (NLL) and the energy score (ES). Notably, LR yields a recalibrated density with a closed-form PDF enabling direct computation of the NLL, which is not possible with HDR-R. Since the scales of NLL and ES vary across datasets, we report relative values, defined as the difference with the score achieved by BASE, All metrics are negatively oriented. Exact definitions are provided in Section E. 1 For the image dataset, we report L-FCE and the bits per dimension (BPD), following Zhai et al. 2024). BPD corresponds to a rescaled version of the NLL (details in Section E.1
8

---

5.3 Results
Tabular datasets. Figure presents the normalized difference relative to BASE for NLL and ES. We observe that LR reduces the NLL on the ma- jority of datasets. Since the NLL is a strictly proper scoring rule, this in- dicates that the recalibrated density frix produced by LR generally pro- vides better fit to the true data distri- bution than the original model fy xx In contrast to the NLL, the ES of LR and HDR-R largely unchanged. In Section [.3] we attribute this phe- nomenon to the metric's 's weaker dis- criminative ability relative to misspecifications in variance, correlation, and overall dependency structure. notable exception is the increased ES on the blog_data data (BI.)) dataset, where one output is discrete and single value is repeated across 64% of the instances. Based on this observation, we suggest not using LR on datasets with discrete outputs.
Relativv ES Relative NLL
LR HDR-R
BASE
10
10
-10
10
-10"
SLU
EDM
AT2
SF1
OE2
ATI
JUR
DEL
END
WQ
SP
SCP
ANS
HO2
SC2
RE1
S01
H12
BU
WAG
ME3
MEI
HOI
BIO
BLO
Figure 2: Relative NLL and
datasets sorted by size, using
convex potential flow model.
Figure also shows the L-ECE (as a measure of latent calibration) and HDR-ECE (as measure of HDR calibration), respectively. We see that BASE exhibits significant latent miscalibration across many datasets, with L-ECE values reaching up to 0.25 out of a maximum of 0.5. In contrast, LR consistently and substantially reduces L-ECE, demonstrating its effectiveness in achieving the desired latent calibration. Moreover, L-ECE tends to decrease as dataset size increases, which aligns with the sample guarantees discussed in Section 333 Reliability diagrams in Section further confirm this improvement. Additional experiments on misspecified model are provided in Section M with significant improvements given by LR across all metrics including ES.
BASE
LR
HDR-R
I-ECE
0.2
0.0
HDR-ECE
0.2
0.0
SLU
FDM
AT2
SF1
OF2
JUR
OEI
ENB
WQ
SF2
SCP
HOH
S02
193
B12
B81
WAG
ME3
ME1
MD2
HO1
BIO
BLO
CAL
TAX
AV
SO
AXY
127
110
Figure 3: L-ECE and HDR-ECE datasets sorted by size, using convex potential flow model.
In Figure 3, while LR does not explicitly target HDR calibration, we observe that significantly improves EDR-ECE compared to BASE most datasets, often performing par with HDR-R. As expected, HDR-R achieves low HDR-ECE values by design. These results suggest that improving latent calibration also enhances the calibration of HDRs.
Access to full, calibrated PDF crucial for any task requiring estimation of the probability mass within arbitrary, non-standard region of the output space, capability that set-based methods (like CP) or pure sampling- based methods (like HDR-R) RR) do not provide. A direct application anomaly detection, where low-density points are classified anomaly (Rozner et al., 2024; Perini et al., 2024).. Other examples include risk assessment in engineering, targeted material design, or optimal control. To make the benefits of full PDF concrete, provide experiment decision- task in Section
Further detailed results with the same convex potential flow base predictor are provided in Section [] Section (.TT shows that the primary NLL. gain obtained by LR is due to finding more "plausible" latent codes for the observed data under the base latent distribution. Section J.2 shows that LR significantly

---

Table 2: Performance of LR compared to BASE the AFHQ dataset with TarFlow (standard across 20
evaluations).
L-ECE
BPD
BASE
LR
BASE
LR
0.4 00.722500025
0.008950.0110
5.477
5.465
improves the time to compute the calibration map compared to HDR-R. Section J.3 hypothesizes through theoretical and empirical considerations that the reason the ES remains largely unchanged after LR is due to its relative insensitivity to misspecifications in variance and correlation.
Image dataset. The goal of our image data experiment is to understand the behaviour of latent calibration and recalibration with high-dimensional outputs. Thble@]shows that BASE suffers from severe latent miscalibration, with L-ECE values approaching the maximum of 0.5. LR dramatically improves latent calibration. reducing L-ECE to below 0.01. We also report the bits bits-per-dee per-dimension (BPD), a scaled version of the NLL. Notably, LR does not degrade the original NLL; in fact, slightly reduces it. LR preserves the visual quality of the samples from the base model, with no perceptually visible changes, which aligns with the very small change we observed NI.L.
Conclusion and limitations
We introduced latent recalibration (LR), novel post-hoc method for calibrating conditional normaliz- ing flows in multi-output regression. By transforming the latent space based on calibration scores derived from latent distances, LR achieves latent calibration, ensuring that prediction regions defined in the latent space have correct coverage. Unlike many conformal prediction methods that only output sets, and unlike sampling-based based recalibration methods, LR yields fully specified, recalibrated PDF This offers significant advantages in terms of computational efficiency and applicability to tasks requiring density estimates. Our extensive experiments tabular and high-dimensional image data demonstrate that LR consistently improves NLL, latent calibration, and HDR calibration.
We identify the main limitations of LR as follows. Firstly. LR intentionally adjusts only the magnitude of latent vectors, not their direction, and thus cannot fix miscalibration arising from errors in the orientation of the learned latent manifold. While LR can only perform simple adjustments, this allows simplifying the difficult multivariate calibration problem into a tractable univariate one (calibrating norms). This enables connections with conformal prediction and recalibration methods, and has good empirical performance. Secondly, LR requires the norm of the latent distribution to follow a simple distribution. This is usually the case, normalizing flows predominantly latent variable. Thirdly, LR requires an invertible transformation between the response and latent spaces, and latent random variable with known, tractable density, which makes it incompatible with models such as variational auto-encoders (Kingma and Welling, 2014) or denoising diffusion probabilistic models (Ho et al., 2020p. Instead, NF and FM models are natural fits for LR. Despite these considerations, LR provides a practical and effective tool for obtaining reliable, calibrated multivariate predictive distributions from generative models.
a standard Gaussian
References
[1] C. Alexander et al. "Evaluating the discrimination ability of proper multi-variate scoring rules".. Annals of operations research 334 (1 2022), pp. 857-883.
[2] Sam Allen, Johanna Ziegel, and David Ginsbourger. "Assessing the calibration of multivari- ate probabilistic forecasts". Quarterly journal of the Royal Meteorological Society. Royal Meteorological Society (Greal Britain) 150 (760 2024), pp. 1315-1335.
[3] Brandon Amos, Lei Xu, and J Zico Kolter. "Input Convex Neural Networks". Proceedings of the 34th International Conference on Machine Learning. Vol. 70. Proceedings of Machine Learning Research. PMLR, 2017, pp. 146-155.
[4] Anastasios N Angelopoulos and Stephen Bates. "A gentle introduction to conformal prediction and distribution-free uncertainty quantification". arXiv [cs.LG] (2021). arXiv: 2107 .05111
10

---

[5] Eustasio del Barrio, Alberto González Sanz, and Marc Hallin. "Nonparametric multiple-output center-outward quantile regression". "egression".Jouraee Journal of the American Statistical Association (2024),
pp. 1-43.
[6] George P Box and George Tiao. Bayesian inference in statistical analysis: Box/Bayesian.
Wiley Classics Library. Nashville, TN: John Wiley & Sons, 1992. 608 pp.
[7] Annika Camehl, Dennis Fok, and Kathrin Gruber. "On superlevel sets of conditional densities and multivariate quantile regression".. Journal of Econometrics (105807 2024), p. 105807,
[8] Domagoj Cevid et al. "Distributional Random Forests: Heterogeneity Adjustment and Mul- tivariate Distributional Regression". Journal of machine learning research: JMLR 23 (333 2022), pp. 1-79.
[9] T Chen et al. "Neural ordinary differential equations". Neural Information Processing Systems
31 (2018), pp. 6572-6583.
[10] Victor Chernozhukov, Kaspar Wüthrich, and Yinchu Zhu. "Distributional conformal predic- tion". Proceedings of the National Academy of Sciences of the United States of America 118 (48 2021).
[1]] Yunjey Choi et al. "StarGAN v2: Diverse Image Synthesis for Multiple Domains". 2020 IEEE/CVF Conference Computer Vision and Pultern Recognition (CVPR). 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (Seattle, WA, USA). IEEE, 2020, pp. 8188-8197.
[12] Youngseog Chung. Ian Char, and Jeff Schneider. "SSmppling Multi-dimensional Re- calibration". Interntiiona Conference on Machine Learning. International Conference on Machine Learning. PMLR, 2024, pp. 8919-8940.
[13] Youngseog Chung. Aaron Rumack, and Chirag Gupta. "Parity calibration", Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence. Vol. 216. Proceedings of Machine Learning Research. PMLR, 2023, pp. 413-423,
[14] Liwen Cui al. "Prediction the healthcare resource utilization using multi-output regression models". IISE Iransactions healthcure systems engineering 8 (4 2018), pp. 291-302.
[15] Nicolas Dewolf, Bernard De Bacts, and Willem Waegeman. "Valid prediction intervals for regression problems". Artificial Intelligence Review (2022).
[16] Victor Dheur and Souhaib Ben Taieb. "A Large-Scale Study of Probabilistic Calibration in Neural Network Regression". The 40th International Conference on Machine Learning. PMLR, 2023.
[17] Victor Dheur and Souhaib Ben Taieb, "Probabilistic Calibration by Design for Neural Network Regression". Proceedings of The 27th International Conference on Artificial Intelligence and Statistics. Vol. 238. Proceedings of Machine Learning Research. PMLR, 2024, pp. 3133-3141.
[18] Victor Dheur et al. "A unified comparative study with generalized conformity scores for multi-output conformal regression". The 42th International Conference on Machine Learning. PMLR, 2025.
[19] Laurent Dinh, Jascha Sohl-Dickstein. and Samy Bengio. "Density estimation using Real NVP". arXiv [G) (2016). arXiv: 1605. 08803 [cs LG]
[20] Hailiang Du. "Beyond Strictly Proper Scoring Rules: The Importance of Being Local", Weather
and Forecasting 36 (2 2021), pp. 457-468.
[21] Durkan, Bekasov, Murray, al. "Neural spline flows". Advances in neural information processing systems (2019).
[22] Rasool Fakoor et al. "Flexible Model Aggregation for Quantile Regression". Journal of machine leurning research: JMLR 24 (162 2023), pp. 1-45.
[23] Zhenhan Fang, Aixin Tan, and Jian Huang. "CONTRA: Conformal prediction region via normalizing flow transformation". The Thirteenth International Conference Learning Representations. 2025.
[24] Shai Feldman, Stephen Bates, and Yaniv Romano. "Calibrated Multiple-Output Quantile Regression with Representation Learning".. Journal of machine learning research: JMLR 24 (24 2023), pp. 1 1-48.
[25] Jakob Gawlikowski et al. "A survey of uncertainty in deep neural n Artificial intelli- gence review 56 (S1 2023), pp. 1513-1589. arXiv: 2107 03342 [cs.LG]
[26] Christian Genest and Louis-Paul Rivest. "On the multivariate probability integral transforma-
tion". Statistics & probability letters 53 (4 2001), pp. 391-399.

---

[27] Mathieu Germain et al. "MADE: Masked Autoencoder for Distribution Estimation". Pro- ceedings of the 32nd International Conference on Machine Learning. Vol. 37. Proceedings of Machine Learning Research. Lille, France: PMLR, 2015, pp. 881-889.
[28] Tilmann Gneiting, Fadoua Balabdaoui, and Adrian E Raftery. "Probabilistic forecasts, calibra- tion and sharpness". .Journal of the Royal Statistical Society. Series B, Stafistical methodology 69 (2 2007), pp. 243-268.
[29] Tilmann Gneiting and Johannes Resin. "Regression diagnostics meets forecast evaluation: conditional calibration, reliability diagrams, and coefficient of determination". Electronic journal of statistics 17 (2 2023), pp. 3226-3286.
[30] Tilmann Gneiting et al. "Model Diagnostics and Forecast Evaluation for Quantiles". Quantiles".Anmuu .Anuual Re- view of Statistics and Its Application (2023). doi: 10. 144//annrrv-sstatitiie Satiics-03222---20000
[31] Léo Grinsztajn, Edouard Oyallon, and G Varoquaux. "Why do tree-based models still out- perform deep learning on typical tabular data?" Neural Information Processing Systems 35 (2022), pp. 507-520.
[32] Chuan Guo et al. "On Calibration of Modern Neural Networks". Proceedings of the 34th International Conference on Machine Learning. Vol. 70. Proceedings of Machine Learning Research. PMLR, 2017, pp. 1321-1330.
[33] Ho, Jain, and Abbeel. "Denoising diffusion probabilistic models". Advances in neural infor- mation processing systems (2020).
[34] Chin Wei Huang et al. "Convex Potential Flows: Universal Probability Distributions with
Optimal Transport and Convex Optimization" (2020).
[35] Rob J Hyndman. "Computing and Graphing Highest Density Regions". The American statisti- cian 50 (2 1996). pp. 120-126.
[36] Rafael Izbicki, Gilson Shimizu, and Rafael B Stern. "CD-split and HPD-split: Efficient Conformal Regions in High Dimensions", Journal of machine learning research: JMLR 23 (87 2022), pp. 1-32.
Journal
[37] Diederik P Kingma and Jimmy Ba. "Adam: A Method for Stochastic Optimization". arXiv [cs.LG] (2014). arXiv: 1412. 6980 LG]
[38] Diederik P Kingma and Prafulla Dhariwal. "Glow: Generative flow with invertible 1x1 convo-
lutions". " Neural Information Processing Systems abs//1807.035 000.00039 (2018).
[39] Diederik P Kingma and Max Welling. "Auto-Encoding Variational Bayes". International
Conference on Learning Representations. 2014.
[40] Nadja Klein. "Distributional regression for data analysis". Annual review of statisties and its application 11 (1 2024), pp. 321 136..
[41] Volodymyr Kuleshov and Shachi Deshpande. "Calibrated and Sharp Uncertainties in Deep Learning via Density Estimation". Proceedings of the 39th International Conference on Machine Learning. Vol. 162. Proceedings of Machine Learning Research. PMLR, 2022,
pp. 1 11683-11693.
[42] Volodymyr Kuleshov and Shachi Deshpande. "Calibrated and sharp uncertainties in deep learning via density estimation". arXiv [cs (2021). arXiv: 2112.07184 [cs. LG]
[43] Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon. "Accurate Uncertainties for Deep Learning Using Calibrated Regression". Proceedings of the 35th International Conference on Machine Learning. Vol. 80. Proceedings Machine Learning Research. PMLR, 2018, pp. 2796-2804.
[44] Yaron Lipman et al. "Flow Matching for Generative Modeling". The Eleventh International Conference on Learning Representations. 2022.
[45] Saeed Makaremi. "A multi-output deep learning model for energy demand and port availability forecasting in EV charging infrastructure". Energy (Oxford, England) 317 (134582 2025), 134582.
[46] Charles Marx, Sofian Zalouk, and Stefano Ermon. "Calibration by distribution matching: Trainable kernel calibration metrics". Neural Information Processing Systems abs/2310.2021 (2023), pp. 25910-25928.
[47] Charles Marx et al. "Modular Conformal Calibration". Proceedings of the 39th International Conference on Machine Learning. Vol. 162. Proceedings of Machine Learning Research. PMLR, 2022, pp. 15180-15195.
12

---

[48]
Harris Papadopoulos et al. "Inductive Confidence Machines for Regression". Machine Learn- ing: ECMI. 2002. Springer Berlin Heidelberg, 2002, pp- 345-356.
[49] George Papamakarios, Theo Pavlakou, et al. "Masked autoregressive flow for density estima- tion". Advances in neural information processing systems (2017).
[50] George Papamakarios et al. "Normalizing Flows for Probabilistic Modeling and Inference".
Journal of Machine Learning Research 22 (57 2021), pp. 1-64.
[51] Lorenzo Perini al. "Uncertainty-awwr -wware evaluation of auxiliary anomalies with the expected anomaly posterior". Transactions on Machine Learning Research 2025 (2024).
[52]
Pierre Pinson and Julija Tastu. "Discrimination ability of the Energy score" (2013).
[53]
Geoff Pleiss et al. "On Faimesss and Calibration". Neural Information Processing Systems 30
(2017).
[54] Danilo Rezende and Shakir Mohamed. "Variational Inference with Normalizing Flows". Proceedings of the 32nd International Conference on Machine Learning. Vol. 37. Proceedings of Machine Learning Research. Lille, France: PMLR, 2015. pp. 1530-1538.
[55] Yaniv Romano, Evan Patterson, and Emmanuel Candes. "Conformalized quantile regression". Advances in neural information processing systems (2019).
[56] Amit Rozner et al. "Anomaly Detection with Variance Stabilized Density Estimation". The
40th Conference on Uncertainly in Artificial Intelligence. 2024.
[57] Roshni Sahoo et al. "Reliable decisions with threshold calibration". Advances in Neural
Information Processing Systems. 2021.
[58] Matteo Sesia and Yaniv Romano. "Conformal prediction using conditional histograms". Advances in neural information processing systems (2021).
[59] Karli Eka Setiawan, Hafizh Ash Shiddiqi, and Pandu Wicaksono. "Multi-output machine learning regression for climate prediction: a comparative study of precipitation and temperature forecasting in Jakarta and East Kalimantan, Indonesia". Communications in mathematical biology and neuroscience 2024 (0 2024), Article ID 97.
[60] Hao Song et al. "Distribution calibration for regression". Proceedings of the 36th International Conference on Machine Learning. Vol. 97. Proceedings of Machine Learning Research. PMLR, 2019, pp. 5897-5906.
[61] Grigorios Tsoumakas et al. "MULAN: A Java Library for Multi-Label Learning".Journe Journal of machine learning research: JMLR 12 (71 2011), pp. 2411-2414.
[62] Alexander Tsyplakov. Evaluation of Probabilistic Forecusts: Proper Scoring Rules and Mo- ments. Research rep. 45186. 2013.
[63] Saiteja Utpala and Piyush Rai. "Quantile Regularization: Towards Implicit Calibration of Regression Models". arXiv [cs.LG) (2020). arxiv: 2002. 12860 [cs. LG]]
[64] Vladimir Vovk. Alexander Gammerman, and Glenn Shafer. Algorithmic Learning in Random World. Springer International Publishing, 2005. 26 pp.
[65] Vladimir Vovk et al. "Conformal calibrators". Proceedings of the Ninth Symposium on Con- formal and Probabilistic Prediction and Applications. Vol. 128. Proceedings of Machine Learning Research. PMLR, 2020. pp. 84-99.
[66] Vladimir Vovk et al. "Nonparametric predictive distributions based on conformal prediction". Machine learning 108 (3 2019), pp. 445-474.
[67] Zhendong Wang et al. "Probabilistic Conformal Prediction Using Conditional Random Sam- ples". International Conference Artificial Intelligence and Statistics. International Confer- ence on Artificial Intelligence and Statistics. PMLR, 2023, pp. 8814-8836.
[68] Hee Suk Yoon et al. "ESD: Expected Squared Difference as a Tuning- -Fee Trainable Calibra- tion Measure" (2023).
[69] Shuangfei Zhai al. "Normalizing Flows are Capable Generative Models". arXiv [cs.CV] (2024). arvi 2412. 06329 [cs.CV]
[70] Shengjia Zhao, Tengyu Ma, and Stefano Ermon. "Individual Calibration with Randomized Forecasting", Proceedings of the 37th International Conference on Machine Learning. Vol. 119. Proceedings of Machine Learning Research. PMLR, 2020, pp. 11387-11397.
[71] Johanna Ziegel and Tilmann Gneiting. "Copula calibration". Electronic journal of statistics
8 (2 2014), pp. 2619-2638.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Claims are backed by precise methodology and extensive experiments. Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Limitations are explicitly mentioned in the conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
Proofs provided in the main text and Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Details of the experimental setup are provided in the Appendix. The code is also provided ensure reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification: The code is provided in the Supplementary Material, including data instruc- tions to download the data.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.co cc/
public/guides/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to reproduce the results. See the NeurlPS code and data submission guidelines (https caips.cc/pulle wttpcc//ppbu. htppppulbuuddeeeeeeeeppeppppeeee httpl////guudd//cddeubbbiisss// popddeSubmisssooPoo..
//nips.co
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Full details are provided in the main text and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
The standard error is reported in both bar plots and tables.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative enror rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: This is provided in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The method does present ethical concerns.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The method targets improving the reliability of conditional normalizing flows, which does not present direct negative impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
There is no high risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Authors are properly credited for each dataset in the Appendix. Credits are also given in the README of the code base. MEPS datasets require approving a specific license as mentioned in the README.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcon com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
18

---

For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release new assets. Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Bthics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
19

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non non non-standard components.
Please refer to our LLM policy (https: http:s/s//aaarrricoc////ooffreee for what should or should not be described.
20

---

A Datasets
Table shows the datasets considered in our experiments, with associated reference papers. The datasets are characterized by their total number of instances, number of features and number of outcomes d. The preprocessing follows the setup described Grinsztajn et al., 2022
Table Lists evaluated tabular datasets.
Total size
d
Paper
Dataset
Abbreviation
Tsoumakas et al., 2011
slump edm
SLU
103
EDM
154
16
atp7d
AT2
296
355
sfl
SFI
323
31
oes97
OE2
334
263
16
atpld
ATI
337
354
jura
JUR
359
15
oes10
OEI
403
298
16
enb
ENB
768
wq
WQ
1060
16
14
sf2
SF2
1066
31
3
scpf
SCP
1137
Barrio et al., 2024
ansur2
ANS
1986
2
Camehl al., 2024
households
HO2
7207
14
Tsoumakas et al., 201
scm20d rfl
SC2
8966
60
16
RFI
9005
64
semld
SC1
9803
279
16
Cevid et al.. 2022
births2
B12
10000
24
air
AIR
10000
15
6
birthsl
BI1
10000
23
wage
WAG
10000
78
Feldman et al., 2023
meps 21
ME3
15656
138
meps_ 19
MEI
15785
138
meps_20
ME2
17541
138
house
HO1
21613
17
bio
BIO
45730
2
blog data
BLO
50000
269
Barrio et al., 2024 Wang et al., 2023
calcofi
CAL
50000
2
taxi
TAX
50000
B Extended related work
Various NF architectures exist, including RealNVP (Dinh al., 2016). MAF (Papamakarios, Pavlakou, et al., 2017), Glow (Kingma and Dhariwal, 2018), spline flows (Durkan al., 2019), convex potential flows (Huang et al., 2020) and transformer flows (Zhai al., 2024). NFs are well-suited for LR due to their invertible mapping and explicit density. Radial flows (Rezende and Mohamed, 2015) are special case of (9 with r(t) (a + ()/(a + t) where a R+ and B E IR are learned parameters.
Normalizing flows.
Conditional notions of calibration. Multiple extensions of notions of calibration have been pro- posed by imposing different conditions, including group calibration (Pleiss et al., 2017), distribution calibration (or auto-calibration) (Song al., 2019 Tsyplakov, 2013), individual calibration (Zhao et al., 2020), and threshold calibration (Sahoo et al., 2021). Latent calibration could be extended by imposing similar conditions.
Calibration methods. Improving calibration often involves post-hoe recalibration or regularization during training. Recalibration methods adjust pre-trained models. Notable methods include Kuleshov
21

---

al., 2018 Kuleshov and Deshpande, 2022
Chung al., 2024
Regularization methods incorporate
calibration objectives into training (Utpala and Rai, 2020] Marx et al., 2023; Dheur and Ben Taieb, 2023). However, these methods target univariate settings and can trade off predictive accuracy (NLL,
CRPS) for calibration (Yoon et al., 2023; Dheur and Ben Taieb, 2023). Dheur and Ben Taieb (2024)
integrated QR end-to-end into training, followed by post-hoc recalibration.
C
Proofs
C.1
Jacobian determinant of the radial transform
Recall that the transformation R is defined
R(=) r(1)
(19)
where = ||=]|, for 2 * 0. It maps 2 to a new vector R(2) such that its norm becomes r(I) while its di- rection z/l is preserved (for * 0). We analyze this transformation using hyperspherical coordinates [T,W,..... wd-1). where = ||ж|| is the radial distance and (wi,..., wa-1) are the angular coordi- nates. The transformation R maps these coordinates from (I. wa-1) (r(4), wi,...,,a-1). as only the radial distance is altered.
The Cartesian volume element d is related to the hyperspherical volume element by d dd-i 1 where d is the surface element on the unit (d - 1)-sphere. Under the transfor- mation R, the new radial coordinate is l' = r()), so its differential is di' ar()) dl. The angular part dΩ remains unchanged. The transformed volume element d R(2) is thus given by:
at
R²)()) = (r(1))=1 (2r()) dl dip---
(20)
The Jacobian determinant |det (V=R(=)) is the ratio of the transformed volume element dd R(=) to the original volume element 2d
|dct (V:R(=))] (r(1)) (r(l) 1
di d
(r(1) d-1 Ør(1) 1
(21)
1d d
which corresponds to 13 This holds for I = ||||| > Since r R2 - RR,, r(1) 0. Furthermore, i8 a composition of non-decreasing functions (a CDF and an inverse CDF),
r(),, as defined by 8 making it non-decreasing, ar(() 2 0. Thus, the expression is inherently non-negative.
D Differentiable calibration maps using density estimation
To obtain a differentiable calibration map F L. we estimate the PDF fę of the calibration data using density estimation. We identified two approaches that performed well in experiments.
As an implementation detail for both approaches. density estimation was generally improved by first applying the transformation (t) fl/8 to the calibration scores { After density estimation
in the transformed space, the data is rescaled using the inverse transformation g (f) =3. e
D.1 Kernel density estimation with Gamma kernels
We found that kernel density estimation (KDE) with Gamma kernels is effective because the Gamma distribution has positive support, which is appropriate for the calibration scores L, 0. Let I'(C, A) denote a Gamma distribution with shape Ç >> 0 and rate À > 0. A Gamma distribution I'(pA, X) has mean of μ. We center Gamma kernel at each calibration score L., using the distribution T (LaA, X), which has mean of L.-
The resulting estimated CDF f (4) is given by the average of the individual kemel CDFs:
1 Σ (t)
f F2((t)
(22)
22

---

The parameter À is chosen by minimizing the NI.L of the calibration dataset under the KDE model. This is done using 10-fold cross-validation over the grid 110-5+t selection process is efficient and performed once per run.
99
5+110-
89
This hyperparameter
==
F an example fit on all datasets, illustrating the empirical and estimated smooth CDFs (left y axis) and the estimated log PDF (right y axis).
1.00
1.25
125
1.5
A71
GEL
203
W0
(77 123
15
1.5
522
S0P
A38
H00
502
1.00
0.0
0.00
1.8
20
1/1
35
25
675
118
1.5
1.3
6.5
148
130
150
135
316
64
WWG
3423
XBS
3122
E0L
Log. drwditt
2
810
BLO
CAL
7AX
1.00
0.00
-35
0.00
-25
155
14
1.
3.8
3.5
2.3
Empiziaa CDF
Smooth CDF
Log dexuity
Figure 4: Density estimation using KDE with a Gamma kemel.
D.2 Rational Quadratic Splines
Rational Quadratic Splines (Durkan et al., 2019) provide a flexible framework for defining invertible and differentiable transformations. A base spline & maps [-1, to [-1, 1]. To handle the unbounded domain of the latent norms, we the transformation Vi tanh $ tanh, which maps R. to IR and retains invertibility and differentiabilityy This transformation is used to model the distribution of the latent norms by learning a mapping from standard Gaussian distribution to the data distribution.
For training, the data normalized to have zero mean and unit variance. The parameters of the spline optimized to minimize the NLL of the calibration dataset under the defined model. Optimization is performed using Adam (Kingma and Ba, 2014). To maximize data information, we perform early stopping the training dataset itself and stop if the loss did not improve by le-4 for 50 epochs. Overfitting is prevented by limiting the number of bins and thus the flexibility the spline. Specifically,
4 bins if & 30, bins if 50, 6 bins if < 70, 7 bins if < 80. bins n & 90, and 9 bins n < 100.
Similar to Figure 4 Figure shows example fit for all datasets.
23

---

X711
8F1
0R3
Engaraatee
(710
3UR
10
1.00
1.66
1.55
1.58
1.50
0.5
13
15
20
25
1.00
1.55
1.50
3.00
3.3
0.5
26
25
Law drtuit
-11
-15
2.2
2.00
135
1.9
1.5
RFI
901
ATR
R83
821
C77 L31
Eng Ahoite
0100
1.
2.1
WiG
MRS
MR1
1001
1.00
0.75
200
35
24
25
0.5
1.6
4.5
16
F01
-50
-25
23
Emperioll
Log draeity
Figure 5: Density estimation using rational quadratic spline.
D.3 Challenges with numerical precision
In this section, we address a potential alternative approach and explain why it is impractical due to numerical precision constraints. Theoretically, could attempt to estimate the calibration map Fü using density estimation on the quantity Ú (2) (Y; Since is expected to follow standard uniform distribution under ideal conditions, estimating its density might appear practical.
However, this approach faces significant numerical precision issues, particularly when the latent space dimensionality large. When d large, the CDF (2)(2) for Pz(Z) ~ Xa becomes extremely steep around its mode vd - 1. For example, in our image application with d = 196, 608, proportion 99% of the probability mass is concentrated in the narrow interval [440.7, 446.2). In single-precision doating-poin arithmetic, the CDF saturates quickly: : F xx (t) is numerically 0.0 for t < 433.4 and 1.0 for 447.2.
the latent model is miscalibrated, the values of (Y;
(Y; X) for the calibration data can fall outside
this narrow range where the CDF has fine-grained variation. This results in many Ú values being numerically 0.0 1.0. For smaller dimensions, similar issues occur, although less frequently. For instance, with d = 1, Fx, (t) is numerically 1.0 for > 5.54 in single precision. When a significant portion of the calibration data for Ú consists of values numerically identical to 0.0 or 1.0, accurate density estimation becomes impossible.
For this reason, in Section 3.2 we based our calibration map on density estimation of i. - eV; directly, using F
24

---

E Additional details on experimental setup
Computing the main tabular data results requires approximately 24 hours on an RTX A6000 GPU, and reproducing the image results requires approximately 6 hours RTX 6000 GPU. Experiments can require up to 48 GB of VRAM, primarily due to large batch sizes during sampling for evaluation metrics. Decreasing the batch size reduces VRAM requirements but increases computation time.
For tabular datasets, we tune hyperparameters using grid search, selecting those that yield the lowest NLL the validation set. For the convex potential flow, the number of units in the imput convex neural network is chosen from [10, 20, 40J. the number of layers from [2, 3, 5], and the learning rate from [5 x 10 10 10-3, 33 2 x 10 4). All models trained by minimizing the NLL with the Adam
optimizer (Kingma and Ba, 2014) using a batch size of 1024.
F.1
Evaluation metrics
NLL. We compute the average NLL over the test set as Diess:
NLL
Σ
llg frix=x(Y).
(23)
Drest
(X,Y)EDa)
L-ECE. For each test point (x(1), y()) Diest, we compute the PIT of the latent norm Ú; F2)(2) (4,(0(;;x))) The Latent Expected Calibration Error (L-ECE) is then estimated as the L1 distance between the empirical CDF of (Ü:)/ar |Pael and the uniform CDF:
Deal
L-ECE-
ÜG
(24)
iiess
Dieal
where Unن denotes the j-th order statistic of the computed PIT values.
Energy Score. For each test point (X,Y) E Diesi we generate two independent sets of samples, Sg and Sy from the predictive distribution fr|x==(1 (-). The Energy Score (ES) is estimated as:
RS
Σ
11 YII
11 5''l
(25)
Disal
K
21
(X,Y)
JES.
J
In our experiments, we use = 100.
f (Y((),, as defined Table I The HDR Expected Calibration Error (HDR-ECE) is estimated similarly to L-ECE:
HDR-ECE.
For each test point (x()) y Diest we compute G; HPD
Denl
HDR-ECE =
G
(26)
iDress
D
where Ggj) is the j-th order statistic of the computed HDR pre-ranks. Note that computing the HDR-ECE for HDR-R exactly is not possible as HDR-R does not yield an explicit recalibrated density
x Following Chung et (2024), we use the density fyix the original (non-recalibrated) model for HDR-R when evaluating its HDR-ECE.
BPD. For image datasets, report the Bits Per Dimension (BPD), calculated in Zhai al. 2024). where d is the output dimensionality (c.g., d = 3 x 256 256 for AFHQ). The BPD is then:
BPD = (NLL + log 128) 2o
(27)
Here, the log 128 term accounts for the scaling of pixel values from [0, 255] to [-1, 1]. and division by log 2 converts the NLL from nats to bits.
25

---

Algorithm Pre-rank recalibration.
1: Input: Calibration dataset Deat pre-rank X' x y - R, number of samples K, number of
bins B, base predictor with predictive distribution Py|x, test input Tist.
2: Calibration:
3: for (X )() E Deal
4: for k 1 K
5: (((.)) ~ Py X=X(I)
((A,))) 9 (x(1), 8((k,) ((.0))().))
6:
7: Define Faxx=xo (c) Σx (GUR.)) sc)
8:
U. + f (g (x((), )(1))
9: Define (u) = ioall E.arl Ü su) // Calibration map
10: Prediction:
11: for
K
12:
((k) Pr|x=zue
13:
((0) 9 Frest: ((k)
14: Define a permutation a such that ((+(1)) ... I ((a(K))
S' 0// Initial set of samples
16: for = to B
The + [KF6 (b)
A=11 // Number of resamples
18:
if mg >
Bg { k(b-1)
20:
Sp - (x(x((b)) JEB, // Samples pool
21:
( Ps,, // Resampling with replacement lk=1
22:
S' S' S'U (} 1k=1
23:
return recalibrated predictive samples S'
Relative NLL or ES. To better visualize improvements in NLL or ES relative to the baseline model BASE, we report the difference in these scores, normalized by the absolute value of the score of BASE, For example, the relative NLL for LR computed as (ÑLLeR NLL ( value indicates improvement by LR.
negative
E.2 HDR recalibration
For completeness, Algorithm [ provides the exact recalibration procedure of the HDR-R baseline (Chung et al., 2024) introduced in Section 2.3 Instead of the HDR recalibration algorithm in Chung al., 2024, we present direct generalization to any pre-rank g x y - R. which we call pre-rank recalibration. HDR-R R a special case when g(x, y) -f))
Decision-making experiment
To make the benefits of full PDF concrete, we have conducted an experiment on feeiisionmmakin task.
Experiment setup. We use the SLUMP dataset, where inputs are ingredients for producing concrete and outputs Y = (S, F. C) E R are three concrete properties. A manufacturer must decide among 3 actions = {A, B, D} whether given batch ingredients is suitable for one of two projects (A or B), each with specífic requirement regions, or if it should be discarded (D). The decision has different financial utilities and risks:
Requirements for Project A: 7 < 20, 55 F 65, C 40.
Requirements for Project B: 20
S < 29, 70 < F < 100, 15 C 30.
26

---

Table Comparison of estimation strategies and methods. The best performing combination highlighted in
bold.
Method
Estimation Strategy
Average Utility
BASE
Sampling
62.53 + 11.33
HDR-R
Sampling
32.38 + 10.81
BASE
PDF (Numerical Integration)
76.23 + 11.99
LR
PDF (Numerical Integration)
113.31 + 12.91
The expected utility for an agent with policy a X - A is given by
2000,
if A and y E Region A
-30,
if = A and y & Region if B and E Region B
E Blu((,a(X))) with u(y,a)=
1500,
(28)
- 13,
if a B and y & Region
10, if C.
The optimal action is chosen by maximizing the estimated expected utility. This requires estimating the probabilities P(Y E A X) and P(Y E Region ø X)), which are computed using two approaches: (1) Monte Carlo estimation with 125 samples, or (2) numerical integration of the PDF over 5x5x5 grid via the trapezoidal rule.
The agent acts according the policy a" a*(X)= arg a (A, Un (X) with
(X) = 2000P(Y E Region, X) 30P(Y & Region, X)
un "xXX) 1500p(Y Region,
X) 15 Region X)
(X))) = -10.
Results.
Table Shoo
show two key observations:
1. Using the PDF via numerical integration leads to better decisions (higher utility) than relying finite number of samples.
2. The improved calibration from LR provides a more accurate PDF, leading to a significant further increase in utility. HDR-R, which relies on resampling from the original uncalibrated density, actually harmed decision quality in this task.
This demonstrates concrete scenario where explicit, calibrated PDF is not just theoretical advantage but a practical necessity for optimal decision-making.
G Examples of predictive distributions on real-world tabular datasets
Figures 6 and 7 display examples of predictive PDFs on real-world tabular datasets with dimensional outputs (d = 2). Each row corresponds to a different dataset. For each dataset, two random test instances, (1,(1), ())) and (xx (())) from Diest are shown.
Columns and 3 show the predictive densities from the uncalibrated base predictor BASE (i.e., fy(x= (.) and frix=>re: (-)). Columns 2 and 4 show the corresponding predictive densities from the LR-recalibrated model (i.e., Py / (-) and f (-)). All densities are visualized in orange.
The true target observations (1) and (2) are marked with a blue dot. The negative log-likelln ligelikll of the true target under the respective predictive density is provided in the bottom right corner of each plot. Black contour lines indicate level sets of the PIT of the latent norm (Foz (2') (L
( LR model, and F (2)( x)) for the BASE model) at probability levels 0.01, 0. 0.5, and 0.9.
(Cg (1;,)) for the
In many cases, when the BASE model is already reasonably well-calibrated, LR applies subtle adjustment that difficult to perceive visually. other instances, the recalibration effect more pronounced, visibly altering the shape and spread of the predictive distribution to better align with
27

---

latent calibration. Note that two-dimensional datasets often benefit from smaller NLL improvements according to Table TO suggesting that stronger adjustments should be perceived in higher dimensions.
EDM
-5-
-30
kg Pla -.18
keg Piyie
llne (ltto 5.13
1.00
1.10
ENB
0.90
0.50
0.25
0100
>ayTolk -202
0.35
0.55
0.5
0.0
0.5
1.0
ANS
2-
/ide):
lokk
-2
-2
-2
2
11
B11
Joy Figler
log Pia -1.5
fin 41755
log Plyte' 4::31
Xi
Figure 6: Examples of 2D predictive densities real-worke datascts for random test points (zc, y) Drest.
28

---

P
10-
100
1.0
WAG
60-
00-
-0.5
16:2
-10
-10
-10-
kg Flsi
-1.5
2
2
2-
ME3
0.5
0.5
0-
bie 1.15
Log./ "Wle)
Folak
lox Plales
0.58
0.57
0.88
0.57
1871
BIO
lce flole
2
0.5
6.2
0.1
00
0.2
0.1
00.0
11.5
11.5
BLO
0.2
012
0.0
0100
-1.2
-1.2
Piales -322
-0.4
-0.
2
2
CAL
0.
-9-
Ion /tsta:: .004
by /anl
los Piskii
TAX
0-
Tog Figtr):
Plokk
1
Figure Examples predictive densities on real-world datasets for random test points (zc, v) E Dies-
H Reliability diagrams
Figure 8 shows reliability diagrams for latent calibration. These diagrams plot the nominal probability
levels [0, against the empirical probabilities fa(a), where ò = Fpz((Z) (e:(Y:)) are the PIT values computed on the test set. We also report 90% consistency bands, represented by the shaded area around the diagonal, as described by Gneiting et al. (2023). The BASE model often exhibits miscalibration (deviations from the diagonal), whereas LR consistently aligns closely with the diagonal, demonstrating significantly improved latent calibration.
29

---

SLU (17)
EDM (24)
AT2 (45)
SF1 (50)
OE2 (51)
1.0
(51)
(55)
(62)
ENB (116)
WQQ (159)
0.2
000
SF2 (161)
SCP (171)
ANS (299)
HO2 (1082)
SC2 (1346)
1.0
0.5
0.6
0.4
RF1 (1351)
SC1 (1472)
AIR (1500)
D12 (1500)
B11 (1500)
WAG (1500)
ME3 (2349)
ME1 (2368)
ME2 (2632)
HO1 (3243)
0.6
0.4
0.2
0.0
BIO (6860)
BLO (7501)
ČAL (7501)
TAX (7501)
1.0
0.8
1.00.00
0.30
1.00.00
1.00.00
0.50
1.00
4
BASE
LR
Figure 8: Latent calibration diagrams
Figure shows reliability diagrams for HDR calibration. These diagrams plot nominal probability levels a against empirical probabilities Fo (a), where Ù = HPD frix (Y) are the HDR pre-rank values from the test set. Again, the BASE model frequently shows miscalibration. Both LR and HDR-R improve HDR calibration, though for LR this improvement beneficial side effect rather than direct optimization target, unlike its consistent improvement of latent calibration shown in Figure 8
30

---

SLU (17)
EDM (24)
AT2 (45)
SF1 (50)
OE2 (51)
1.0
(51)
(55)
(62)
END (116)
WQ (159)
0.2
009
SF2 (161)
SCP (171)
ANS (299)
HO2 (1082)
SC2 (1346)
1.0
0.5
0.6
0.4
RF1 (1351)
SC1 (1472)
AIR (1500)
D12 (1500)
B11 (1500)
WAG (1500)
ME3 (2349)
ME1 (2368)
ME2 (2632)
HO1 (3243)
0.6
0.4
0.2
0.0
BIO (6860)
BLO (7501)
ČAL (7501)
TAX (7501)
1.0
0.8
1.00.00
0.30
1.00.00
1.0.000
0.50
1.00
BASE
HDR-R
LR
Figure 9: HIDR-calibration diagrams
I
Base predictors
This section provides details on the base predictors considered in this paper.
I.1 Convex potential flows
Convex potential flows (Huang et al., 2020) parameterize the bijective transformation 7 via strongly convex potential whose gradient yields the inverse map. Given æ e X' let the model define scalar
31

---

potential V
y X - IR where ((3x)) is strongly convex for every fixed The associated transformation is the gradient
= (yr ER.
To ensure convexity in y. ((;x) is parameterized by an imput input-convex convex neural network (ICNN, Amos et al., 2017). A small quadratic term ||lvl enforces strong convexity of (=;z).
Since (:;+)) is strongly convex, (+; x) is bijection Y + 2 with inverse ((:.r). Generating
new conditional samples requires inverting T (1:z). Given 2, recovers ((2; x) as the unique minimizer the convex objective
T(2;)) E arg min yEY
(Vy; - -2 vTy
(29)
Indeed, since (+;)) is differentiable and strongly convex, the minimum is attained when
v, (v(y)x)- - = + (y) = + T (v; - = (30)
In practice 29 can be solved efficiently with gradient- based convex optimization (e.g., L-BFGS), and strong convexity ensures convergence unique solution.
The Hessian (v::)) of is positive definite, with
det V, V,,i-(1:: x det (V:V
One approach to compute this determinant is explicitly forming the Hessian H V2V (y; Concretely, (y; x)) = v. Vy; x) is first evaluated with single forward pass and backpropagation, and then the Hessian is formed as
@f (y:xr
aî-1 (yy
E dddd
Oy1
Oya
requiring additional backpropagations. Finally, the determinant can be computed explicitly in (dd) (e.g., via Cholesky). Hence, the overall computational complexity is O (d Chuckpopp + d3) where Chackpoop denotes the cost of single backpropagation. This brute-force approach practical for small d but becomes prohibitive as d grows; more efficient strategies for large are discussed in Huang et al., 2020
I.2 Masked autoregressive flows (MAFs)
Masked autoregressive flows (Papamakarios, Pavlakou, et al., 2017 implement Í as an autoregres- sive affine transformation. For each coordinate [d]. the model uses two autoregressive NNs (e.g. MADE, (Germain et al., 2015)) to parameterize
A : x +
P R
R.
and (B .)) (p ensures positive outputs. The masking mechanism ensures that (8, 2) and (yei, 2) depend only on the preceding coordinates yes (V1, -9:-1) and the conditioning variable z.
The inverse transformation for each coordinate then
(V(C)))))
i == 1.....,,d
Since this mapping is triangular in y. the Jacobian of f-1 is also triangular, which makes the determinant computation efficient:
d
det V v, (v:x) =
II
32

---

1.3 Flow matching (FM)
Flow matching (Lipman et al., 2022) is a recent generative modeling paradigm which has rapidly been gaining popularity. 'he key motivation behind flow matching to combine the strengths of normalizing flows (NFs) and diffusion models while alleviating their main limitations. NFs enable exact likelihood estimation and efficient sampling but often suffer from limited expressiveness due to architectural constraints. Diffusion models, on the other hand, offer remarkable expressiveness and stability but typically require slow iterative sampling and do not provide tractable likelihoods. Flow matching addresses these issues by framing generative modeling as the learning of a continuous-time flow that transports noise to data, enabling both efficient training and fast sampling while retaining theoretical connections to likelihood- methods.
Given X, model the conditional predictive PDF using transformation defined by the ordinary differential equation (ODE) dy 0 (t. ū, with paramaemeteri vector field
à: [0, 1] x y x X V Training uses the straight-line interpolant between latent 2 ~ N (0, 7) and data f(- x).
7(1) (1 - 1) = + ly:
whose target velocity is constant with respect to L:
dt [t) y
The Conditional Flow Matching objective is min E 0 (t, (1 +V (Y 22)
(31)
where the expectation is over u (0, 1), (X Px,r, and Z N (0. I).
After training,
forward numerical integration generates samples from fyxx== (Chen et al., 2018):
!. à (1) x)) dt,
=
7 (0) z.
(32)
Reverse-time integration encodes y into its latent
7-1 x) ( (0) y J. 0 (1) dl,
i (1)
(33)
For y V/ set f x)) The log-likelihood follows from the instantaneous change-of-
variables formula along the unique ODE path ÿ (1) with ÿ (0) and ÿ (1) = y:
log frix (v) log fz (z) S( Tr TV (t. 0 (t) x)) dt.
(34)
The trace of the Jacobian can be computed using d backpropagations, which can be prohibitive if d large. It can also be efficiently approximated using Hutchinson's 's estimator:
K
Tr T (t) x)) = Ee-N((.n) [eT (Vo0 (t,
(V,0 (t, x)) ek.
K
(35)
with independent eØ. N 10,1), which enables practical likelihood computation.
J Additional results with convex potential flows
J.1 Investigating the NLL performance gain
To investigate the source of the NLL. improvement, we consider the decomposition of the NLLL of the recalibrated model:
log P (2) log Fz (z) log |det (V= det (V,7 (v::))
(36)
with z' = - and (=').. The third term identical for both BASE and LR. All reported terms averaged over the test set and over 10 runs.
33

---

Table 5: Analysis of the NLL of LR compared BASE with the terms described in (36
BASE
BASE
log (2')
log (2)
log det (V.R(=))I
log fr (2)
log fr X=
SLU
5.49
4.35
0.828
2.61
2.29
EDM
4.94
2.81
2.04
-0.0350
-0.123
AT2
11.7
8.65
0.889
4.05
1,86
SFI
11.7
4.33
2.53
4.41
-0.381
OE2
54.6
22.6
0.896
37.6
6.46
ATI
8.47
1.63
0.0783
JUR
5.17
4.24
0.638
2.16
1.87
OEI
98.0
22.6
-1.65
80.0
2.93
ENB
3.08
2.80
0.244
-1.88
-1.12
WQ 74.3
19.9
1.51
60.8
7.95
SF2 10.9
4.29
-1.10
-3.37
-11.1
SCP
32.1
4.19
-0.757
20.1
-8.55
ANS 2.82
2.78
0.0700
1.76
1.79
HO2 5.11
5.67
0.143
2.39
2.38
SC2 25.1
22.6
0.795
0.189
RFI 9,70++02
11.3
-0.0264
9.54c+02
-450
SCI 24.9
22.7
1.39
-1.86
-2.63
AIR 9.15
8.47
0.565
3.03
2.91
BI2 6.99
5.68
-0.208
-11.5
-13.0
3.10
2.82
0.151
-2.77
-2.40
WAG 3.09
2.82
0.184
-3.25
-3.34
ME3
3.07
2,83
0.146
2260
-2.69
MEI
3.21
2.81
0.225
-2.00
-2.17
ME2
3.19
2.82
0.143
-3.00
-3,2
HO1
3.03
2.82
0.144
-0.299
-03664
BIO
3.08
2.82
0.128
-1.12
-1.25
BLO
9.18
2.82
1.14
3.11
-2.11
CAL 2.84
2.82
0.0255
0.575
0.575
TAX
2.85
2.83
0.0208
1.53
1.53
The table reveals clear pattern. The NLL improvement from LR is primarily driven by the first By radially transforming the latent codes 2 to new points that are more consistent with the base density fz, the latent density term log (2) is significantly reduced. The recalibration Jacobian (second term) typically adds small penalty (increases NLL), but this is almost always outweighed by the large gains from the first term. This confirms that LR works by finding more "plausible" latent codes for the observed data under the base latent distribution.
J.2 Computational efficiency
The difference in computation time can be measured in two aspects:
For calibration, the computational complexity of HDR-R O(MFn) and LR is O(Rn) where M = 100 corresponds to the number of samples of HDR-R per instance, F the time for the forward mapping 7 and R the time for the reverse mapping -1
For inference, it is a bit more subtle. Given test insance x, HDR-R R requires to sample at least M times (O(MF) F)) to obtain recalibrated sample, which can be weakness, e.g., if only one conditional sample is needed. LR only incurs low fixed cost C' for evaluating the recalibration map (O(C F)). Thus, the inference time is not directly comparable.
We report the calibration time HDR-R and LR in seconds on the largest datasets using the convex potential flow model and averaged over 10 runs.
34

---

Table 6: Calibration times (part 1)
Method
HO2
SC2
RF1
SCI
AIR
B12
B11
WAG
HDR-R
1.56
2.78
4.90
2.57
4.86
18.80
8.80
15.00
LR
0.232
0.185
0.156
0.187
0.149
0.142
0.133
0.133
Table 7: Calibration times (part 2)
Method
ME3
ME1
ME2
HO1
BIO
BLO
CAL
TAX
HDR-R
9.98
12.70
19.30
7.19
37.70
168.00
8.53
10.50
LR
0.152
0.153
0.163
0.192
0.290
0.328
0.482
0.324
On CIFAR-10 with TarFlow, the time difference is larger and can be prohibitive for HDR-R:
Method CIFAR-10
HDR-R
183182
LR
1259
J.3 Discriminative ability of the energy score and NLL
For comprehensive evaluation of LR, we report the ES in addition the NLL. While LR often leads to improved NLL, the ES remains largely unchanged. We hypothesize that this stems from the score's fundamental limitations in discriminative ability.
Theoretical considerations. As established in Pinson and Tastu, 2013 and corroborated by Alexan- der et al., 2022, the ES is sensitive to shifts in the mean but notoriously insensitive to misspecifications variance, correlation, and overall dependency structure. LR is a post-hoc procedure that primarily corrects the shape and spread of the predictive distribution. Therefore, the ES is fundamentally ill-suited to capture the specific improvements LR provides.
In contrast, the NLL is uniquely suited for this evaluation. As the only local strictly proper scoring rule, its value depends only on the probability density at the precise location of the observed outcome (Du. 2021). This locality makes it highly discerning of the very improvements LR makes to the distributional shape, which is why we observe significant and consistent NLL reductions.
Empirical illustration. To provide clear, empirical illustration, we designed a controlled synthetic experiment based on the dataset in Figure T The goal here isolate this specific property of the scoring rules in setting free from the confounding variables of complex, real-world data.
We an oracle predictor that knows the data-generating distribution from Figure for every- thing except the spread around the arc, which is controlled by a standard deviation parameter a. In Table 8 we then evaluate the predictor's NLL and ES (based 100 samples) as we vary its estimate of a. The metrics are averaged over 10 runs, and the true value is o 0.05.
This experiment clearly illustrates the issue:
The NLL shows sharp, clear minimum the true value of 0.05, correctly identifying the best model.
The ES remains almost completely flat for a wide range of a values (from 0.01 to 0.10). fails to reliably distinguish a model with the correct variance from one that is substantially over- or under-confidentt.
This insensitivity is so profound that detecting a statistically significant signal with the ES requires an impractically large number of samples. The table below shows that only with 5000 runs does the ES minimum align with the true a, and even then the differences are minuscule:
This controlled experiment, therefore, proposes explanation for the insentivity of the ES LR.
35

---

Table 8: Metrics averaged 10 runs, with standard
NLL
ES
0.01
12.01o.195
0.87330.0029s
0.03
1.2740.0222
457249.00267
0.04
0.9232p.012
0) 0.87230.0022
0.05 (True)
0.85570.ck 0.08707 00870
0.87440..0077
0.06
0.8781 0.00629
0.8741o.00
0.07
0.9365g. 0.0483
0.87530.00214
0.10
1.161g.oo
0.0722 0.1612
0.87416.00163
0.20
1.774..000 40.000
0.87820.00176
Table 9: Metrics averaged over 5000 runs, with standard error
0
NLL
ES
0.01
11.910.00773
0.87540.0001399
0.03
1.263 0.0030.000868 00n868
0.875 1.8751o.000139
0.04
0.9177 0.00009
0.8751..000133 0.000139
0.875
0.05 (True)
0.8487 o. 000323
0.8750 000139
0.06
0.8732o 0000231
L.875Lo.000133
0.07
0.9329p 000176
0.875 0.875 10.00133
0.10
1.159p..00100
0.8756g.000r
0.20
1.7746.91e-05
0.8797 0.00033
J.4 Additional tables
For reference, Tables 10 and provide the precise mean values and standard errors for NLL, Energy Score, L-ECE, and HDR-ECE across all tabular datasets when using convex potential flows as the base predictor. For each metric and dataset, all values that are statistically indistinguishable to the best value according to Z-test significance level 0.1 highlighted in bold.
Table 10: Full comparative table, using a convex potential flow model.
NL.
Energy score HDR-R
BASE
BASE
LR
SLU
2.61a19
2.29
0.791
0.795
0.785mmm
EDM
a,0350
-0.123,,
0.647
0.648
0.635
AT2
4.05
1.86,
0.861
0.870u044
0.8624142
SFI
4.41
-0.3813s
0.6733u
0.639aum
0.670uues
OE2
37.611e-11
6,46,,3
1.25
1 1.26
1.26
0.07883.24
0.582uusz
0.587
JUR
2.161124
1.876.15
0.617me34
0.618u033
0.618ku34
OE1
80.0
80.0,
2.93a.7
1.23
1.23,,s
1.17a.15
ENB
-1.08,11
-1.12a.19
0.249
0.250umm
0.249
WQ
60.8
7.95
2.47 2.47
2.44
2.47
-3.37
-11.1aa0
0.587
0.593
0.598uu4
SCP
20.12
-8.55,,49
0.389
0.383
0.382aum
1.76um
1.79
0.529
0.531amor
a.529
HO2
2.39Ix04
2.380,035
0.862a.076
0,866u0075
0,862a0076
SC2
0.795
0.189
1.25
1.28
1.26
RFI
9,54c+022
-.50
0.534
0.528um
0.529
SCI
-1 -1866un0
-2.63um55
0.824m.047
0.833ur045
0.825
3.03
1.17amms6
1.19
1.18
B12
-11.5
-13.0ax
0.833
0.848
0.834mm
.2.27a26
.2.40
0.708
0.71
0.708uus57
WAG
-3,25.31
-3,34
0.802n.4
0.803
0.805
ME3
2..60
22.69112
0.358
0.362
0.360lum0
MEI
-2,00
-2.17.13
0.357
0.370uuoss
0.365uuu7
ME2
-3,00
-3.22,13
0.361
0.362ume
0.362ama
k.29
-0.36-uuaz
0.346
0.351amr
0.340hum
BIO
-1122um7
.1.25gurs
0.207
0.210kmm
0.204
3.1
22.1
0.305
0.365
0.473aue
CAL
0.5756.007
0.575mm008
0.41990004
0.421amss
0.419
TAX
L,3
1,53
0.692
0.696
0.69thum27
36

---

Table 11: Full comparative table, using convex potential flow model.
L-FCE
HDR-FCE
BASE
LR
BASE
HDR-R
LR
SLU
0.146maa26
0.106mms
0.129ume
0.116
0.102aa13
EDM
0.122
0.0905
0.128um4
0.101aa19
0.16%
AT2
0.
0.06374.01
0. 0904440054
0.0817amms
0.0688
SFI
0.279
0.0701
0.32 lanz2
0.
0.0786
OE2
133601
0.0785
0.129s
0.0768umz
0.07666,879
ATI
0. 124400
0.06688..008
0. 25
0.0906u mee
0.06433.11
JUR
0.08830111
9,0520
0.0866m.011
0.0515uma
0.0537a.m44
OE1
0. 1977
0.05894
0.195mm4
0.144aa
0,0636uua77
ENB
0.0272umm3
0.0292, .824
0.0373
0.0347amm9
0.0397
WQ
068 00884
2.0479%.0044
0,0975uu002
0.0443uuse
0.04332.0044
SF2
0.2 0.272au004
0.0434
0,312
1640011
0.0797
SCP
2.050
0.225ums
0.0543ume
0.06144uus
ANS
0,0190uum
0.0254
0.01 10.99
0.0234um226
0.0249%
HO2
0.0162aum24
0.0158
0.01 101790
0.0122ummase
0.0121
SC2
0.101
0.0122a
0.105
0.0223
0.0117a.011
RFI
0.1 22
0.0260 0940
0.1 19
0.0193
0.0288g
SCI
0. 0.0855
0.0106
0,0857
0.0116u0016
AIR
0.0527am0s9
0.0162
0.0449kco::
0.0160hmm
0.0286uas52
0.122ac06
0.0170
0.167ams
0.0219tumzs
0.0731g
B11
0.027%uumt
0.0102
0.11
0.05800u024
0.0923
WAG
0.0478
0.01374.0818
0.267ause
0.1614.71
0.290u.ess
MIE
0.0732pm12
0.00957
0.0520h.cm
0.00899
0.0323po043
MEI
8.0853
0.0103
0.0779a.010
0.00976pa402
0.043260050
ME2
0.0497a009
0.0101
0,0415uc0m
0.0136un22
0.04170.0si
HOI
0.0166umm
0.00950k0003
0.0103
0.00782gaM71
0.0109
0.0178umm9
1.00561..0m6
0.0220h.077
0.00667
0.00727umon
BLO
0.231
0.007355.0000
0.207
0.0485
0.1 1.114
CAL
0.0U749
0.00502a.0044
0.00760mme57
0.00555
0.00543..0044
TAX
0.00949
0.005222.t0033 00005
0. 0.01554400063
0.00684pamsi
0.00561
K
Results with MAFs
For completeness, this section reports results using a MAF (Papamakarios, Pavlakou, et al., 2017) as the base predictor. The architecture consists of stacked flow layers, where each layer's conditioner is a masked autoencoder (Germain et al., 2013) parameterizing rational quadratic spline transformations (Durkan et al., 2019). We tune hyperparameters using grid search. The number of stacked flows is chosen from [3, 5, 8J. the number of hidden units per flow from [32, 64], and the number of hidden layers per flow from [2, 3]. The learning rate is selected from [5 x 10 10-3]. rational quadratic spline transformation.
3J. Each flow learns
The findings, illustrated in Figure I0 and Figure II (and detailed in Tables 12 and 13 consistent
with the main tabular results reported Section S Specifically, LR provides notable improvements in L-ECE, NLL., and HDR-ECE, while achieving energy score comparable to that of the BASE model.
BASE
LR
L-BCE
0.0
Relative NLL
SLU
EDM
AT2
SF1
OE2
ATI
JUR
OBL
ENH-
wQQ
SP2
SCP
ANS
HO2
SC2
RPI
SC1
AIR
BT2
B11
WAG
ME3
MEE
ME2
H01
BLO
CAL
TAX
Figure 10: Latent calibration and NLL on datascts sorted by size, using MAF model.
37

---

LR
HDR,ECE
0.25
0.00
Relative ES
10a
10
-10
-10"
SLU
EDM
AT2
SFI
OE2
ATI
JUR
OE
ENB
WQ
SP2
SCP
ANS
HO2
SC2
RFI
SC1
AIR
BI2
WAG
ME3
MEI
ME2
111
111
BLO
CAL
TAX
Figure Latent calibration NLL on datasets sorted by size, using MAF model.
Table 12: Full comparative table, using MAF model.
NLLI
Energy score HDR-R
BASE
LR
BASE
SLU
4.50p.v
5.08
0.831
0.945mes4
0,858us25
EDM
0.202,.00
0.5900.3
0.577
0.602
0.578ua22
AT2
7. 7.45u
6.22,
0.970
1.1
1.05
SFI
-1.47a.39
-3.424 26
0.698
0.710uss
0.700kess
OE2
22. 42.426
16.8
1.67.1
185
1. 1.79
4.63Ast
3,80
0.698
0.81.5m0s
0.754me29
JUR
3,99u.2
3.65u15
0,6800u87
0.710na31
0.686u822
OE1
16.9,7
9.68
1.31,1
1.51a11
1s8
ENB
11.939
-0.877
0.273n.
0.273
0.275ums
WQ
0. 0.944g9)
-1.23
2.50
26
2.47
SF2
-6.21
-8.7
0.640l.ss1
0.637ase
SCP
-5.1722
-7.58ks1
0.392xam
0.398
0.400
1.89um
1.91mm
0.531
0.53muaa
0.532
HO2
3.06un82
2.87au29
0,881
0,894as00
0,881am00
SC2
1.88
0.9366.11
1.1umm
1.770026
1 1.0%
RFI
.14.3
-15.5,
0.2033.
0.210
0.208u016
SCI
4.4821
.5.00
2.62
2.52
2.65..14
3.74n.
1.2
1.21amm
B12
-11.61777
-12.3
0.831
0.839
0.830
0.6222,11
0.443
0.719
0.722amua
0.718
WAG
22.12
-2.22ame
0.721
0.716m807
0.710u.027
ME3
1195am
-2.37ant
n.401
0.403
0.398
MEI
-1,51
-1,9,
0.531
0.517
0.540uuss
ME2
-1 1194mms
-2.35un45
0.412xm077
0.412am045
0.409
20.1530u1
-0.323nazs
A.327
0.335Mumm
0.330
BIO
1 11.10
-1.42ums
0.203
0.207
0.204
3.337.35
-3.85
0.3724m15
0.372amux
0.362am00
CAL
0.5814363
0.578
0.419
0.421ns0u
0.419
TAX
1.62am85
1.54,m4
0.696
0.698
0.695us015
38

---

Table 13: Full comparative table, using MAF model.
L-FCE
HDR-FCE
BASE
LR
BASE
HDR-R
LR
SLU
0. 1.146
0.101am3
0.328ums
0.110
0.277
EDM
0. 0.128
0.0734
0.153
0.0849hume
0.08899mm
AT2
0. 2626000
0.065
0.376am14
0.118
0.208
SFI
0.1 760uo0
0.0487
0. 0.195
0.1299012
0.0935umm2
OE2
1.300
0.0595
0.418um
0.201aam1
0.220ku1s
ATI
0. 258
0.0566
0.342om11
0.106
0.159
JUR
1850013
0.0544a.0042
0.314uut
0.0717amm4
0.212a.000
OE1
0. 2925s
0.0676xue4
0.39 0.39Ia
0.174aa16
0.153kut1
ENB
0.0277amat
0.0346, 9825
0.0362a.see
0.04281.0s
0.0461amm0
WQ
0.3 pu328s
0,0312 2.0312 24
0.349
0.134
0.188
SF2
0.0310.002
0.197mm1s
0.123
0.07 0.0726um4z
SCP
0.0397
0.0445umss
0.0464aum11
ANS
0 200032
0.0272 J0Z
0,0443ncess
0.0253umm
0.0478o0516
HO2
0.0806
0.0139
0.13
0.0157amm
0,0712g0023
SC2
0. 25050
0.0166,,0155
0,313
0.0672ams1
0.136mcurr
RFI
0.092300046
0.0172 8838
0.0860u.002
0.0139
0.018% m0zs
SCI
0.1 220ss
0.009 0.00913.00011
0,417am3
0.163
0.3944.013
AIR
0.0936
0.0132
0.1440009
0.0193um12
0.0790gu237
0.06077am0
0.0122gu001s
0.0156umm
0.03855.0074
B11
0.0408
0.0105 80081
0.0739p.004
0.01 28
0.04633,0027
WAG
0.113300000
0.0118
0.194
0.01344.0009
0.119hcc015
MIE
0.0744gm055
0.09 0,0919h.000
0.0128umm1
0.0395p.0011
MEI
0.0673u0113
0.0932a.16
0.02744,003
0.0766g.01
ME2
0.0716mu05
0.00501a.oes
0.0848um04
0.0135uu0s7
0.0333,001s
HOI
0.0843um00
0.80693
0.0986
0,0089
0.0292
0.07133mms
0.00615
0.0850hco04
0.00723pamss
0.0230g.006
BLO
0,105501P
0.005655.000s
0. 0.102ani
0.0152
0.028500034
CAL
0.00491es007
0.00543mmees
0.00524a0006
0.00416u.0077
TAX
0.0235uusss
0.005633..000
0,0424g0017
0.00797
0.0235g001
L Results with Flow Matching
While our paper focuses on normalizing flows, LR is fully compatible with flow matching (FM) models (Section[..3). which also learn invertible mappings and assume known latent distribution. For these models, we tune hyperparameters using grid search. The number of hidden units is chosen from [32, 64), the number of hidden layers from [2, 3, 5), and the learning rate from 10-3 1 2 10 10-41
The FM results aligned with the NFs results, with LR standing out particularly the L-ECE and NLL metrics.
BASE
LR
L-BCE
0.2
0.0
Rolative NLL
10"
10
100
EDM
AT2
SFL
OE2
ATI
JUR
OBL
ENH-
wQQ
SP2
SCP
ANS
HO2
SC2
RPI
SC1
AIR
BT2
BU1
WAG
ME3
MEE
ME2
101
BLO
CAL
TAX
Figure 12: Latent calibration and NLL on datascts sorted size, using FM model.
39

---

HDR-ECE
0.2
0.0
Relative ES
10
10
-10
-10"
SLU
EDM
AT2
SF1
OE2
ATI
JUR
OE
ENB
WQ
SP2
SCP
ANS
HO2
SC2
RFI
SC1
AIR
BI2
WAC
ME3
MEI
MD2
1111
111
BLO
CAL
TAX
Figure Latent calibration NLL on datasets sorted by size, using model.
Table 14: Full comparative table, using a FM model.
NL.I
Energy score HDR-R
BASE
SLU
2.16G10
2.0%
0.756uass
0.75%
0.754
EDM
2.20
2.36xx
0.642am16
0.639
0.643aass
AT2
4.76p.2
2.93
0.770
0.770
0.756
2.38
0.90
0.845
0.826
0.841gas4
OE2
9,53
8.94,.
1.20,.06
1.22
1.18.m
1.46,0
1.22
0.57
0.573
0.57
JUR
2.45
2.02a.11
0.613
0.615aanz7
0.613pa26
OE1
3,99,4
2.24uu
0.86laas7
0.866aasx
0.855
ENB
0.9
0.172
0.313
0.316
0.313
WQ
15 0p.7
14.3
2.40,.05
2.44
2.45
2.30
-1.06
0.668
0.648
0.6624a41
SCP
-2.04
33,30
0.39
0.395
0.386
1.78
1.77
0.529
0.533
0.529
HO2 2,53
2.53na2
0.85 28852.nee
0.856
0.852a0nc
SC2
2.20
182,17
1.02
1.02au09
1.02auee
RFI
0.570
.4,53u20
0.367
0.367
0.364
SCI
0.525
0.0719
0.818
0.825
0.819
4.20muen
1.16
1.1umes
1.16,09
B12
-4,20
4,31g.
0.788
6.796aa13
0.792
2.115
2.10n.00
0.7033
0.706
0.7033
WAG
0,346,
0,308
0.699
0.700
0.697em011
ME3
m0.350mm
0.4233ns
0.392
0.3933
0.391
MEI
4. 44.444
0,518
0.384
0.384
0.383
ME2
-0.342uss
-0,422ns5
0.395
0.397
0.396
+0636umms
0.214amm
0.216
BIO
10.56lans
0,5700mm
0.236pmmat
0.236
0.236
1.066
-1.14403s
0.258
0.25%
0.257
CAL
0.645umos
0.643
0.421amm1
0.422
0.421amm1
TAX
1.66,,79
1.66
0.693
0.696
0.693
40

---

Table 15: Full comparative table, using FM model.
L.ECE
HDR-RCE
BASE
LR
BASE
HD8-R
LR
SLU
0.105
0,080%
0.10%
0,0888
0.076laam9
EDM
0.0906ums
0.0776ums7
0.0926a014
0.0820kmm7
4.06544
AT2
1600cso
0.0621eme
0.165uu00
0.07844m1s
0.06015e.e997
SFI
0.1 0.119
0.0593
0.1 22.07
0.06044.000
0.0572x0ers
OE2
199
0.0615umms
0.210
0.0722umms
0.0626
ATI
0959kmm
0.0597amm74
0.083 Jacoso
0.0679kuas
0.0615
JUR
19188m1
0.0596um11
0.0865
0.063Puums
0.0590
OE1
109
0.0647a.m19
0,0863uo07
0.0514
0.0618
ENB
0.0314
0.0344p.004
0.0312am1
0.0318
WQ
0.0349%,004
0.0842
0.0437a007
0.0385
SF2
0.239ac06
0.0339
0.226
0.0397amm51
0.0400
SCP
0.268poon
0.0457aum7
0.253 me
3.048
0.0607
ANS
0.022
0.02369.137
0.023
0.0244
0.0234g.0885
HO2
0.0138pmm
0.01 80000ss
0.0109umm1
0.0116
SC2
0,0590
0.012
0.0332
0.0142umm
0.0127
RFI
0.0843g 0075
0.0273
0.0740g.002
0.0125
3.035300029
SCI
0.057
0.0133,u020
0.0720p0006
0.0156uu021
0.0267000877
AIR
0.0130
0.0151am22
0.0158
0.0132am211
0.0220
0.199s
0.00979
0.0916
0.0288-.oom
BII
0.02430002
0.00924440m3
0.0249p0000
0.00927
0.0101a.0013
WAG
0.0407ax051
0.01444,0013
0.0419b.050
0.0120u0014
0,0307cm031
ME3
0.0412g007
p
0,0330b.006
0.01044m999
0.017600029
MEI
0.0299g Doss
0.00915
0.025
0.00935
0.0143
ME2
0.05370m0e0
0.003
0.054
0.00892,,0044
0.0150u0014
HO1
0.0285
0.00990
0.0239a0043
0.00974m.011
0.0119
1.0253
0.00766
po00ss
2600726,,rww
0.0102-0001
BLOO
0.0346umm
0.00498
0.04022..00m
0.006-48p.000
0.015
CAL
1.00951
0.00395
0.0112,0018
0.00547c.0000
0.00746u.011
TAX
0.01 10000s
0.0122g0011
0.006600.0054
0.013360022
M Results with a misspecified convex potential flow
Tables 16 17 along with Figure 14 and Figure [3 present results for deliberately misspecified convex potential flow. This misspecification was induced by training the base predictor for only two epochs, ensuring it has low predictive accuracy and is likely poorly calibrated.
We observe that, in this additional scenario, LR also leads to improved L-ECE and NLL most datasets, indicating enhanced predictive accuracy compared to the BASE misspecified model. LR achieves similar or improved HDR-ECE and ES compared to HDR-R. R. These results highlight LR's ability to improve misspecified base predictors.
BASE
0.1
L-ECE
0.2
0.0
Relative NLL
10a
10
100
SLU
EDM
AT2
SPI
0122
ATT
JUR
OEL
ENB
WQ
SF2
SCP
ANS
HO2
SC2
RFI
SC1
AIR
R12
RII
WAG
MES
MEL
ME2
HO1
BIO
BLO
CAL
TAX
Dataset
Figure 14: Latent calibration and NLL on datasets sorted by size, using misspecified convex potential flow
model.
41

---

LR
HDR-R
HDDREBC
0.2
Relative euergy score
0.0
10
10
-10
SLU
EDM
AT2
SF
O
A11
JUR
OE1
ENB
WQ
SF2
SCT
HO2
SC2
RF
SCI
AIR
B12
B1
WAG
MB8
MEN
ME2
HO1
D1O
BLO
TAX
Dataset
Figure 15: Latent calibration and NLL on datasets sorted by size, using misspecified convex potential flow model.
Table 16: Full comparative table, using a misspecified convex potential flow model.
NL.
Energy soore HDR-R
BASE
BASE
SLU
3,86,11
3.9311
1.0lsaa
1.03usse
1.0us
EDM
2,84
1.53us3
0.880
0.879
0.871a
AT2
7,99
6.03
1.39
1. 1.36us5
1_34,084
4.05g.3v
0.736
0.857aa75
0.754
0.753
OE2
23. 15.2
13.5um
2.24
2.24
2.07m.16
3.
1.42um
1.40ums
1.39
JUR
4.19
4.09uur
1.080034
1.08003
1.08.04
OE1
23.8
11.1ass
2.14.11
2.12a14
1.91.1
ENB
2,05
2.02am9
0.815
0.8233441
018
WQ
20.0
19.6u15
2.59
2.61
2.59
SF2
le.s
0.2988.15
0.82 looss
0.703
0.688
SCP
5.16r.
1.25
0.738
0.657
0.658
ANS
2.61a
2.6laue
0.817
0.819
0.817
HO2
5,460020
5.18a121
1.230.0033
1.23Gu07
1.22a1007
SC2
22. 20.11
20.8111
2.67nm19
2.69
2.66.m21
RFI
10.8
16.875
1.69
1.70ues
1.69um22
SCI
21 Ag,99
20.0A.10
2.56u823
2.5%
2.54
16.2..0
1.88
1.03
1.0amn
B11
2.660000
2.39
0.779
0.776
0.773
7.58unss
1.50hares
1.50
1.49
WAG
2,78a0009
2.43a.17
0.876u0029
0.875
0.868
ME3
2.3 16/045
1.12amss
0.602s001
0.548
0.549
MEI
2,20g.46
1.10un4s
0.588
0.539
0.540
ME2
2. 156017
1.13urs
0,591amsss
0.545
0.546
2.46ca20
2.34uue
0.743
0.743
0.741a.077
BIO
0.963n
0.302
0.335pmms
0.308
0.31 0.311
CAT
1.376043
1.28uuas
0.472pmu6
0.46%
0.468
BLO
1.21ea33
0.785
0.685p.024
0.69
0.691oo000
0.687
TAX
2.29
2.31m11
0.724
0.725
0.723
42

---

Table 17: Full comparative table, using misspecified convex potential flow model.
L.ECE
HDR-FCE
BASE
LR
BASE
HDR-R
LR
SLU
0.120ku18
0.107sar7
0.127
0.119aa17
0,101nais
EDM
0.135
0.0734umus
0.128
0.0778hum
0.08466mm2
AT2
0.319
0.05930003
0.327
0.0673uuu
0.064240m9
SFI
0.35
0.110
0.354
0.0838u012
0.177ac14
062
0.379
0,0773xuw7
0.379ku12
0.335on15
0.0769
ATI
0.278am
0,0533uss1
D. 0.2977mm
0. 100
0.0565umms
0,0785ougp
0.0494uums
0.0819
0.05 0.0518u022
0.0501amms
OE1
0.410vrr3
0.0733uws7
0.410nor5
0.395om1s
0,07499mm5
ENB
0.0458
0.0326,um2
D.0854g00se
0.0367muus
0.114pc10
0.1 12
0.0405
0.1
0,0467ams
0.039
SF2
0.386
0.0869
0.387
0.150
0.139
0.36laurs
0.0808
0.369pams
0.115ncss
ANS
(042850943
0,0172um1
0.0524pa947
0.0162umm
0,01833m19
HO2
0.1255.00
0.0155
0.130u0005
0.0155uums
0.0156umze
SC2
0.150
0.0123umms
0.153
I.049
0.0124umes
RFI
0.17 0.173u.00
0.0172u002
1816
0.0422ax03
0.0180
SCI
0. 1664
0.00858
1700004)
0,054lacms
0.0103
B12
0,286
0.0133uu0s
p.287
0.0616umm
0.01466m11
0.096laxa
0,00962..014
0.1000
0.0126mrmm
0.0102am00
AIR
0.178
0.0146.u066
0.18
0.0151amm1
0.0140
WAG
0.116
0.0127am021
0.11310011
0,0158
0.0166am11
ME3
0.287
0.0105uumns
0.294
0.0304m
0.0140gmm
MFI
0.287
0.011000011
0.0237
0.0149
ME2
0 0.282am0:
3.00893
2900o000
0.0220uuo
0.0144u001
HO1
0.0801am24
0.0102
0.0880
0.0098%mmec
0.0120am22
0,200h ouse
0.00695..0005
0.213
0.009
16
CAL
0.071 1ooss
0.00549
3.0883pm1
0.0068
I.00681
0.016%
BLO
0,097200899
0.00657e.0en2
0.117a002
0,008433.m122
0.0241o00
TAX
0.044440053
0.0094902,0000
0.05
0. 0,008044,0163
0.009
43


Reviews:
[{"review_id": "HzFhbeTf2M", "reviewer": "Reviewer_4tSF", "summary": "This paper shows a simple way to make multivariate normalizing flows give better uncertainty by fixing their hidden codes. Instead of tweaking outputs directly, the authors look at the length of each hidden code and remap it to match a known length distribution. They call this latent recalibration (LR). After a model is trained, LR rescales the codes using a quantile map so their spread is correct. The new modle still keeps a clear density formula, runs fast, and has math bounds for coverage. Tests on table data sets and one image set show that LR cuts down calibration error and improves the fit of predicitons. The method also links to other conformal tools and matches standard recalibration in the one-dimensional case.", "questions": "Please also see questions in above section.\n\n- for the improved NLL performance, does it mainly come from the base or the jacobian term?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n  - the paper provides definitions (latent calibration, L-ECE) and proves that the recalibrated model has finite-sample coverage guarantees, and unlike sampling-based methods, LR yields a closed-form PDF, so downstream tasks requiring likelihoods can still be done efficiently.\n  \n  - this work also addresses a real gap, that existing recalibration methods either only cover univariate outputs or lack an explicit density. LR fills this need for multivariate flows. Also the method can be applied post-hoc to any normalizing flow model with a simple radial transform, without retraining the base model. Assessing calibration in the latent space via norms is a fresh perspective, mapping latent norms by quantiles seems to be effective, and ties neatly to conformal frameworks.\n\nWeaknesses\n\n  - the proposed LR method seems to be inherently tied to normalizing flows, assuming models with an invertible transformation to a known latent distribution. While NFs are a powerful model class, is it possible to extend these ideas to other popular generative models like VAEs or diffusion models, which are now dominant in many domains. The paper would be stronger if it discussed potential pathways or challenges for extending these ideas to other model architectures.\n\n  - authors states that producing a full PDF is a major advantage over set-based CP methods. However, the argument could be made more specific. The authors mention that this is \"essential for many applications,\" but they do not provide specific examples of downstream tasks where having the explicit density is a critical, enabling feature. Adding such examples and evaluations would further help.\n\n  - LR only adjusts the length of latent vectors, not their directions. is it possible that if a flow’s miscalibration comes from skewed or rotated latent structure , simply remapping norms may fail to correct important dependencies? Also, by focusing on the norm distribution, LR assumes all misshapes live in magnitude, not orientation. In practice, does this always hold?", "was_revised": true, "final_justification": "I've raised my score to a weak accept as most of my concerns are well addressed by the rebuttal.", "rebuttal": "The core requirements for `LR` are **(1) an invertible transformation** between the response and latent spaces, and **(2) a latent random variable with a known, tractable density**. This makes Normalizing Flows (NFs) a natural and ideal fit for our method.\n\nYou are correct that this formulation does not directly apply to models like VAEs or Denoising Diffusion Probabilistic Models (DDPMs), which lack an exact, tractable inverse mapping. We will add a discussion of this limitation to the paper.\n\nWhile it is not the main focus of our paper, `LR` is directly compatible with the more recent **Conditional Flow Matching (CFM)** methods, which also learn invertible maps and have a known latent distribution. We performed additional experiments that we will include in the appendix.\n\n**Details on Conditional Flow Matching:**\nGiven $x \\in \\mathcal{X}$, we model the conditional PDF $\\hat{f}(y|x)$ using a transformation defined by an Ordinary Differential Equation (ODE), $\\frac{d\\tilde{y}}{dt} = \\hat{v}(t, \\tilde{y}, x)$, with a neural network-parameterized vector field $\\hat{v}$. Trained via the Conditional Flow Matching (CFM) objective, the model learns to map samples from a latent variable $Z \\sim \\mathcal{N}(0, I)$ to the target conditional distribution. Forward numerical integration of this dynamic generates samples, while reverse integration performs encoding. To compute the likelihood for $y \\in \\mathcal{Y}$, we first obtain its latent representation $z = \\hat{T}^{-1}(y|x)$. The log-likelihood is then given by the continuous change of variables formula, which integrates along the unique path $\\tilde{y}(t)$ that solves the ODE with boundary conditions $\\tilde{y}(0)=z$ and $\\tilde{y}(1)=y$:\n$$ \\log \\hat{f}(y|x) = \\log f_Z(z) - \\int_0^1 \\text{Tr}\\left(\\nabla_{\\tilde{y}} \\hat{v}(t, \\tilde{y}(t), x)\\right) dt $$\nThe computationally intractable Jacobian trace is efficiently approximated using the unbiased Hutchinson's estimator, enabling practical likelihood computation.\n\n**Results:**\nTo avoid large tables, we compare the methods on the largest datasets. The CFMs results are aligned with the NFs results, with `LR` standing out particularly on the L-ECE and NLL metrics, with a good performance on HDR-ECE and ES.\n\n| Metric | Method |  AIR |     BI2 |     BI1 |      WAG |      ME3 |     ME1 |     ME2 |      HO1 |      BIO |     BLO |    CAL |     TAX |\n|:----|:--|-------:|--------:|--------:|---------:|---------:|--------:|--------:|---------:|---------:|--------:|-------:|--------:|\n| L-ECE | `BASE` | 0.179  |  0.285  | 0.123  |  0.118  |  0.263  |  0.263  |  0.257  |  0.0538  |  0.145   |  0.133   | 0.013   | 0.108   |\n| L-ECE | `LR` | 0.0146 |  0.0136 | 0.0112 |  0.0139 |  0.0145 |  0.0141 |  0.0119 |  0.00944 |  0.00601 |  0.00721 | 0.00487 | 0.0042  |\n| NLL | `BASE` | 4.23   | -1.39   | 1.94   |  0.0692 | -0.304  | -0.438  | -0.364  | -0.228   | -0.487   | -1.75    | 0.575   | 1.49    |\n| NLL | `LR` | 3.43   | -3.45   | 1.47   | -0.222  | -2.04   | -2.11   | -1.98   |  0.216   | -0.957   | -2.5     | 0.599   | 1.2     |\n| HDR-ECE | `BASE` | 0.198  |  0.258  | 0.129  |  0.0763 |  0.246  |  0.241  |  0.225  |  0.155   |  0.14    |  0.197   | 0.0209  | 0.134   |\n| HDR-ECE | `HDR-R` | 0.0201 |  0.0225 | 0.0131 |  0.0125 |  0.0146 |  0.0141 |  0.0112 |  0.00919 |  0.00748 |  0.0115  | 0.00615 | 0.00881 |\n| HDR-ECE | `LR`   | 0.0579 |  0.0214 | 0.0198 |  0.0549 |  0.0587 |  0.0514 |  0.036  |  0.112   |  0.0083  |  0.09    | 0.0204  | 0.0407  |\n| ES | `BASE` | 1.52   |  1.1    | 0.807  |  0.898  |  0.701  |  0.7    |  0.703  |  0.835   |  0.744   |  0.75    | 0.881   | 0.827   |\n| ES | `HDR-R`  | 1.53   |  1.07   | 0.798  |  0.888  |  0.671  |  0.675  |  0.685  |  0.796   |  0.733   |  0.742   | 0.882   | 0.811   |\n| ES | `LR`   | 1.51   |  1.03   | 0.795  |  0.89   |  0.644  |  0.643  |  0.649  |  0.833   |  0.725   |  0.735   | 0.881   | 0.818   |\n\n**Practical advantages of producing an explicit PDF**\n\nWe designed a new experiment to show a practical scenario where having a calibrated, explicit PDF is beneficial. To avoid redundancy, you can find the details of this experiment in the answer to Reviewer qdRU.\n\n**LR only adjusts the length of latent vectors, not their directions**\n\nYou are right, and it is a core design choice of our method.\n\n`LR` intentionally adjusts only the magnitude of latent vectors, not their direction. We acknowledge that this means `LR` cannot fix miscalibration arising from errors in the *orientation* of the learned latent manifold.\n\nHowever, this focused approach is a principled trade-off that yields significant benefits:\n\n- **Tractability and guarantees:** It simplifies the difficult multivariate calibration problem into a tractable univariate one (calibrating norms). This is precisely what enables us to provide finite-sample guarantees and establish a clean connection to conformal prediction and existing Quantile Recalibration.\n- **Empirical performance:** Our experiments demonstrate that this radial-only adjustment is remarkably effective, suggesting that correcting the norm distribution addresses a major source of miscalibration in many conditional normalizing flows.\n\nWe will add a discussion of this deliberate design choice and its implications to the limitations section of the paper.\n\n**Does the NLL performance gain come from the base or the jacobian term?**\n\nTo investigate the source of the NLL improvement, we consider the decomposition of the recalibrated NLL:\n$$-\\log \\hat{f}'(y \\mid x) = -\\log f_{Z}\\left( z \\right) - \\log \\left| \\det\\left( \\nabla_z R(z) \\right) \\right|^{-1} -\\log \\left| \\det\\left( \\nabla_y \\hat{T}^{-1}(y; x) \\right) \\right|$$\n with $z' = \\hat{T}^{-1}(y; x)$ and $z = R^{-1}(z')$. The third term is identical for both `BASE` and `LR`.\nWe analyze the contributions of the first two terms across the largest tabular datasets to avoid the table being too large.\nAll reported terms are averaged over the test set and over 10 runs.\n\n| Model | Expression | SC1 |   AIR |     BI2 |    BI1 |     WAG |    ME3 |    ME1 |    ME2 |    HO1 |     BIO |     BLO |    CAL |    TAX |\n|:-|:--------------------------------------|------:|------:|------:|-------:|-------:|-------:|-------:|-------:|-------:|----------:|-------:|--------:|-------:|\n| `BASE` | $-\\log f_{Z}(z)$ | 25.1  | 9.14  |   7.68 |  3.19  |  3.55  |  3.28  |  3.29  |  3.14  |  3.11     |  2.92  |  3.08   | 2.84   | 2.87   |\n| `LR` | $-\\log f_{Z}(z')$ | 22.7  | 8.47  |   5.68 |  2.82  |  2.8   |  2.82  |  2.81  |  2.81  |  2.82     |  2.82  |  2.83   | 2.82   | 2.83   |\n| `LR` | $-\\log \\left\\| \\det\\left( \\nabla_z R(z) \\right) \\right\\|^{-1}$ |  1.58 | 0.474 |   0.14 |  0.27  |  0.376 |  0.244 |  0.137 |  0.149 |  0.217    |  0.108 |  0.0512 | 0.0247 | 0.0427 |\n| `BASE` | $-\\log \\hat{f}(y \\mid x)$ | -1.91 | 2.85  | -10.2  | -0.879 | -1.25  | -1.57  | -2.01  | -1.88  |  0.000887 | -1.1   | -3.01   | 0.577  | 1.54   |\n| `LR` | $-\\log \\hat{f}'(y \\mid x)$ | -2.73 | 2.65  | -12.1  | -0.98  | -1.62  | -1.79  | -2.35  | -2.05  | -0.0742   | -1.09  | -3.22   | 0.586  | 1.55   |\n\nThe table reveals a clear pattern. The NLL improvement from `LR` is primarily driven by the first term. By radially transforming the latent codes $z'$ to new points $z$ that are more consistent with the base density $f_Z$, the latent density term $-\\log f_Z(z)$ is significantly reduced. The recalibration Jacobian (second term) typically adds a small penalty (increases NLL), but this is almost always outweighed by the large gains from the first term. This confirms that `LR` works by finding more \"plausible\" latent codes for the observed data under the base latent distribution."}, {"review_id": "QQ7brcXtDG", "reviewer": "Reviewer_qdRU", "summary": "This paper propose a new notion \"Latent Calibration\" for multivariate calibration, which aims to overcome the shortcomings of HDR-R, which calibrates the highest-density regtions (HDR). Extensive experiments are conducted to demonstrate the usefulness of the proposed method \"LR\", and it has been demonstrated that 1) the NLL can be improved with LR, 2) LR performs the best when evaluating with the L-ECE.", "questions": "I have the following minor questions/suggestions: \n- In table 4, I find it quite surprising that the energy score for BASE seems to be very close to LR and HDR-R. Do you have any intuitions why this happened?\n\n- You've already included the image dataset in your experiments. Do you observe any \"quality\" difference when you use HDR-R/LR to calibrate NF and then to generate new samples?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength: \n- The paper is in genereal well-written and very easy to follow. \n- The scale of the experiments is quite large. \n- Multivariate calibration is a hard problem in general, and having explicit PDF is a nice property in my opinion. \n\nWeakness: \n- The target of the paper is a bit vague to me. It's not clear if the primary goal is 1) to promote the latent calibration over HDR calibration as a task/research question, 2) to propose an alternative multivariate calibration method that is faster and has explicit PDF. \n\n- The paper does not provide sufficient evidence to either of the two targets above. Specifically: \n\na) For target (1), some theoretic work or empirical experiments might be needed to show the advantage of latent calibration over HDR calibration; e.g., a synthetic case for which HDR calibration fails but latent calibration performs well, or LR improves the quality of generative samples of that of HDR in some cases, or a case study to demonstrate the usefulness of having explicit PDF. \n\nb) For target (2), the motivation of having explicit PDFs is lacking in the introduction; further, the experiment to demonstrate the runtime is missing.", "was_revised": true, "final_justification": "Strength: \n- Having explicit pdf is a hard task, and the proposed method is nice and novel\n- The paper is well-written and very easy to follow. \n- The scale of the experiments is quite large. \n\nWeakness: \n- The experiment results on synthetic datasets are quite strong, yet those on the real-world datasets are less convincing. \n\nI hence think this is a boardline paper (towards acceptance).", "rebuttal": "Thank you for your valuable feedback and recognizing the paper's clarity and the experimental evaluation.\n\n**Clarifying the paper's goal and contributions**\n\nOur main contribution is to introduce latent calibration, a principled framework for multivariate calibration, and simultaneously propose Latent Recalibration (`LR`), a practical and efficient method that realizes this framework. The two are intrinsically linked: the framework provides the theoretical foundation, and the method demonstrates its practical value.\n\nA key advantage of our approach is that `LR` produces a fully-specified, recalibrated PDF. This is a critical feature that distinguishes it from set-based or sampling-based methods and enables superior performance in important downstream tasks, as we demonstrate below.\n\n**1) Performance of LR on a synthetic experiment**\n\nTo provide further evidence of the improved performance of `LR`, we compare the performance of `LR` and `HDR-R` on the synthetic dataset of Figure 1.\nThe `BASE` model is a Masked Autoregressive Flow that has been misspecified by being only trained for 5 epochs.\nThe dataset is divided into 2500 training points, 500 validation/calibration points and 2000 test points.\nThe metrics, averaged over 200 runs, are displayed below with the standard error of the mean.\n\n| NLL `BASE` | NLL `LR` | ES `BASE` | ES `HDR-R` | ES `LR` |\n|:---|:---|:---|:---|:---|\n| 1.83 ± 0.022 | **1.29 ± 0.019** | 0.930 ± 0.0029 | 0.922 ± 0.0027 | **0.881 ± 0.0012** |\n\n`LR` significantly improves both NLL and the Energy Score (ES). `HDR-R`, which is limited to ES evaluation, shows a much smaller improvement. This highlights that `LR` can correct model deficiencies that `HDR-R` does not.\n\n**2) Practical advantages of producing an explicit PDF**\n\nThis is an excellent point. To make the benefits of a full PDF concrete, we have conducted a new experiment on a decision-making task, which we will add to the paper.\n\n**Motivation:** Access to a full, calibrated PDF is crucial for any task requiring estimation of the probability mass within an **arbitrary, non-standard region** of the output space, a capability that set-based methods (like Conformal Prediction) or pure sampling-based methods do not provide. A direct application is anomaly detection, where low-density points are classified as anomaly [1, 2]. Other examples include risk assessment in engineering, targeted material design, or optimal control.\n\n[1] Amit Rozner et al (2024). Anomaly Detection with Variance Stabilized Density Estimation. UAI.\n\n[2] Lorenzo Perini et al (2024). Uncertainty-aware Evaluation of Auxiliary Anomalies with the Expected Anomaly Posterior. TMLR.\n\n**Experimental Setup:** We use the SLUMP dataset (first dataset in Figure 3), where inputs are ingredients for producing concrete and outputs $Y = (S, F, C) \\in \\mathbb{R}^3$ are three concrete properties. A manufacturer must decide whether a given batch of ingredients is suitable for one of two projects (A or B), each with specific, non-negotiable requirement regions, or if it should be discarded. The decision has different financial utilities and risks.\n*   *Project A requirements*: $7 \\leq S \\leq 20, 55 \\leq F \\leq 65, 25 \\leq C \\leq 40$.\n*   *Project B requirements*: $20 \\leq S \\leq 29, 70 \\leq F \\leq 100, 15 \\leq C \\leq 30$.\n*   The optimal action is chosen by maximizing the expected utility, which depends on the estimated probabilities $\\hat{P}(Y \\in \\text{Region}_A | X)$ and $\\hat{P}(Y \\in \\text{Region}_B | X)$. These probabilities are estimated either by (1) sampling or (2) numerical integration of the PDF.\nWe use either 125 samples for sampling or a 5 x 5 x 5 grid for integration using trapezoidal rule.\n\n**Results:**\n| Method | Estimation Strategy | Average Utility |\n| :--- | :--- | :--- |\n| `BASE` | Sampling | 62.53 ± 11.33 |\n| `HDR-R` | Sampling | 32.38 ± 10.81 |\n| `BASE` | PDF (Numerical Integration) | 76.23 ± 11.99 |\n| **LR** | **PDF (Numerical Integration)** | **113.31 ± 12.91** |\n\nThe results show two key observations:\n1.  Using the PDF via numerical integration leads to better decisions (higher utility) than relying on a finite number of samples.\n2.  The improved calibration from `LR` provides a more accurate PDF, leading to a significant further increase in utility. `HDR-R`, which relies on resampling from the original uncalibrated density, actually harmed decision quality in this task.\n\nThis demonstrates a concrete scenario where an explicit, calibrated PDF is practically useful for optimal decision-making.\n\n**3) Computational efficiency**\n\nYou are correct that we should substantiate our claim of computational efficiency with experimental results. As detailed in our response to Reviewer 54XG, we have now benchmarked the calibration time of `LR` against `HDR-R`. The results confirm that `LR` is substantially faster, particularly on larger datasets and computationally intensive models.\n\n**Energy score**\n\nThank you for the insightful question, which touches upon a point we briefly noted in Section 5.3 regarding the Energy Score's (ES) weaker discriminative ability. The observation that the ES remains largely unchanged is an expected outcome that stems from the score's fundamental limitations in discriminative ability. As established in [1] and corroborated by [2], the ES is sensitive to shifts in the mean but notoriously insensitive to misspecifications in variance, correlation, and overall dependency structure. `LR` is a post-hoc procedure that primarily corrects the shape and spread of the predictive distribution. Therefore, the ES is fundamentally ill-suited to capture the specific improvements `LR` provides.\n\nIn contrast, the NLL is uniquely suited for this evaluation. As the only local strictly proper scoring rule, its value depends only on the probability density at the precise location of the observed outcome [3]. This locality makes it highly discerning of the very improvements `LR` makes to the distributional shape, which is why we observe significant and consistent NLL reductions. This result thus highlights an inherent property of the evaluation metric itself, not a shortcoming of our method, and we will clarify this important distinction in the revised manuscript.\n\n[1] Pinson, P., & Tastu, J. (2013). Discrimination ability of the energy score. Technical Report.\n\n[2] Alexander, C., et al (2022). Evaluating the discrimination ability of proper multi-variate scoring rules. Annals of Operations Research.\n\n[3] Du, H. (2021). Beyond Strictly Proper Scoring Rules: The Importance of Being Local. Weather and Forecasting.\n\n**Image generation quality**\n\nThis is a great question, and we appreciate the opportunity to clarify this point. To be perfectly clear, `LR` is not expected to improve the visual quality of generated samples. We will state this explicitly in the paper to avoid any potential confusion. The goal of our image data experiment is to understand the behaviour of latent calibration and recalibration in a very-high dimensional setting.\n\nIn the noisy setting, `LR` preserves the visual quality of the samples from the base model, with no perceptually visible changes, which aligns with the very small change we observed in NLL. In the noiseless setting, where the test data is out-of-distribution relative to the noisy training data, applying `LR` can cause samples to appear less realistic with more uniform colors. This is an expected outcome, as a simple post-hoc recalibration step cannot be expected to generalize to a domain shift and improve generation quality.\n\nSamples from `HDR-R` are only resampled from an initial pool of samples. Thus, differences in visual quality are not expected either."}, {"review_id": "h0ei7hR2X3", "reviewer": "Reviewer_54XG", "summary": "This paper proposes latent calibration and latent recalibration (LR) for calibrating multivariate predictive distributions in conditional normalizing flows. Latent calibration is formalized through the probability integral transform (PIT) of the norm of the latent vectors. LR is a post-hoc procedure, which learns a transformation in the latent space and outputs an explicit probability density. In experiments on tabular and image datasets, the proposed LR method is empirically shown to be effective in increasing predictive accuracy and reducing calibration errors in terms of L-ECE and HDR-ECE.", "questions": "- I personally don't get what the black bars in Figure 2 and the green bars in Figure 3 mean. Perhaps the green bars are HDR-R. Can the authors clarify these two figures more?\n- For the image task, can the authors provide some qualitative inspections and analysis of how LR performs?", "limitations": "Yes.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The paper is well organized.\n- The empirical evaluations are extensive, considering a large set of tabular datasets and a high-dimensional image dataset.\n\n\nWeaknesses:\n- It seems LR relies on invertibility and the latent (norm) distribution needs to be known, which limits the extension in other generative models (e.g. VAEs and diffusion models).\n- The authors claim that LR is computationally efficient, which is not backed up by experiments.\n- It seems there are no comparisons against other recalibration methods in experiments.", "was_revised": true, "final_justification": "The additional clarifications and the new experiments the authors have conducted are helpful. My initial concerns about claims on computational efficiencies have been resolved by new experimental results. I increase my score to 4.", "rebuttal": "Thank you for the valuable feedback and for recognizing the extensiveness of our experiments.\n\n**On the extension to other models**\n\nWe discuss this point in detail in our response to Reviewer 4tSF. While `LR` is not applicable to VAEs and standard diffusion models, it is directly extendable to **Conditional Flow Matching (CFM)**. We showed that `LR` also leads to significantly improved performance with CFM.\n\n**On the comparison to other methods**\n\nWe compared `LR` to the uncalibrated baseline and `HDR-R` because, to our knowledge, these are the only existing methods for post-hoc recalibration of multi-dimensional distributions. The authors of `HDR-R` [1] also noted that \"methods which account for the joint multi-dimensional distribution in assessing calibration and recalibrating the prediction are generally lacking.\" Our work helps fill this important gap.\n\n[1] Chung et al (2024). Sampling-based Multi-dimensional Recalibration. ICML.\n\n**Computational efficiency**\n\nWe agree that reporting the computation time will strengthen the paper.\nThe difference in computation time can be measured in two aspects:\n- For calibration, the computational complexity of `HDR-R` is $O(M F n)$ and `LR` is $O(R n)$ where $M = 100$ corresponds to the number of samples of `HDR-R` per instance, $F$ the time for the forward mapping $\\hat{T}$ and $R$ the time for the reverse mapping $\\hat{T}^{-1}$.\n- For inference, it is a bit more subtle. Given a test insance $x$, `HDR-R` requires to sample at least $M$ times ($O(M F)$) to obtain a recalibrated sample, which can be a weakness, e.g., if only one conditional sample is needed. `LR` only incurs a low fixed cost $C$ for evaluating the recalibration map ($O(C + F)$). Thus, the inference time is not directly comparable.\n\nWe report the calibration time of `HDR-R` and `LR` in seconds, using the convex potential flow model and averaged over 10 runs. To avoid the table being too large, we restrict the table to the largest datasets.\n\n| Method |      HO2 |      SC2 |      RF1 |     SC1 |      AIR |       BI2 |      BI1 |       WAG |      ME3 |       ME1 |       ME2 |      HO1 |       BIO |        BLO |     CAL |       TAX |\n|:-----|---------:|---------:|---------:|--------:|---------:|----------:|---------:|----------:|---------:|----------:|----------:|---------:|----------:|-----------:|--------:|----------:|\n| `HDR-R` | 1.56     | 2.78     | 4.9      | 2.57    | 4.86     | 18.8      | 8.8      | 15        | 9.98     | 12.7      | 19.3      | 7.19     | 37.7      | 168        | 8.53    | 10.5      |\n| `LR`    | 0.232    | 0.185    | 0.156    | 0.187   | 0.149    |  0.142    | 0.133    |  0.133    | 0.152    |  0.153    |  0.163    | 0.192    |  0.29     |   0.328    | 0.482   |  0.324    |\n\nOn CIFAR-10 with TarFlow, the time difference is larger and can be prohibitive for `HDR-R`:\n\n| Method | CIFAR-10 |\n|:---|---:|\n| `HDR-R` | 183182 | \n| `LR`    | 1259 |\n\n**Clarifications for the figures**\n\nThank you for pointing out the ambiguity in our figures. You are correct that the green bars in Figure 3 represent the `HDR-R` method. The bars in Figures 2 and 3 show the mean metric value across 10 independent runs, and the black error bars depict one standard error of the mean. We apologize for the omission and will update the figure captions and legends to make this explicit.\n\n**Image generation quality**\n\nThis is a great question. We clarify this point in detail in our answer to Reviewer qdRU."}]

Author rebuttal:

Meta-review: ### Summary of scientific claims and findings

The paper introduces a framework for multivariate calibration and a post-hoc method for conditional normalizing flows. They rescale latent vector norms to match a known distribution, producing a calibrated model with an explicit PDF. The method is theoretically grounded with finite-sample coverage guarantees and links to conformal prediction. Empirical results show improvements in calibration, likelihood, and efficiency.

### Strengths

* Clear and well-written presentation.
* Conceptually simple and novel approach.
* Theoretical guarantees.
* Extensive empirical evaluation.
* Additional experiments during rebuttal addressed concerns about runtime, decision-making utility, and metric sensitivity.

### Weaknesses and missing elements

* Applicability limited to invertible models.
* Improvements on real-world datasets less pronounced compared to synthetic cases.
* Method adjusts latent magnitudes only, leaving potential miscalibrations in orientations unaddressed.
* Some evaluation metrics show limited sensitivity, raising interpretability challenges.

### Key reasons for acceptance

1. The method is simple, yet principled.
2. Produces explicit PDFs.
3. Empirical evaluation is extensive.
4. Limitations are clearly acknowledged and well-documented.

### Summary of discussion and rebuttal

Before rebuttal:

* `4tSF`: Positive on novelty and framing but cautious about applicability beyond normalizing flows. Initially rated borderline, later upgraded to weak accept.
* `54XG`: Found the method clear and evaluations extensive, but questioned efficiency and comparisons; rated borderline accept.
* `qdRU`: Appreciated novelty and explicit PDFs but remained unconvinced by real-world dataset performance; rated borderline accept.

Rebuttal:

* Authors added scaling results, runtime benchmarks, a decision-making experiment, and controlled experiments clarifying Energy Score behavior.

Post-rebuttal:

* `4tSF`: Upgraded to weak accept, concerns largely resolved.
* `54XG`: Maintained borderline accept, noting computational advantages and extensibility.
* `qdRU`: Maintained borderline accept, still cautious about real-world impact but acknowledged novelty and usefulness.

### Weighing these points

All active reviewers leaned toward acceptance after rebuttal. While two remained at borderline, the novelty, theoretical grounding, and practical utility outweigh the limitations.
Decision: Accept (poster)