Title: Multivariate Latent Recalibration for Conditional Normalizing Flows
Venue: NeurIPS 2025
Accepted: yes
Keywords: Uncertainty Quantification

Model Calibration

Multi-response regression

Model Recalibration

Generative Models
Abstract: A reliable estimate of the full conditional distribution of a multivariate response given a set of covariates is essential in many decision-making applications. However, misspecified or miscalibrated models can lead to poor approximations of the joint distribution, resulting in unreliable predictions and suboptimal decisions. Standard recalibration methods are largely restricted to univariate settings, and while conformal prediction techniques yield multivariate regions with coverage guarantees, they do not provide an explicit form of the underlying probability distribution. We address this gap by first introducing a novel notion of latent calibration, which assesses probabilistic calibration in the latent space of conditional invertible generative models such as normalizing flows and flow matching. Second, we propose latent recalibration (LR), a post-hoc model recalibration method that learns a transformation of the latent space with finite-sample bounds on latent calibration. Unlike existing recalibration methods, LR produces a recalibrated distribution with an explicit multivariate density function while remaining computationally efficient. Extensive experiments on both tabular and image datasets show that LR consistently improves latent calibration error and the negative log-likelihood of the recalibrated models.


Peer reviews:
comments: SUMMARY:
This paper proposes latent calibration and latent recalibration (LR) for calibrating multivariate predictive distributions in conditional normalizing flows. Latent calibration is formalized through the probability integral transform (PIT) of the norm of the latent vectors. LR is a post-hoc procedure, which learns a transformation in the latent space and outputs an explicit probability density. In experiments on tabular and image datasets, the proposed LR method is empirically shown to be effective in increasing predictive accuracy and reducing calibration errors in terms of L-ECE and HDR-ECE.

STRENGTHS AND WEAKNESSES:
Strengths:
- The paper is well organized.
- The empirical evaluations are extensive, considering a large set of tabular datasets and a high-dimensional image dataset.


Weaknesses:
- It seems LR relies on invertibility and the latent (norm) distribution needs to be known, which limits the extension in other generative models (e.g. VAEs and diffusion models).
- The authors claim that LR is computationally efficient, which is not backed up by experiments.
- It seems there are no comparisons against other recalibration methods in experiments.

QUESTIONS:
- I personally don't get what the black bars in Figure 2 and the green bars in Figure 3 mean. Perhaps the green bars are HDR-R. Can the authors clarify these two figures more?
- For the image task, can the authors provide some qualitative inspections and analysis of how LR performs?

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper propose a new notion \"Latent Calibration\" for multivariate calibration, which aims to overcome the shortcomings of HDR-R, which calibrates the highest-density regtions (HDR). Extensive experiments are conducted to demonstrate the usefulness of the proposed method \"LR\", and it has been demonstrated that 1) the NLL can be improved with LR, 2) LR performs the best when evaluating with the L-ECE.\n\nSTRENGTHS AND WEAKNESSES:\nStrength: \n- The paper is in genereal well-written and very easy to follow. \n- The scale of the experiments is quite large. \n- Multivariate calibration is a hard problem in general, and having explicit PDF is a nice property in my opinion. \n\nWeakness: \n- The target of the paper is a bit vague to me. It's not clear if the primary goal is 1) to promote the latent calibration over HDR calibration as a task/research question, 2) to propose an alternative multivariate calibration method that is faster and has explicit PDF. \n\n- The paper does not provide sufficient evidence to either of the two targets above. Specifically: \n\na) For target (1), some theoretic work or empirical experiments might be needed to show the advantage of latent calibration over HDR calibration; e.g., a synthetic case for which HDR calibration fails but latent calibration performs well, or LR improves the quality of generative samples of that of HDR in some cases, or a case study to demonstrate the usefulness of having explicit PDF. \n\nb) For target (2), the motivation of having explicit PDFs is lacking in the introduction; further, the experiment to demonstrate the runtime is missing.\n\nQUESTIONS:\nI have the following minor questions/suggestions: \n- In table 4, I find it quite surprising that the energy score for BASE seems to be very close to LR and HDR-R. Do you have any intuitions why this happened?\n\n- You've already included the image dataset in your experiments. Do you observe any \"quality\" difference when you use HDR-R/LR to calibrate NF and then to generate new samples?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper shows a simple way to make multivariate normalizing flows give better uncertainty by fixing their hidden codes. Instead of tweaking outputs directly, the authors look at the length of each hidden code and remap it to match a known length distribution. They call this latent recalibration (LR). After a model is trained, LR rescales the codes using a quantile map so their spread is correct. The new modle still keeps a clear density formula, runs fast, and has math bounds for coverage. Tests on table data sets and one image set show that LR cuts down calibration error and improves the fit of predicitons. The method also links to other conformal tools and matches standard recalibration in the one-dimensional case.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n  - the paper provides definitions (latent calibration, L-ECE) and proves that the recalibrated model has finite-sample coverage guarantees, and unlike sampling-based methods, LR yields a closed-form PDF, so downstream tasks requiring likelihoods can still be done efficiently.\n  \n  - this work also addresses a real gap, that existing recalibration methods either only cover univariate outputs or lack an explicit density. LR fills this need for multivariate flows. Also the method can be applied post-hoc to any normalizing flow model with a simple radial transform, without retraining the base model. Assessing calibration in the latent space via norms is a fresh perspective, mapping latent norms by quantiles seems to be effective, and ties neatly to conformal frameworks.\n\nWeaknesses\n\n  - the proposed LR method seems to be inherently tied to normalizing flows, assuming models with an invertible transformation to a known latent distribution. While NFs are a powerful model class, is it possible to extend these ideas to other popular generative models like VAEs or diffusion models, which are now dominant in many domains. The paper would be stronger if it discussed potential pathways or challenges for extending these ideas to other model architectures.\n\n  - authors states that producing a full PDF is a major advantage over set-based CP methods. However, the argument could be made more specific. The authors mention that this is \"essential for many applications,\" but they do not provide specific examples of downstream tasks where having the explicit density is a critical, enabling feature. Adding such examples and evaluations would further help.\n\n  - LR only adjusts the length of latent vectors, not their directions. is it possible that if a flow’s miscalibration comes from skewed or rotated latent structure , simply remapping norms may fail to correct important dependencies? Also, by focusing on the norm distribution, LR assumes all misshapes live in magnitude, not orientation. In practice, does this always hold?\n\nQUESTIONS:\nPlease also see questions in above section.\n\n- for the improved NLL performance, does it mainly come from the base or the jacobian term?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}