Title: Asymptotically exact variational flows via involutive MCMC kernels
Venue: NeurIPS 2025
Accepted: yes
Keywords: Bayesian

variational inference

MCMC

convergence
Abstract: Most expressive variational families---such as normalizing flows---lack practical convergence guarantees, 
as their theoretical assurances typically hold only at the intractable global optimum.
In this work, we present a general recipe for constructing tuning-free, asymptotically exact variational ﬂows on *arbitrary* state
spaces from involutive MCMC kernels.
The core methodological component is a novel representation of general involutive MCMC kernels as invertible, measure-
preserving iterated random function systems, which act as the ﬂow maps of our variational ﬂows. This leads to three new variational families with provable total variation convergence. Our framework resolves key practical limitations of existing variational families with similar guarantees (e.g., MixFlows), while requiring substantially weaker theoretical assumptions. Finally, we demonstrate the competitive performance of our ﬂows across tasks including posterior approximation, Monte Carlo estimates, and normalization constant estimation, outperforming or matching No-U-Turn sampler (NUTS) and black-box normalizing ﬂows.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims at addressing convergence guarantees in variational inference (VI) models. \nThe quality of a variational approximation is fundamentally determined by the expressiveness of the variational model. Nonetheless, an expressive model is not necessarily guaranteed to converge due to the typical non-convexity of the optimization problem. \nConversely, MCMC methods are asymptotically exact but computationally intensive and not always differentiable or easily tunable. The paper tries to bridge this gap with this work. The authors introduce involutive MCMC kernels as invertible, measure-preserving iterated random functions (IRFs) acting as the flow maps of the (exact) variational flows. \n\nThe authors introduce some theorems to motivate their theorethical findings and validate their approach with extensive numerical experiments.\n\nSTRENGTHS AND WEAKNESSES:\n# **Strengths**\n- The paper addresses an important and timely topic in the field of variational inference.\n- The experimental evaluation is comprehensive, covering a diverse range of benchmarks, including both synthetic and real-world datasets.\n\n# **Weaknesses**\n- The paper is difficult to follow at times and I believe is requires some efforts in enhancing clarity and accessibility.\n- The notation is occasionally inconsistent, and some notations or acronyms are not properly defined.\n- Certain concepts are assumed as known without sufficient explanation or background, which may hinder accessibility to a broader audience.\n- The writing lacks clarity in presenting the main contribution. In particular, the significance of constructing *exact* variational families via involutive MCMC kernels does not come through clearly in the exposition.\n- The visual presentation of experimental results is suboptimal. Figures are too small, as are the axis labels and legends. In addition, some plots lack sufficient detail in their captions (e.g., see Fig. 4).\n- The conclusions drawn from the experimental results appear broader than the initial scope outlined in the abstract. This mismatch may confuse readers—for example, Fig. 2 is more confusing than illuminating, as it does not clearly support the central claim that the proposed methods outperform standard normalizing flows.\n- Additional concerns are discussed further in the responses to the review questions.\n\nQUESTIONS:\n- line 36: what do authors mean with \"optimal tuning\"?\n- line 52-53: how about stochastic normalizing flows or recent methods combining MCMC sampling with training of variational families?\n- eq(1) what is the difference between $x$ and $X$?\n- I am having a hard time understanding the rationale behind eq (1): is this essentially representing an ensemble of flows, each of which comes with a different number of transformations/diffeomorphisms? How are these diffeomorphisms constructed in the first place?\n- step 3 in Sec. 2.2: is the second term in the min a second order partial derivative or what does this notation represent?\n- Sec 2.3: eq (2): Isn't this the same as a NF with multiple couplings? Could the authors clearly comment on differences and analogies between traditional normalizing flows and IRFs-based flows?\n- can the author comment on what they mean by $\\mu-a.s.$ and $\\pi-a.e.$?\n- In equation at bottom of page 3: this seems to be just the same as the average of normalizing Flow with increasing number of coupling layers, where $f_\\theta_i$ represents a coupling layer with weights $\\theta_i$. Can the authors comment on this intuition?\n- Eq. 6 is reference on line 124 and later but never appears. \n- I am confused by first equation in Sec. 3.2: Shouldn't $f_\\theta$ act on $x$? How come it acts on the density $q_\\theta$ in that equation? (Please reference all equations with numbers so that it is easier to identify them).\n- First equation below line 151: I find the notation hard to parse and potentially not entirely consistent with the notation before. Shouldn't there be two sums in there? one running over t and one running over m?\n- Lines 157/158: Do the authors there mean to average over different M flows all with fixed depth T? Does T have to be the same for every flow?\n- Sec. 3.4 **relaxing ergodicity**: up to this point in the paper it is still very unclear to me where the MCMC involutive kernel contributes there. I believe it would be crucial for the authors to carefully revise the paper to make this concept emerging much clearer since the earlier sections of the manuscript. \n- line 179: with using flow density and not $q_{\\theta/T}$ here?\n- The confusion stemming from section 3.4 still applies to me in Sec. 4. Can the authors try to re-elaborate in different words on the crucial differences between their approach and standard NFs method? I believe a related work section and also a more focused section on differences/analogies between their deterministic flows and NFs would be crucial to enhance clarity. \n- Line 228-229: I am unsure if I get this statement. What makes an approach *reliably invertible*? To my understanding, only RWMH IRF seems to be reliably invertible, while all the other seems to quickly explode in errors after 100 steps. \n- Line 243-244: Why not start from a standard normal distribution?\n- Fig 4 is extremely hard to parse. Moreover, in some plots, it looks as if some approaches do not even appear in the plots. I wonder if this is a plotting issue or if the training was unsuccessful. If that is the case, I believe a more thorough discussion of these plots is needed. Perhaps it might be a good idea to carefully select a subset of these plots for the main text and thoroughly focus on those.\n\nLIMITATIONS:\nAs far as I can tell, there's no section dedicated to the **Limitations** of the method, neither in the paper nor in the appendix.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce a IRF-based method for asymptotically exact variational inference that exempts the ergodicity assumptions from prior approaches. Towards this objective, they develop an involutive (self-invertible) MCMC kernel that parameterizes a measure-preserving IRF, which is shown to be invertible (Section 4) and asymptotically exact (Proposition 4.1). An ensemble algorithm with potentially better convergence rates is also proposed (Section 3.3). \n\nIn summary, the paper is well organized and nicely written, though somewhat dense. The contributions are clear and stated in a mathematically precise manner. However, the empirical analysis could be more comprehensive and rigorously executed.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths \n\n1. The paper is theoretically sound and rigorously built upon the well-founded ergodic theory.\n\n2. The work’s contributions and the limitations of its predecessors are clearly laid out and revisited throughout the paper. \n\n3. Empirical results suggest that the proposed approach converges faster than traditional flow-based methods - while being asymptotically exact. \n\n4. Computer code has been released for improved reproducibility. \n\n5. Proofs for all statements and a simpler proof for the asymptotic exactness of MixFlows have been included in the appendix.\n\n6. The proposed algorithm is, to the best of my knowledge, the first asymptotically exact variational method that doesn’t explicitly require ergodicity. \n\n## Weaknesses\n\nMy concerns are mostly concentrated on the paper’s limited empirical analysis and on isolated claims. I will be happy to increase my score if the issues below are properly addressed.\n\n1. Section 4 states that, “without loss of generality”, the IRF construction is described for a one-dimensional target distribution. It is unclear for me how such a construction can be straightforwardly generalized to higher dimensions. Could the authors elaborate on this extension?\n\n2. Sections 5.1 and 5.2 report the results for the learning rate minimizing the TV distance and maximizing the ELBO, respectively. Although I acknowledge that this promotes a fair assessment of the proposed method, one is often interested in how the algorithm’s sensibility to hyperparametric changes. Authors should consider reporting the average and standard deviation of the selected metrics across runs. \n\n3. Contrarily to NUTS, RealNVP, and NSF, the IRF variants seem to require a warm-up phase to appropriately define a step size for the RWMH kernel. It is unclear, however, whether this initial step is incorporated into the computational cost analysis in Figure 4(d). Also, I could not understand whether this hyperparameter is the same for every experiment or was independently chosen for each target distribution. Authors should clarify the cost of IRF tuning. \n\n4. Synthetic experiments are based on 2-dimensional distributions that can be easily plotted. Could the authors show the resulting samples for each of the considered methods?\n\nQUESTIONS:\nSee weaknesses. \n\nAlso, traditionally, variational methods are characterized by finding the best tractable approximation to an intractable target distribution via gradient-based search. IRF-based approaches, however, take a different path. Could the authors discuss the connection between the proposed methodology and traditional VI?\n\nLIMITATIONS:\nLimitations are properly addressed and future directions, e.g., designing automatically tuned and asymptotically exact variational methods, are concretely described.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a variational family for performing variational inference in Bayesian models, designed to have the theoretical convergence guarantees of MCMC. The key idea is to construct the variational family by generalising mix variational flows, which use Markov kernels to approximate the target distribution. The authors focus on involutive kernels and, to improve expressiveness and theoretical guarantees, the paper composes multiple involutive kernels using iterated random functions (IRFs), making random mix variational flows. These random mix flows relax the ergodic condition for convergence of the flow. Moreover, the method uses the backward process of the IRFs for training, arguing that this reduces computational cost. They further enhance their approach by proposing an ensemble of mix variational flows based on IRFs, and they provide theoretical results suggesting this ensemble can accelerate convergence. They introduce invertible versions of the kernel compositions and evaluate their performance on both synthetic and real datasets.\n\nSTRENGTHS AND WEAKNESSES:\nQuality:\nThe paper is technically sound and builds on a foundation of existing theory, the proposal of using involutive kernels and their composition through IRFs is well motivated for the task of building variational families with weaker conditions for convergence. The practical algorithm presented for invertible measure-preserving IRFs makes the method applicable beyond theory. However, the work feels somewhat incomplete in places, particularly around the justification for using the backward IRF process, which is claimed to reduce computational cost but lacks convincing explanation or empirical evidence. The experiments do not fully explore the implications or variations of the proposed method, especially regarding architectural choices and computational costs. The authors acknowledge some limitations but could be more thorough in critically evaluating different design choices.\n\nClarity:\nThe paper is generally well organised and clearly written. However, the rationale and benefits of using the backward process of IRFs instead of the forward require better explanations. The limited discussion on implementation details, such as architecture choices and computational trade-offs for ensemble methods, reduces clarity for readers looking to apply the approach or reproduce results.\n\nSignificance:\nBy linking MCMC kernels, IRFs, and variational inference, the paper opens a promising direction for constructing flexible yet theoretically grounded approximations. The potential impact lies in enabling practitioners and researchers to build variational approximations that converge in total variation to the true posterior, addressing a known limitation of many variational approaches. The open questions regarding computational efficiency, backward vs. forward IRF usage, and ensemble trade-offs leave open how impactful the method will be in practice.\n\nOriginality:\nWhile the paper largely combines and extends several existing ideas, it does so in a novel and meaningful way. The framing of variational families as compositions of involutive kernels via IRFs, particularly using the backward process, is an original contribution that advances understanding of how to construct variational approximations. The ensemble approach to accelerate convergence also adds novelty.\n\nQUESTIONS:\nCan you explain exactly how is it that the inverse of the backward IRF MixFlow has lower computational cost than the inverse of the forward IRF MixFlow?\n\nHow do the data benchmarks and computational cost of experiments compare to MALA or RWMH instead of NUTS? Maybe the computational cost figures would be clearer without the baseline. Maybe also present wall clock time?\n\nWhat is the computing cluster used for experiments? is it a cluster of CPUs or GPUs? Using GPUs would parallelise many of the gradient and density evaluations of the computing cost. It would be interesting to know the wall clock time of training and inference of your method in comparison with normalizing flows when using parallel computing.\n\nWhat are the computational costs of sample generation? especially comparing random mix flows with normalizing flows, or MCMC.\n\nLIMITATIONS:\nThere is no discussion on the limitations of the method. More about the computational costs, which are clearly expensive for an ensemble method but at what gain in expressiveness/convergence? and the limitations on architecture design and how this affects the outcome.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper extends the recently introduced class of Mixflows algorithms - to construct variational approximations via ergodic measure preserving transformations - with non-deterministic maps. This allows for some results in terms of total variation convergence to the target distribution. They compare against normalizing flows and MCMC on estimating log normalization constants and certain posterior moments respectively.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: the paper provides a nice continuation of the Mixflows research line with a nontrivial and interesting extension of this class of algorithms. The paper is largely clearly written, and the mathematical notation is generally precise. Experiments touch realistic posterior distributions and carefully take into account total computation cost (for instance, of training the neural networks in normalizing flows). \n\nWeaknesses: Re-emphasizing this is a good paper, there can always be some reservations. Some parts remain partially unclear, although this may be a limitation of my understanding of iterated random function theory and/or Markov chain theory. In 3.2, is the simulation of the backward process required ? Reference [34] is cited, but it seems overkill to have to check it. Section 4 is also quite dense and hard to follow. Further, concrete examples of four MCMC-based IRFs arrive only in page 7. I recommend concrete instances to come earlier in the presentation of the IRF framework, and describe in a concrete way how they can be (approximately) inverted. \nFurther, the comment starting line 230: \"Notably, the corrected HMC IRF is consistently more stable than its uncorrected counterpart used in past MixFlows work, because the additional MH accept reject step discards trajectories with large numerical error that would otherwise cause the dynamics to diverge.\" seems a bit directly at odds with the comment immediately after in line 235. Perhaps the authors can explain better here. Finally, the assumption in Theorem 3.3 of bounded (reverse) importance ratio with respect to the initial distribution seems strong, although I may be wrong. Could you comment in general about the importance of the choice of initial distribution ? (For instance, in SMC it matters quite a bit).\n\nQUESTIONS:\nAdditional comments/questions/suggestions for the authors:\n\n- MixFlows and their extension here seem (to me, at least) clearly closely connected to adaptive importance sampling - which has a relatively well-developed literature - where exact density evaluations are provided and final estimators are often able to use all generated samples during the learning process. It would be good to comment on any connections that the authors see or do not see. Adaptive IS literature dates back to at least Pugh (1966), with other common references being Oh et al (1992), Bugallo et al (2017), Portier et al (2018). Of particular relevance here is that there exist adaptive IS algorithms where the proposal learning is driven by MCMC (Martino et al (2017)). Perhaps the authors could expand as it is sometimes possible to obtain central limit theorems for AIS estimates (see Portier et al (2018)), in contrast to estimates with MixFlows it seems. \n- As it seems that all generated samples can be used in the final estimation here, it would be interesting to explicitly write down the Z (or log Z) estimate with importance sampling \n- An empirical comparison with tempered SMC with HMC/NUTS kernels, seen by some as a very good reference for Z estimation, may strengthen further the relevance of the proposed approach ( default implementations exploiting GPUs are available in modern Python libraries, such as BlackJax), and commenting the advantages of the proposed approach (for example, not having to choose a sequence of distributions as in SMC). It may also be worth noting that SMC has been viewed also an adaptive importance sampling algorithm with intermediate target distributions. \n- Aside of / instead of Theorem 2.2, I would have expected a theorem for the convergence of the normalizing constant estimates Z (or log Z thereof). After all $q_T$ for any $T$ does provide exact density evaluation, therefore any valid Z estimate would be naturally based on importance sampling, and this is what is done in the experiments as I understand. Further, the variational family would provide consistent estimates of expectations of integrable test functions even for finite $T$ when IS is used. \n- In practice, is it easier to tune the MCMC kernels in your method than it is to tune the hyperaparameters of the normalizing flows training ? As in a real world scenario, that (difficult to quantify) cost may be also taken into account. \n- Using the ELBO metric seems somewhat redundant (but perhaps the authors can explain) with estimation of log Z with importance sampling - as clearly the ELBO is intractable and has to be MC estimated anyway, so one may as well just refer to the estimation of log Z (which is of course a biased but consistent estimator the true log Z, with an MSE relating also to KL between target and proposal). \n- Is Theorem 3.1. a convergence in distribution ? Please state it clearly.  \n\n\nReferences: \n- Pugh EL. A gradient technique of adaptive Monte Carlo. SIAM Review. 1966 Jul;8(3):346-55.\n- Oh, M.S. and Berger, J.O., 1992. Adaptive importance sampling in Monte Carlo integration. Journal of statistical computation and simulation, 41(3-4), pp.143-168.\n- Bugallo, M.F., Elvira, V., Martino, L., Luengo, D., Miguez, J. and Djuric, P.M., 2017. Adaptive importance sampling: The past, the present, and the future. IEEE Signal Processing Magazine, 34(4), pp.60-79. \n- Portier, F. and Delyon, B., 2018. Asymptotic optimality of adaptive importance sampling. Advances in neural information processing systems, 31.\n- Martino, L., Elvira, V., Luengo, D. and Corander, J., 2017. Layered adaptive importance sampling. Statistics and Computing, 27, pp.599-623.\n\nLIMITATIONS:\nThe authors have a fairly honest presentation and discuss limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}