ICML 2025

Variational Learning of Fractional Posteriors
Kian Ming A. Chai 1 Edwin V. Bonilla 2
Abstract
We introduce a novel one-parameter variational
objective that lower bounds the data evidence and
enables the estimation of approximate fractional
posteriors. We extend this framework to hierar-
chical construction and Bayes posteriors, offering
a versatile tool for probabilistic modelling. We
demonstrate two cases where gradients can be ob-
tained analytically and a simulation study on mix-
ture models showing that our fractional posteriors
can be used to achieve better calibration compared
to posteriors from the conventional variational
bound. When applied to variational autoencoders
(VAEs), our approach attains higher evidence
bounds and enables learning of high-performing
approximate Bayes posteriors jointly with frac-
tional posteriors. We show that VAEs trained with
fractional posteriors produce decoders that are
better aligned for generation from the prior.
1. Introduction
Exact Bayesian inference is intractable for most models of
interest in machine learning. Variational methods (Jordan
et al., 1999; Minka, 2001; Opper & Winther, 2005; Blei
et al., 2017) address this by casting the required integration
as optimisation. These methods have two objectives: to
estimate the marginal likelihood or data evidence (MacKay,
2003) for model comparison or model optimisation; and to
obtain an approximate Bayes posterior for prediction.
The widely used evidence lower bound (ELBO) (Jordan
et al., 1999) often leads to underestimated uncertainty
and suboptimal posterior calibration (Wang & Tittering-
ton, 2005; Bishop, 2006; Yao et al., 2018). These deficien-
cies can be compounded by challenges inherent to general
Bayesian modelling, such as misspecification. Fractional
1DSO
National
Laboratories,
Singapore
2CSIRO’s
Data61,
Australia.
Correspondence to:
Kian Ming A.
Chai
<ckianmin@dso.org.sg>,
Edwin
V.
Bonilla
<ed-
win.bonilla@data61.csiro.au>.
Proceedings of the 42 nd International...