NeurIPS 2025

A Black-Box Debiasing Framework for Conditional
Sampling
Han Cui
University of Illinois at Urbana-Champaign
Champaign, IL
hancui5@illinois.edu
Jingbo Liu
University of Illinois at Urbana-Champaign
Champaign, IL
jingbol@illinois.edu
Abstract
Conditional sampling is a fundamental task in Bayesian statistics and genera-
tive modeling. Consider the problem of sampling from the posterior distribution
PX|Y =y∗for some observation y∗, where the likelihood PY |X is known, and we
are given n i.i.d. samples D = {Xi}n
i=1 drawn from an unknown prior distribution
πX. Suppose that f(ˆπXn) is the distribution of a posterior sample generated by
an algorithm (e.g. a conditional generative model or the Bayes rule) when ˆπXn
is the empirical distribution of the training data. Although averaging over the
randomness of the training data D, we have ED (ˆπXn) = πX, we do not have
ED {f(ˆπXn)} = f(πX) due to the nonlinearity of f, leading to a bias. In this
paper we propose a black-box debiasing scheme that improves the accuracy of
such a naive plug-in approach. For any integer k and under boundedness of the
likelihood and smoothness of f, we generate samples ˆX(1), . . . , ˆX(k) and weights
w1, . . . , wk such that Pk
i=1 wiP ˆ
X(i) is a k-th order approximation of f(πX), where
the generation process treats f as a black-box. Our generation process achieves
higher accuracy when averaged over the randomness of the training data, with-
out degrading the variance, which can be interpreted as improving memorization
without compromising generalization in generative models.
1
Introduction
Conditional sampling is a major task in Bayesian statistics and generative modeling. Given an
observation y∗, the objective is to draw samples from the posterior distribution PX|Y =y∗, where the
likelihood PY |X is known but the prior distribution πX is unknown. Instead, we are provided with a
dataset D = {Xi}n
i=1 consisting of n i.i.d. samples drawn from πX.
The setting is common in a wide range of applicati...