ICML 2025

Continuous Semi-Implicit Models
Longlin Yu∗1 Jiajun Zha∗2 Tong Yang 3 4 Tianyu Xie 1 Xiangyu Zhang 4 S.-H. Gary Chan 2 Cheng Zhang 1 5
Abstract
Semi-implicit distributions have shown great
promise in variational inference and generative
modeling.
Hierarchical semi-implicit models,
which stack multiple semi-implicit layers, en-
hance the expressiveness of semi-implicit distri-
butions and can be used to accelerate diffusion
models given pretrained score networks. How-
ever, their sequential training often suffers from
slow convergence. In this paper, we introduce
CoSIM, a continuous semi-implicit model that
extends hierarchical semi-implicit models into a
continuous framework. By incorporating a contin-
uous transition kernel, CoSIM enables efficient,
simulation-free training. Furthermore, we show
that CoSIM achieves consistency with a care-
fully designed transition kernel, offering a novel
approach for multistep distillation of generative
models at the distributional level. Extensive ex-
periments on image generation demonstrate that
CoSIM performs on par or better than existing
diffusion model acceleration methods, achieving
superior performance on FD-DINOv2.
1. Introduction
Semi-implicit distributions, which are constructed through
the convolution of explicit conditional distributions and im-
plicit mixing distributions, have gained significant traction
in variational inference and generative modeling. Unlike
traditional approximating distributions with explicit density
forms, semi-implicit distributions enable a more expressive
family of variational posteriors, leading to improved approx-
*Equal contribution. This work was done during an internship
at StepFun.
1School of Mathematical Sciences, Peking University,
Beijing, China 2Department of Computer Science and Engineering,
Hong Kong University of Science and Technology 3School of
Computer Science, Fudan University, Shanghai, China 4Megvii
Technology Inc., Beijing, China 5Center for Statistical Science,
Peking ...